Possible memory leak in Wicket 1.5.12

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Possible memory leak in Wicket 1.5.12

Satrix
Hello,

I've developed an Wicket website in Wicket 1.5.12. It's hosted on Glassfish server 3.1.2.2 and lately I've run into the following problem:

The problem only occurs when a crawler (for example googlebot) visits my website and then when it's crawling, Glassfish opens a lot of file descriptors (for images) and doesn't close it. This leads to "Too many open files" exception and right after that my server crashes. Those file descriptors are open all the time - I've to restart the server to delete the file descriptors.

So I wanted to ask if it's possible that it's a wicket related problem or it's just Glassfish problem ?

Thank you for any help.
Reply | Threaded
Open this post in threaded view
|

Re: Possible memory leak in Wicket 1.5.12

Francois Meillet
Could you show us the code you use to serve these images ?

François Meillet





Le 19 déc. 2014 à 17:02, Satrix <[hidden email]> a écrit :

> Hello,
>
> I've developed an Wicket website in Wicket 1.5.12. It's hosted on Glassfish
> server 3.1.2.2 and lately I've run into the following problem:
>
> The problem only occurs when a crawler (for example googlebot) visits my
> website and then when it's crawling, Glassfish opens a lot of file
> descriptors (for images) and doesn't close it. This leads to "Too many open
> files" exception and right after that my server crashes. Those file
> descriptors are open all the time - I've to restart the server to delete the
> file descriptors.
>
> So I wanted to ask if it's possible that it's a wicket related problem or
> it's just Glassfish problem ?
>
> Thank you for any help.
>
> --
> View this message in context: http://apache-wicket.1842946.n4.nabble.com/Possible-memory-leak-in-Wicket-1-5-12-tp4668856.html
> Sent from the Users forum mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

Reply | Threaded
Open this post in threaded view
|

Re: Possible memory leak in Wicket 1.5.12

Martin Grigorov-4
In reply to this post by Satrix
Hi,

If you are able to reproduce this in a quickstart application then please
create a ticket and attach it there.
Thanks!

Martin Grigorov
Wicket Training and Consulting
https://twitter.com/mtgrigorov

On Fri, Dec 19, 2014 at 6:02 PM, Satrix <[hidden email]> wrote:

>
> Hello,
>
> I've developed an Wicket website in Wicket 1.5.12. It's hosted on Glassfish
> server 3.1.2.2 and lately I've run into the following problem:
>
> The problem only occurs when a crawler (for example googlebot) visits my
> website and then when it's crawling, Glassfish opens a lot of file
> descriptors (for images) and doesn't close it. This leads to "Too many open
> files" exception and right after that my server crashes. Those file
> descriptors are open all the time - I've to restart the server to delete
> the
> file descriptors.
>
> So I wanted to ask if it's possible that it's a wicket related problem or
> it's just Glassfish problem ?
>
> Thank you for any help.
>
> --
> View this message in context:
> http://apache-wicket.1842946.n4.nabble.com/Possible-memory-leak-in-Wicket-1-5-12-tp4668856.html
> Sent from the Users forum mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Possible memory leak in Wicket 1.5.12

Satrix
Hi,

It would be quite hard to reproduce the problem in quickstart application. Here is the code of DynamicImageResource:

@Override
    protected byte[] getImageData(IResource.Attributes attributes) {
        try {
            return IOUtils.toByteArray(stream);
        } catch (IOException ex) {
            LOG.error("Could not return byte[] from " + stream, ex);
           
            return null;
        } finally {
            if(stream != null){
                try {                              
                    stream.close();                    
                } catch (IOException ex) {
                    LOG.error("Could not close stream", ex);
                }
            }
        }
    }

And then I use the NonCachingImage in conjunction with my DynamicImageResource implementation class.
Reply | Threaded
Open this post in threaded view
|

Re: Possible memory leak in Wicket 1.5.12

Francois Meillet
Can you show up the entire class ?


François Meillet
Formation Wicket - Développement Wicket





Le 20 déc. 2014 à 09:56, Satrix <[hidden email]> a écrit :

> Hi,
>
> It would be quite hard to reproduce the problem in quickstart application.
> Here is the code of DynamicImageResource:
>
> @Override
>    protected byte[] getImageData(IResource.Attributes attributes) {
>        try {
>            return IOUtils.toByteArray(stream);
>        } catch (IOException ex) {
>            LOG.error("Could not return byte[] from " + stream, ex);
>
>            return null;
>        } finally {
>            if(stream != null){
>                try {                              
>                    stream.close();                    
>                } catch (IOException ex) {
>                    LOG.error("Could not close stream", ex);
>                }
>            }
>        }
>    }
>
> And then I use the NonCachingImage in conjunction with my
> DynamicImageResource implementation class.
>
> --
> View this message in context: http://apache-wicket.1842946.n4.nabble.com/Possible-memory-leak-in-Wicket-1-5-12-tp4668856p4668863.html
> Sent from the Users forum mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

Reply | Threaded
Open this post in threaded view
|

Re: Possible memory leak in Wicket 1.5.12

Satrix
Yeah, sure. Here is the code:

public class LocalFileResource extends DynamicImageResource {
    private final Logger LOG = Logger.getLogger(LocalFileResource.class);
   
    private String filePath;
    private InputStream stream;

    public LocalFileResource(String filePath) throws FileNotFoundException {
        this.filePath = filePath;
       
        File file = new File(filePath);
        if(!file.exists()){
            throw new FileNotFoundException("Could not load file " + filePath);
        }
       
        try {            
            stream = new FileInputStream(file);
        } catch (FileNotFoundException ex) {
            throw new FileNotFoundException("Could not obtain stream from " + filePath);
        }
    }        
   
    public LocalFileResource(InputStream stream) {
        if(stream == null){
            throw new IllegalArgumentException("Stream cannot be null");
        }
       
        this.stream = stream;
    }
   
    @Override
    protected byte[] getImageData(IResource.Attributes attributes) {
        try {
            return IOUtils.toByteArray(stream);
        } catch (IOException ex) {
            LOG.error("Could not return byte[] from " + stream);
           
            return null;
        } finally {
            if(stream != null){
                try {                              
                    stream.close();                    
                } catch (IOException ex) {
                    LOG.error("Could not close stream: " + ex.getMessage());
                }
            }
        }
    }        
}
Reply | Threaded
Open this post in threaded view
|

Re: Possible memory leak in Wicket 1.5.12

Francois Meillet
Hi Satrix,

When the second constructor is used, with the inputsream argument,
the stream is open.

When this inputsream is opened ?


François Meillet





Le 20 déc. 2014 à 10:03, Satrix <[hidden email]> a écrit :

> Yeah, sure. Here is the code:
>
> public class LocalFileResource extends DynamicImageResource {
>    private final Logger LOG = Logger.getLogger(LocalFileResource.class);
>
>    private String filePath;
>    private InputStream stream;
>
>    public LocalFileResource(String filePath) throws FileNotFoundException {
>        this.filePath = filePath;
>
>        File file = new File(filePath);
>        if(!file.exists()){
>            throw new FileNotFoundException("Could not load file " +
> filePath);
>        }
>
>        try {            
>            stream = new FileInputStream(file);
>        } catch (FileNotFoundException ex) {
>            throw new FileNotFoundException("Could not obtain stream from "
> + filePath);
>        }
>    }        
>
>    public LocalFileResource(InputStream stream) {
>        if(stream == null){
>            throw new IllegalArgumentException("Stream cannot be null");
>        }
>
>        this.stream = stream;
>    }
>
>    @Override
>    protected byte[] getImageData(IResource.Attributes attributes) {
>        try {
>            return IOUtils.toByteArray(stream);
>        } catch (IOException ex) {
>            LOG.error("Could not return byte[] from " + stream);
>
>            return null;
>        } finally {
>            if(stream != null){
>                try {                              
>                    stream.close();                    
>                } catch (IOException ex) {
>                    LOG.error("Could not close stream: " + ex.getMessage());
>                }
>            }
>        }
>    }        
> }
>
>
> --
> View this message in context: http://apache-wicket.1842946.n4.nabble.com/Possible-memory-leak-in-Wicket-1-5-12-tp4668856p4668865.html
> Sent from the Users forum mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

Reply | Threaded
Open this post in threaded view
|

Re: Possible memory leak in Wicket 1.5.12

Satrix
Hi,

It's used very rarely so it should not be a problem. However I've moved the stream creation from the constructor to getImageData method.

The important point here - it only happens when the crawler visits my website. When normal users visit my website the file descriptors are closed successfully.
Reply | Threaded
Open this post in threaded view
|

Re: Possible memory leak in Wicket 1.5.12

Francois Meillet
Can you show me the class when you open the stream ?

Did you moved the stream creation to the getImageData this morning ?

François Meillet
Formation Wicket - Développement Wicket





Le 20 déc. 2014 à 10:58, Satrix <[hidden email]> a écrit :

> Hi,
>
> It's used very rarely so it should not be a problem. However I've moved the
> stream creation from the constructor to getImageData method.
>
> The important point here - it only happens when the crawler visits my
> website. When normal users visit my website the file descriptors are closed
> successfully.
>
> --
> View this message in context: http://apache-wicket.1842946.n4.nabble.com/Possible-memory-leak-in-Wicket-1-5-12-tp4668856p4668868.html
> Sent from the Users forum mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

Reply | Threaded
Open this post in threaded view
|

Re: Possible memory leak in Wicket 1.5.12

Satrix
Actually I've checked the code in the second constructor and the stream is the ByteArrayInputStream instance. But I've moved the stream creation from the first constructor to the getImageData because that's the constructor I use to create the 99% of the images in my website.

And yes I've changed it today.
Reply | Threaded
Open this post in threaded view
|

Re: Possible memory leak in Wicket 1.5.12

Francois Meillet
Moving the stream creation, opening and closing it in getImageData is a good thing.
You should also remove the second constructor (with the inputstream argument).

Now with your tests classes with the WicketTester, (you do have tests ;-)  ), you can play yours pages with a debugger and see what happens.

François Meillet
Formation Wicket - Développement Wicket





Le 20 déc. 2014 à 11:40, Satrix <[hidden email]> a écrit :

> Actually I've checked the code in the second constructor and the stream is
> the ByteArrayInputStream instance. But I've moved the stream creation from
> the first constructor to the getImageData because that's the constructor I
> use to create the 99% of the images in my website.
>
> And yes I've changed it today.
>
> --
> View this message in context: http://apache-wicket.1842946.n4.nabble.com/Possible-memory-leak-in-Wicket-1-5-12-tp4668856p4668870.html
> Sent from the Users forum mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

Reply | Threaded
Open this post in threaded view
|

Re: Possible memory leak in Wicket 1.5.12

Satrix
Looks like it's working now :)

Thank you Francois for your help :)
Reply | Threaded
Open this post in threaded view
|

Re: Possible memory leak in Wicket 1.5.12

Francois Meillet
Hi Satrix,

Make sure you remove the second constructor. (the one wih the inputstream argument)
And the stream could be a local property (in the getImageData method) and not an instance property.

François Meillet
Formation Wicket - Développement Wicket





Le 20 déc. 2014 à 14:26, Satrix <[hidden email]> a écrit :

> Looks like it's working now :)
>
> Thank you Francois for your help :)
>
> --
> View this message in context: http://apache-wicket.1842946.n4.nabble.com/Possible-memory-leak-in-Wicket-1-5-12-tp4668856p4668874.html
> Sent from the Users forum mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>