Question

I am using Spring3.1 on a standalone env.

(this problem not necessary related to Spring. it’s behaving the same also on a standalone env.)

I have implements a listener which receives messages from Topic. the messages rate is very very high (talking about 20/30 m/s).

some messages can take more processing time then other.

The listener works with the same instance which means that if one message being processed too long it hits our performances pretty much.

We thought about doing our own pool of objects instead of using the same listener instance but then I found out the Executors(java.util.concurrent.Executors).

So for each message received a different thread will be allocated to it. this will make sure our listener instance will be free to process messages parallel.

private ExecutorService  threadPool = Executors.newFixedThreadPool(100);
    @Override
    public void onMessage(final Message msg)
    {
        Runnable t = new Runnable()
        {
            public void run()
            {
                onSessionMessage(msg);
                log.trace("AbstractSessionBean, received messge");
            }
        };
        threadPool.execute(t);
    }

That seems to solve our performance issue. but after monitoring the application with jconsole we facing now huge memory leaks.

The heap memory usage being increased significantly in time.

So I tried to “play” a bit with the FixedThreadPool size number. still having huge memory usage:

Any idea how can I solve this? any other ideas to solve my key problem?

After running heap dump I got two problem suspects:

thanks,
ray.

Answer 1

I think you have no problem with memory leak, I cannot see it from your jconsole chart at least. There is no major GC collection. So it seems only more and more objet allocated into tenured(old) generation. To ensure about memory leak you should Perform GC and after that compare allocated memory. If you find a leak you are able to make a heap dump with jmap or visual tool(standart JDK tools). After this heap dump can be analyzed with MAT. Before taking heap dump it is better to Perform GC to decrease heap dump file size.

Some notes:

• Threads count shouldn’t affect heap memory explicitly. It maybe useful for you to review next java memory structure. So thread require stack memory not a heap.
• In general it is not good idea to create cache for not heavy object, because of GC works algorithm.
• Also I think you should consider cached thread pool or calibrate ThreadPoolSize according server hardware.