Question

Which Java blocking queue is best for multiple producer and single or multiple consumers scenarios?

I am testing with LinkedBlockingQueue but I am getting OutOfMemoryError exception.

I am trying to achieve following things.

1. producer creates a object & put in a queue.
2. consumer grabs the data from queue & insert into database. There would be 400 producers and I can adjust consumers as I wish.

Let me know any idea.

Update

Producer : It should listen to Server Socket. It reads the data from socket & construct the object (Domain objects) and put in a queue.

Consumer : Take the object from queue & insert into DB (Hiberante & connection pooling supported)

This my real environment. Process should be able to process at least 200 records/sec.I am testing the scalability of process and how to improve that. I hope this will give better idea.

Helpful links :

vmoptions

Monitoring and Managing Java SE 6 Platform Applications

BlockingQueue

Answer 1

The problem is not which Queue implementation you use but rather how to approach the problem of throttling your producers if your consumers cannot keep up. One possible solution is to create a LinkedBlockingQueue with a fixed capacity, and have your producers call offer(E e), which will return false if the queue is full.

Another possible solution is to tailor the number of producers and consumers accordiingly.