I am building a Java EE application that includes functionality to process AMQP “topics”. My idea is to create a Listener class that is initialized via the contextInitialized method of javax.servlet.ServletContextEvent.
This class would run two separate threads; one would listen for messages and add the received messages to a FIFO queue and the second would process the messages from the queue. The reason for this is that I don’t what the AMQP client to be impacted by delays in processing the messages.
In a complex application that has multiple “Listeners”, I could end up with a large number of threads running asynchronous tasks. I am not experienced enough to know from an architectural perspective if breaking an application up into multiple threads in this way is a reasonable approach?
Maybe each message should be processed in its “own” thread rather than by a queue process. Any advice or guidance regarding either the use of multiple threads to manage application flow or the approach that I have proposed would be much appreciated.
Running stuff in separate threads does not necessarily make them go faster.
The throughput of a system running on a single machine is limited by the processing bandwidth of the machine; i.e. number and speed of cores, the memory system, disc and network I/O and so on. In a multi-threaded application, the threads all effectively share the resources. So for instance, if you have more runnable threads than cores at a given instant, some of the threads will be waiting to be scheduled to a code.
The second issue is that threads typically need to communicate with each other and / or update shared data structures. Both of these entail some kind of synchronization. If you have a lot of this going on, there is a potential for the synchronization to become a bottleneck that reduces throughput.
So how does this apply to your system? Well, the potential problem is that the extra threads that are doing the background processing are going to use resources, and if there is too much work for the threads:
– they may not be able to keep up, and the queue lengths could get out of control, resulting in long delays and worse, and
– this could interfere with the listeners’ ability to accept new messages.
From a performance perspective, one thing you want to avoid is having too many threads. Beyond a certain point (which depends on the application), adding more threads can actually reduce throughput for a variety of reasons. As a rule of thumb, try to limit the number threads to 1 to 2 times the number of cores.
If you think that it is likely that your system will be swamped with more messages that it can handle, it needs to be designed so that it can shed load; e.g. by stopping accepting new requests, or dumping existing ones. You do not want unbounded queues or an unbounded numbers of worker threads, and these can lead to catastrophic feedback and systems collapsing / crashing under heavy load. (And also be aware that heavy load is going to lead to more contention and increase the chance of failures due to undetected concurrency bugs.)
Updates: