Question

I have a production system that uses ActiveMQ (5.3.2) to send messages from server A to server B. A few weeks ago, the system inexplicably started taking 10+ second to send a message. After a reboot of the producer, the system worked fine.

After investigation, I’m pretty sure this is due to producer flow control. (I have a fairly standard activemq setup). The day before this happened (for other reasons) my consumer software had been acting erratically and had even stopped accepting connections for a while. So I’m guessing this triggered this. (It does puzzle me that the requests were still being throttled a day later).

Question — how can I confirm that the requests were being throttled. I took a heap dump of the server — is there data in memory I can look for?

Edit: I’ve found the following:
WireFormatNegotiator.tcpNoDelayEnabled=false for one of three WireFormatNegotiator instances in the memory. I’m trying to figure out what sets this.

And second (and more important), is there a way I can use JMX to tell if the messages are being throttled? I’d like to set up a Nagios alert to let me know if this happens in the future. What property should I check for with JMX?

Answer 1

you can configure your producer client to throw javax.jms.ResourceAllocationException exceptions which can then be detected/logged, etc. just set one of the following…

<systemUsage>
   <systemUsage sendFailIfNoSpaceAfterTimeout="3000">
   ...OR...   
   <systemUsage sendFailIfNoSpace="true">