Question

Is there a way to consume messages from a WebSphere MQ queue in batches? e.g.

messages = queue.receiveBatch( BATCH_SIZE )

Where it would return a collection of messages ( e.g. List[Message] ) up to a BATCH_SIZE in a single “receive”?

( I can call receive BATCH_SIZE times within a transaction, but that is not what I am looking for )

If message groups can solve it, what would it look like? Although it does not seem to be the right solution as “message groups” seem to be reading messages one by one.

Answer 1

No. WMQ has read-ahead in which non-persistent messages are streamed to the client but even there the client executes a single GET (or language equivalent) per message.

One of the attributes of async messaging is supposed to be that messages are atomic. Any dependency of messages on other messages is called an affinity and message affinities break the async model. (However every transport makes some accommodation for affinities and message grouping is an example of that.) So a GET is the atomic unit of work which has a 1:1 correspondence to a message.

WMQ messages can be of arbitrary size up to 100MB and messages larger than that can be segmented where one logical message is split across multiple physical messages. In the case of a segmented message, one GET will result in consumption of multiple messages, however to the app only one logical message is returned.

WMQ Channels use batches and until recently these were constructed strictly by message count. However the aforementioned variability in message sizes made tuning channels extremely difficult if both large and small messages traversed the same channel. In v7.1 of WMQ channel batches now are bounded by a message count or a byte count, whichever is hit first. Any notion of message GET batching would require similar tuning functionality in order to prevent severe impacts to memory. There would be additional interactions with logging, queue sizes, etc. Note also that if one message failed, the entire batch would have to be rolled back.

If the intent is to improve performance, use transactions. When receiving messages under syncpoint you can tune the number of messages to GET between COMMIT calls. You can considerably improve performance with COMMIT intervals > 1, however this falls off above a threshold which you can determine with a bit of trial and error. The Performance Reports can help determine the best number for your platform. These are available from the SupportPacs main page. Look for the entries with names like MPxx.

If the messages are non-persistent and you can afford to lose some of them on disconnect, then you can use read-ahead – assuming you have a modern version of WMQ. This will stream messages to the application’s WMQ client so that the next message is already in the local buffer when GET is called.

By the way, the link in the original post was to the WMB docs. The WMQ docs can be found here.