Question

I have a queue into which multiple messages, all independent and which can be processed in parallel.

I have seen a mutlicast router which takes the same in message and splits it across multiple receivers and I think I have also tried a few others like throttle inflight route policy, thread pool profile etc.

However, what I did want was for a single route to encapsulate multiple parallel JMS sessions, the total which I hope to configure and have the messages processed all at once.

I would like to state an assumption of mine, that a single route with a from would mean a single session and not n parallel sessions. Please correct me if I am wrong.

My camel context looks a bit like this:

<bean id="throttlePolicy" class="org.apache.camel.impl.ThrottlingInflightRoutePolicy">
    <!-- define the scope to be context scoped so we measure against total 
        inflight exchanges that means for both route1, route2 and route3 all together -->
    <property name="scope" value="Context" />
    <!-- when we hit > 20 inflight exchanges then kick in and suspend the routes -->
    <property name="maxInflightExchanges" value="20" />
    <!-- when we hit lower than 10% of the max = 2 then kick in and resume 
        the routes the default percentage is 70% but in this demo we want a low value -->
    <property name="resumePercentOfMax" value="10" />
    <!-- output throttling activity at WARN level -->
    <property name="loggingLevel" value="WARN" />
</bean>

<camelContext id="camel" errorHandlerRef="dlc"
    xmlns="http://camel.apache.org/schema/spring">

    <!-- this is the Dead Letter Channel error handler, where we send failed 
        message to a log endpoint -->
    <errorHandler id="dlc" type="DeadLetterChannel"
        deadLetterUri="jms:deadLetterQueue">
        <redeliveryPolicy retryAttemptedLogLevel="INFO"
            maximumRedeliveries="3" redeliveryDelay="250" backOffMultiplier="2"
            useExponentialBackOff="true" />
    </errorHandler>

    <route routePolicyRef="throttlePolicy">
        <from uri="jms:camel.design.md5InputQueue" />
        <transacted ref="required" />
        <process ref="basicProcessor" />
        <to uri="jms:camel.integrationTest.reply" />
    </route>
</camelContext>

As you can see, what I am doing is calculating the MD5 of a source. I was hoping to do this and transfer the result into a reply queue and have all those done in parallel.

To simulate this, I put a sleep (one second) in the basic processor and what I am seeing is a sequential processing of a message one after another rather than parallel processing. For example, if there are 10 messages it takes 10 seconds, if there are 20 messages, it takes 20 seconds etc.

How do I get this to work in parallel and have all the MD5 calculations, say 10 input messages done in about 2 seconds even after putting a sleep condition within the processor.

Answer 1

just set the maxConcurrentConsumers property to enable multiple consumer threads from your queue…

<route routePolicyRef="throttlePolicy">
    <from uri="jms:camel.design.md5InputQueue?maxConcurrentConsumers=5" />
    <transacted ref="required" />
    <process ref="basicProcessor" />
    <to uri="jms:camel.integrationTest.reply" />
</route>