What is the recommended architecture for accessing WebSphere MQ message queues across internet (i.e. 100+ ms latency) and over organizational boundaries?
The two approaches that we are considering are to access the other organization’s Queue Manager directly from our clients and the alternative is to have a Queue manager locally that would pump the messages from the remote queue and then local clients will access it. I think that both have merit but I am not sure of the trade-offs between the two architectures.
The volume that we would have to handle is 600 per second with message size of about 50 bytes. The other org’s queue manager is not changeable (and it is WebSphere MQ). The messages have to be processed in order. Perhaps they can be split between different queues and then each queue to be processed by separate client but in each queue the order is still very important. In general there would be one transaction processing client. There could be one additional business intelligence client that would process a copy of the message.
Does anyone have any perf metrics of MQSeries to MQSeries queue manager throughput and a comparison of WebSphere MQ queue manager to client throughput?
The recommended answer from a security standpoint is that the other org should be requiring you to use a full QMgr and not a client. A channel connection from an external QMgr will only ever issue
CONNECT,INQUIREandPUTcommands. A client connection has access to the entire WMQ API and can execute any API call on any object. For example, if the other party uses structured data in their queue names (an account number for example) a client app can cycle through all possible names to enumerate all the account numbers. If the call returns 2035, the object exists but authorization failed. If the call returns 2085 the object does not exist. In addition to allowing various types of enumeration, a client that gets stuck in a connect loop can throw hundreds of reconnect attempts per second at a QMgr that will completely tie up a listener. So clients are inherently more dangerous to accept connections from on a QMgr and clients from 3rd parties even more so. However, clients are free and the cost often outweighs the risk, especially when the application is not moving high-value transactions or sensitive data. If I were charged with connecting to a vendor’s QMgr and they allowed the choice of client connections or QMgr connections and the application was high-visibility or mission critical, I’d choose a full QMgr.Another aspect to consider is that the QMgr-to-QMgr channels are more resilient to network connectivity problems. This is because two QMgrs keep track of message sequence numbers, hold batches under syncpoint until acknowledged and are capable of negotiating channel recovery automatically without losing or duplicating any persistent messages. Because a client channel gives you as the developer full access to the API, including the freedom to write programs that sacrifice reliability for performance, it is possible to write a client app that loses or duplicates messages. In fact, an inherent problem with async message over a network is that session recovery issues can create ambiguous outcomes that lead to duplicate messages. This is not specific to WebSphere MQ and in fact, the JMS specification talks about this issue and notes that it is the application’s responsibility to properly account for “functionally duplicate” messages produced as a result of session recovery. You can eliminate the possibility of message loss by always using transacted sessions but eliminating the possibility of sending a dupe message takes a bit of work. Two QMgrs talking use a protocol that eliminates any such ambiguity.
As for the performance metrics, take a look at the Performance Reports for your platform. These are all available from the SupportPacs landing page. Look for the SupportPacs with names like
MP**such as SupportPac MP71 for Windows or SupportPac MPL5 for Linux.