Question

I’m working in the microsoft world and needing to create a transaction processing webservice. The webservice will receive a transaction and submit it to a queue. A second service will pull the transactions out of the queue and process them.

I have been considering two different approaches: database table (roll my own queue) and microsot queues (MSMQ).

Do people have pros/cons to either approach or suggestion for a different queue technology?

Answer 1

It’s not really a development question, but I do have some input.

It depends on how advanced your queueing requirements are, and what operational experience and skills your operations people will have.

if you will deploy into a shop where they know and love SQL Server, but have no experience managing MSMQ, then using SQL for the queue (essentially adding an additional table to SQL) will be easy for them to do – basically no additional cost for operations. On the other hand using MSMQ will entail the staff getting trained, developing and verifying new procedures, new backup plans, new monitoring and administration tools, and so on. And vice versa – in the rare case a shop has good experience with MSMQ but little experience with SQL Server. In that case, the reverse is true: go with MSMQ.

It may be that your use of the queue is structured so that you don’t care about backup or mitigating data loss. In this case, you don’t plan to do any monitoring or maintenance. In that case, using MSMQ will imply almost zero additional operations cost, regardless of the existing skills on staff, because setup will be a one-time thing, and ongoing maintenance is minimal.

The second criterion is your use of the queue. If the queue is to be used heavily, let’s say millions of transactions an hour for every server, then you probably want a dedicated queue service for that, rather than piggy-backing onto an existing database. If you would like to take advantage of the various MSMQ features like dead letter queues, priority messages, triggering, distribution lists, and correlation IDs, then that recommends MSMQ. On the other hand if your volume is low (say, hundreds of transactions an hour), and you’re not using the more advanced queue features, then you can just use a queue facade on top of a SQL table. Easy peasy.

How to balance these two major criteria is for you to decide.

Non criteria include: programmability (both options are easy to develop for in various languages), transaction volume (both can handle small to large volumes), data format (both are flexible w.r.t. data content and format).

In any case I would not recommend using an alternative queue, if you are already running Windows Server. IBM has a good one in Websphere MQ (nee MQSeries), but it is not free, as MSMQ is. Also unless you are doing multi-platform queue networks, you probably don’t need MQSeries. I don’t consider any other queueing systems to be mainstream enough for consideration.

If you don’t mnind changing your model a little – you may be interested in the marriage of WCF and WF that Microsoft has done recently. It allows a WCF service to kick off a workflow (WF). The WF engine itself supports queuing, scheduling, and passivation – essentially there is a queue backing it, but as a programmer you don’t deal directly with the queue metaphor. Instead it is a new “workflow” metaphor. WCF and WF, like MSMQ, are both free (that is to say, no extra cost beyond the cost of licensing Windows Server); and included in Windows Server. This WCF+WF approach may be a more appealing model for you to use for new development, or if you’d prefer to take advantage of the WF designers and tools. As with the rest of the MS app platform, WF supports transactional access, so you can provide fire-and-forget semantics.

Good luck on your decision.