Question

I’m trying to evaluate EventStore as in reliable queuing mechanism internal to a server software.

MSMQ fails as an alternative because it cannot support partial ordering, ordered messages within “conversations” of messages. And because of its 4MB message size limit (which could be overcome with partial ordering). SQL Service Broker does support partial ordering, but is a pain in the butt to set up and manage programmatically.

As documentation on EventStore is admittedly sparse, can someone with experience with EventStore help with the following?

• Does EventStore support transactional processing of events – that is,
  if processing fails, can the dequeue be rolled back?
• With multiple readers in various threads, processes, or machines,
  does EventStore enforce that each event is dispatched(?) to only one
  reader (at a time, perhaps during a transaction)
• Assuming the above are possible, can events on different “conversations” be
  read simultaneously in any order, while messages in the same conversation be
  read singly and in-order?
• I read that EventStore is basically “At-least-once” delivery. Is it
  possible, using certain storage providers, to ensure “exactly-once”
  delivery?
• How are “poison” events handled? Events that error-out during processing. Perhaps the
  error is temporary in nature and can be retried. Perhaps it is permanent in nature and
  requires administrative intervention.
• Can EventStore storage be manipulated by hand if necessary? Can it
  be done while other readers continue to read?

(I read that transactions at the storage engine aren’t required, but I still use the language of transactions to mean whatever replaces transactions at the EventStore level. If there are crucial functional consequences in the switch from transactions to whatever, please comment on them. I don’t need to understand every aspect right away, just need hope with which to buy more time to experiment.)

Answer 1

While the EventStore could potentially be used to build a full-blown queue, it was never designed with that in mind. This means that there are a lot of opinionated decisions that went into building the library that go against the very requirements imposed by your question.

For example, the concept of exactly-once delivery is something that messaging systems don’t really support. Other things mentioned above, like poison messages, aren’t really an issue because the EventStore isn’t hooked into the message pipeline in that way.

The problem that you’re trying to solve doesn’t seem to be one where the EventStore can help you. Therefore, I would recommend evaluating a full-blown message queue, such as RabbitMQ.

Also, what do you have on your messages that makes them larger than 4MB? If you’re pushing around files or large binary streams, why not push those to some kind of highly available “global” storage (like Amazon S3) and then have a pointer to those on the message?