Home/ Questions/Q 6091343
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-23T12:18:59+00:00

I’m trying to design a system which reports activity events to a database via

  • 0

I’m trying to design a system which reports activity events to a database via a web service. The web service and database have already been built (COTS software) – all I have to do is provide the event source.

The catch, though, is that the event source needs to be fault tolerant. We have multiple replicated databases that I can talk to, so if the web service or database I’m talking to goes down, the software can quickly switch to another one that’s up.

What I need help with though is the case when all the databases are down. I’ve already designed a queue that will hold on to the events as they pile in (and burst them out once the connection is restored), but the queue is an in-memory structure: if my app crashes in this state, or if power is lost, etc., then all the events in the queue are lost. This is unacceptable. What I need is a way to persist the events so that when a database comes back online I can send a burst of queued-up events, even in the event of power loss or crash.

I know that I don’t want to re-implement the queue itself to use the file system as a backing store. This would work (and I’ve tried it) – but that method slows the system down dramatically as the hard drive becomes a bottleneck. Aside from this though, I can’t think of a single way to design this system such that all the events are safely stored on the hard drive only when access to the database isn’t available.

Does anyone have any ideas? =)

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    When I need messaging with fault tolerance (and/or guaranteed delivery, which based on your description I am guessing you also need), I usually turn to MSMQ. It provides both fault tolerance (messages are stored on disk in case of machine restart) and guaranteed delivery (messages will automatically and continually resend until they are received), as well as transactional sends and receives, message journaling, poison message handling, and other features.

    I have been able to achieve a throughput of several thousand messages per second using MSMQ. Frankly, I am not sure that you will get too much better than that while still being fault tolerant.

    • 0