Question

I am looking to get some ideas on how I can solve my failover problem in my Java service.

At a high level, my service receives 3 separate object streams of data from another service, performs some amalgamating logic and then writes to a datastore.

Each object in the stream will have a unique key. Data from the 3 streams can arrive simultaneously, there is no guaranteed ordering.

After the data arrives, it will be stored in some java.util.concurrent collection, such as a BlockingQueue or a ConcurrentHashMap.

The problem is that this service must support failover, and I am not sure how to resolve this if failover were to take place when data is stored in an in-memory object.

One simple idea I have is the following:

1. Write to a file/elsewhere when receiving an object, and prior to adding to queue
2. When an object is finally procesed and stored in the datastore
3. When failover occurs, ensure that same file is copied across and we know which objects we need to receive data for

Performance is a big factor in my service, and as IO is expensive this seems like a crude approach and is quite simplistic.

Therefore, I am wondering if there are any libraries etc out there that can solve this problem easily?

Answer 1

I would use Java Chronicle partly because I wrote it but mostly because …

• it can write and read millions of entries per second to disk in a text or a binary format.
• can be shared between processes e.g. active-active clustering with sub-microsecond latency.
• doesn’t require a system call or flush to push out the data.
• the producer is not slowed by the consumer which can be GBs ahead (more than the total memory of the machine)
• it can be used in a low heap GC-less and lockless manner.