We are writing hi-load order processing engine. Every cluster node processing some set of contracts and write action log to local file. This file should be distributed amount some other nodes (for fault tolerance). If node fail there should be way to restore it’s state at one of replication nodes as fast as possible. Currently we use cassandra but there is some problems with partitioner: there is no way to specify what nodes should be used for a specific table.
So we need to replicate file. Is there a solution?
Edit: peak load will be about 200k records per second.
With respect to your Cassandra issue: while you can’t have a different replication layout per table/columnfamily, you can have a different layout per keyspace. This includes a case like yours, where it sounds like you want some set of nodes S1 be wholly responsible for some parts of the data, and some other set S2 to be responsible for another part.
If you represent S1 and S2 as different datacenters to Cassandra (via PropertyFileSnitch or whatever), then you can configure, say, keyspace K1 to have X copies on S1 and none on S2, and vice versa for keyspace K2.