Question

I’m using the synchronous implementation of JRedis, but I’m planning to switch to the asynchronous way to communicate with the redis server.

But before that I would like to ask the community whether the JRedisFuture implementation of alphazero’s jredis is stable enough for production use or not?

Is there anybody out there who is using it or have experience with it?

Thanks!

Answer 1

When JRedis gets support for transaction semantics (Redis 1.3.n, JRedis master branch) then certainly, it should be “stable” enough.

Redis protocol for non-transactional commands, themselves atomic, allows a window of unrecoverable failure when a destructive command has been sent, and on the read phase the connection faults. The client has NO WAY of knowing if Redis in fact processed the last request but the response got dropped due to network failure (for example). Even the basic request/reply client is susceptible to this (and I think this is not limited to Java, per se.)

Since Redis’s protocol does not require any metadata (at all) with the DML and DDL type commands (e.g. no command sequent number) this window of failure is opened.

With pipelining, there is no longer a sequential association between the command that is being written and the response that is being read. (The pipe is sending a command that is N commands behind the one that caused Redis to issue the response being read at the same time. If anything goes kaput, there are a LOT of dishes in air 🙂

That said, every single future object in the pipe will be flagged as faulted and you will know precisely at which response the fault occurred.

Does that qualify as “unstable”? In my opinion, no. That is an issue with pipelining.

Again, Redis 1.3.n with transaction semantics completely addresses this issue.

Outside of that issue, with asynchronous (pipelines), there is a great deal of responsibility on your part for making sure you do not excessively overload the input to the connector. To a huge extent JRedis pipelines protect you from this (since the caller’s thread is used to make the network write thus naturally damping the input load on the pending response queue).

But you still need to run tests — you did say “Production”, right? )) — and size your boxes and put a cap on the number of loading threads on the front end.

I would also potentially recommend not running more than one JRedis pipeline on multi-core machines. In the existing implementation (which does not chunk the write buffer) there is room for efficiencies (in context of full bandwidth utilization and maximizing throughput) to be gained by running multiple pipelines to the same server. While one pipeline is busy creating buffers to write, the other is writing, etc. But, these two pipelines will interfere with one another due to their (inevitable — remember they are queues and some form of synchronization must occur) and periodic cache invalidation (on each dequeue/enqueue in worst case — but in Doug Lea we trust.) So if pipeline A average latency hit d1 (in isolation), then so does pipe B. Regrettably, running two of them on the same cores will result in a new system wide cache invalidation period that is HALF of the original system so TWICE as more cache invalidations occur (on average). So it is self defeating. But test your load conditions, and on your projected production deployment platform.