Question

I am implementing a bus using Rx (reactive framework) and so far it looks good. The problem that I am facing right now is how to add events before the beginning of a stream.

To give more context, this is for an Event Source (a la CQRS/ES). When the bus starts, a number of subscribers get the IObservable. At this point, the bus asks the subscribers what event number/sequence they need to start at. The events are loaded from disk starting at the lowest number and added to the stream, with each subscriber using a where statement to start at the right event. As the application runs, new events get added and go to the subscribers. This part works great.

Some subscribers that attach late, but still need all the events. It looks like ReplaySubject fits the bill, as long as the number of events is small enough. I am thinking that I can probably do some disk/offline caching to keep more them around (any pointers welcome!).

The bigger issue is that when some subscribers get the IObservable, they need to get events that occurred before the ones that were loaded initially. A situation like the following.

When the bus started it loaded event #50 onward (the earliest anyone wanted). Now, a new subscriber requests the IObservable, except they need from #20 onward. So what I want to do is load 20-49, and add them before the start of the stream. None of the existing subscribers should see any of the events (it will be filtered by the Where).

It seems like this should be possible, but I can’t quite get my head around it. Is this possible in Rx? If so, how?

Thanks!
Erick

Answer 1

I would look at this in a very simple Rx way. Don’t use a ReplaySubject, use a single Subject for all of the “live” events and concat the historical events in front of this for each subscriber.

I would imagine that you might have some sort of data structure that keeps track of the historical records. If you’ve loaded/captured events from #50 to say #80 in memory already, then when a new subscriber comes along that requests the events from #20 I’d do something like this:

var fromIndex = 20;
var eventSubject = new Subject<Event>();
capturedEvents
    .GetFromIndex(fromIndex) // Gets a enumerable list of events
    .ToObservable()
    .Concat(eventSubject)
    .Subscribe( ... );

You then can have the eventSubject be used to capture new events, like so:

eventSubject.Subscribe(capturedEvents.CaptureEvent);

Does this meet your need?