Question

I’m learning Reactive Extensions, and I’ve been trying to find out if it’s a match for a task like this.

I have a Process() method that processes a batch of requests as a unit of work, and invoking a callback when all requests have completed.

The important thing here is that each request will call the callback either synchronous or asynchronous depending on it’s implementation, and the batch processor must be able to handle both.

But no threads are started from the batch processor, any new threads (or other async execution) will be initiated from inside the request handlers if necessary. I don’t know if this match the use cases of rx.

My current working code looks (almost) like this:

public void Process(ICollection<IRequest> requests, Action<List<IResponse>> onCompleted)
{
    IUnitOfWork uow = null;
    try
    {
        uow = unitOfWorkFactory.Create();

        var responses = new List<IResponse>();
        var outstandingRequests = requests.Count;
        foreach (var request in requests)
        {
            var correlationId = request.CorrelationId;
            Action<IResponse> requestCallback = response =>
            {
                response.CorrelationId = correlationId;
                responses.Add(response);
                outstandingRequests--;
                if (outstandingRequests != 0)
                    return;

                uow.Commit();
                onCompleted(responses);
            };

            requestProcessor.Process(request, requestCallback);
        }
    }
    catch(Exception)
    {
        if (uow != null) 
            uow.Rollback();
    }

    if (uow != null) 
        uow.Commit();
}        

How would you implement this using rx? Is it reasonable?

Note, that the unit of work is to be committed synchronously even if there are async requests that have not yet returned.

Answer 1

My approach to this is two-step.

First create a general-purpose operator that turns Action<T, Action<R>> into Func<T, IObservable<R>>:

public static class ObservableEx
{
    public static Func<T, IObservable<R>> FromAsyncCallbackPattern<T, R>(
        this Action<T, Action<R>> call)
    {
        if (call == null) throw new ArgumentNullException("call");
        return t =>
        {
            var subject = new AsyncSubject<R>();
            try
            {
                Action<R> callback = r =>
                {
                    subject.OnNext(r);
                    subject.OnCompleted();
                };
                call(t, callback);
            }
            catch (Exception ex)
            {
                return Observable.Throw<R>(ex, Scheduler.ThreadPool);
            }
            return subject.AsObservable<R>();
        };
    }
}

Next, turn the call void Process(ICollection<IRequest> requests, Action<List<IResponse>> onCompleted) into IObservable<IResponse> Process(IObservable<IRequest> requests):

public IObservable<IResponse> Process(IObservable<IRequest> requests)
{
    Func<IRequest, IObservable<IResponse>> rq2rp =
        ObservableEx.FromAsyncCallbackPattern
            <IRequest, IResponse>(requestProcessor.Process);

    var query = (
        from rq in requests
        select rq2rp(rq)).Concat();

    var uow = unitOfWorkFactory.Create();
    var subject = new ReplaySubject<IResponse>();

    query.Subscribe(
        r => subject.OnNext(r),
        ex =>
        {
            uow.Rollback();
            subject.OnError(ex);
        },
        () =>
        {
            uow.Commit();
            subject.OnCompleted();
        });

    return subject.AsObservable();
}

Now, not only does this run the processing async, but it also ensures the correct order of the results.

In fact, since you are starting with a collection, you could even do this:

var rqs = requests.ToObservable();

var rqrps = rqs.Zip(Process(rqs),
    (rq, rp) => new
    {
        Request = rq,
        Response = rp,
    });

Then you would have an observable that pairs up each request/response without the need for a CorrelationId property.

I hope this helps.