Question

I am attempting to upload multiple files from a Silverlight client directly to Amazon S3. The user chooses the files from the standard file open dialog and I want to chain the uploads so they happen serially one at a time. This can happen from multiple places in the app so I was trying to wrap it up in a nice utility class that accepts an IEnumerable of the chosen files exposes an IObservable of the files as they are uploaded so that the UI can respond accordingly as each file is finished.

It is fairly complex due to all the security requirements of both Silverlight and AmazonS3. I’ll try to briefly explain my whole environment for context, but I have reproduced the issue with a small console application that I will post the code to below.

I have a 3rd party utility that handles uploading to S3 from Silverlight that exposes standard event based async methods. I create one instance of that utility per uploaded file. It creates an unsigned request string that I then post to my server to sign with my private key. That signing request happens through a service proxy class that also uses event based async methods. Once I have the signed request, I add it to the uploader instance and initiate the upload.

I’ve tried using Concat, but I end up with only the first file going through the process. When I use Merge, all files complete fine, but in a parallel fashion rather than serially. When I use Merge(2) all files start the first step, but then only 2 make their way through and complete.

Obviously I am missing something related to Rx since it isn’t behaving like I expect.

namespace RxConcat
{
    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Reactive.Linq;
    using System.Timers;

    public class SignCompletedEventArgs : EventArgs
    {
        public string SignedRequest { get; set; }
    }

    public class ChainUploader
    {
        public IObservable<string> StartUploading(IEnumerable<string> files)
        {
            return files.Select(
                     file => from signArgs in this.Sign(file + "_request")
                             from uploadArgs in this.Upload(file, signArgs.EventArgs.SignedRequest)
                             select file).Concat();
        }

        private IObservable<System.Reactive.EventPattern<SignCompletedEventArgs>> Sign(string request)
        {
            Console.WriteLine("Signing request '" + request + "'");
            var signer = new Signer();
            var source = Observable.FromEventPattern<SignCompletedEventArgs>(ev => signer.SignCompleted += ev, ev => signer.SignCompleted -= ev);
            signer.SignAsync(request);
            return source;
        }

        private IObservable<System.Reactive.EventPattern<EventArgs>> Upload(string file, string signedRequest)
        {
            Console.WriteLine("Uploading file '" + file + "'");
            var uploader = new Uploader();
            var source = Observable.FromEventPattern<EventArgs>(ev => uploader.UploadCompleted += ev, ev => uploader.UploadCompleted -= ev);
            uploader.UploadAsync(file, signedRequest);
            return source;
        }
    }

    public class Signer
    {
        public event EventHandler<SignCompletedEventArgs> SignCompleted;

        public void SignAsync(string request)
        {
            var timer = new Timer(1000);
            timer.Elapsed += (sender, args) =>
            {
                timer.Stop();
                if (this.SignCompleted == null)
                {
                    return;
                }

                this.SignCompleted(this, new SignCompletedEventArgs { SignedRequest = request + "signed" });
            };
            timer.Start();
        }
    }

    public class Uploader
    {
        public event EventHandler<EventArgs> UploadCompleted;

        public void UploadAsync(string file, string signedRequest)
        {
            var timer = new Timer(1000);
            timer.Elapsed += (sender, args) =>
            {
                timer.Stop();
                if (this.UploadCompleted == null)
                {
                    return;
                }

                this.UploadCompleted(this, new EventArgs());
            };
            timer.Start();
        }
    }

    internal class Program
    {
        private static void Main(string[] args)
        {
            var files = new[] { "foo", "bar", "baz" };
            var uploader = new ChainUploader();
            var token = uploader.StartUploading(files).Subscribe(file =>   Console.WriteLine("Upload completed for '" + file + "'"));
            Console.ReadLine();
        }
    }
}

Answer 1

The base observable that is handling the 2 step upload for each file is never ‘completing’ which prevents the next one in the chain from starting. Add a Limit(1) to that observable prior to calling Concat() and it will working correctly.

return files.Select(file => (from signArgs in this.Sign(file + "_request")
                             from uploadArgs in this.Upload(file, signArgs.EventArgs.SignedRequest)
                             select file).Take(1)).Concat();