Question

One of the nice things about linq was having infinite data sources processed lazily on request. I tried parallelizing my queries, and found that lazy loading was not working. For example…

class Program
{
    static void Main(string[] args)
    {
        var source = Generator();
        var next = source.AsParallel().Select(i => ExpensiveCall(i));
        foreach (var i in next)
        {
            System.Console.WriteLine(i);
        }
    }

    public static IEnumerable<int> Generator()
    {
        int i = 0;
        while (true)
        {
            yield return i;
            i++;
        }
    }

    public static int ExpensiveCall(int arg)
    {
        System.Threading.Thread.Sleep(5000);
        return arg*arg;
    }
}

This program fails to produce any results, presumably because at each step, its waiting for all calls to the generator to dry up, which of course is never. If I take out the “AsParallel” call, it works just fine. So how do I get my nice lazy loading while using PLINQ to improve performance of my applications?

Answer 1

Take a look at MergeOptions

 var next = source.AsParallel()
              .WithMergeOptions(ParallelMergeOptions.NotBuffered)
              .Select(i => ExpensiveCall(i));