Question

I know a lot has been written about Linq and it’s inner workings. Inspired by Jon Skeets EduLinq, I wanted to to demystify what’s going on behind the Linq Operators. So what I tried to do is to implement Linqs Select() method which sounds pretty boring at first sight. But what I’m actually trying to do is to implement it without the use of the yield keyword.

So here is what I got so far:

class Program
{
    static void Main(string[] args)
    {
        var list = new int[] {1, 2, 3};

        var otherList = list.MySelect(x => x.ToString()).MySelect(x => x + "test");

        foreach (var item in otherList)
        {
            Console.WriteLine(item);
        }

        Console.ReadLine();
    }
}

public static class EnumerableEx
{
    public static IEnumerable<R> MySelect<T, R>(this IEnumerable<T> sequence, Func<T, R> apply)
    {
        return new EnumerableWrapper<R, T>(sequence, apply);
    }
}

public class EnumerableWrapper<T, O> : IEnumerable<T>
{
    private readonly IEnumerable<O> _sequence;
    private readonly Func<O, T> _apply;

    public EnumerableWrapper(IEnumerable<O> sequence, Func<O, T> apply)
    {
        _sequence = sequence;
        _apply = apply;
    }

    public IEnumerator<T> GetEnumerator()
    {
        return new EnumeratorWrapper<T, O>(_sequence, _apply);
    }

    IEnumerator IEnumerable.GetEnumerator()
    {
        return GetEnumerator();
    }
}

public class EnumeratorWrapper<T, O> : IEnumerator<T>
{
    private readonly IEnumerator<O> _enumerator;
    private readonly Func<O, T> _apply;

    public EnumeratorWrapper(IEnumerable<O> sequence, Func<O, T> apply)
    {
        _enumerator = sequence.GetEnumerator();
        _apply = apply;
    }

    public void Dispose()
    {
    }

    public bool MoveNext()
    {
        var hasItems = _enumerator.MoveNext();
        if (hasItems)
            Current = _apply(_enumerator.Current);
        return hasItems;
    }

    public void Reset()
    {
        _enumerator.Reset();
    }

    public T Current { get; private set; }

    object IEnumerator.Current
    {
        get { return Current; }
    }
}

It seems to work. However, I have trouble to follow it’s control flow. As you can see, I chain to projections. This causes to let strange things (strange for me!) happen in the MoveNext() method. If you set breakpoints in every line of the MoveNext() method you will see that the control flow actually jumps between the different instances and never works through the method in one batch. It’s jumping as if it were using different threads or if we were using yield. But in the end, it’s just a normal method so I wonder what’s going on there?

Answer 1

Each time you call MoveNext() on the result it will:

• Call MoveNext() on the final iterator (which I’ll call Y), which will…
• Call MoveNext() on the previous iterator (which I’ll call X), which will…
• Call MoveNext() on the original iterator (of the array), which will return a number.
• MoveNext() in X will then call the projection x => x.ToString() so that it has an appropriate Current member
• MoveNext() in Y will then call the projection x => x + "test" on the result of X.Current, and store the result in Y.Current

So there’s nothing particularly magical going on here – you’ve just got stacked calls, just as normal. The debugging experience will show you that call from one EnumeratorWrapper.MoveNext to another; the only odd jumping around will be when you step through the projections, which are declared in the Main method.

If that doesn’t explain what was confusing you, please give more details about exactly where you don’t understand the flow, and I’ll see how I can help. (It’s great to see you wanting to learn more about how all these things work though. Good to see a kindred spirit!)