Question

LINQ uses a Deferred Execution model which means that resulting sequence is not returned at the time the Linq operators are called, but instead these operators return an object which then yields elements of a sequence only when we enumerate this object.

var results = someCollection.Select(item => item.Foo).Where(foo => foo < 3);

When we enumerate results object, it will iterate through someCollection only once, and for each item requested during the iteration, code ( located inside results object ) performs the map operation and finally performs the filtering.

But I’m having trouble understanding what is going on under the hood:

a) Is Where method the one that actually creates results object?

b) If Where does create results object, then I’m assuming Where needs to also exctract some logic from Select operator ( such as return Item.Foo ) so that it can place that logic into results object?

c) If my assumptions are correct, how is Where able to extract the logic out of Select?

d) Anyways, results object contains the necessary logic L to evaluate each item in someCollection. I assume this logic L doesn’t make any additional calls to Select and Where operators when evaluating each item in someCollection?

Thank you

EDIT:

1)

Your assumption in d) is incorrect – results is just an
IEnumerable which is returned by the Where() extension
method. Only when you iterate over the enumeration (i.e. using foreach
or ToList()) will the sequence be created “for real”. At that point –
you can even see this if you set a break point – all the Linq
extension methods are executed in turn – the Where() extension method
will ask the input IEnumerable for its first item, which will cause
the Select() operator in turn will get the first item from the
underlying collection and spit out a FooType item.

a) So Where and Select are first called in the assignment statement when assigning resulting object to results variable ( var results=... ). And then in turn Where / Select are also called ( from within the results object ) for each item when enumerating someCollection?

b) Assuming results instance is of type C – when is C class defined/created? Is it defined by Where method, or is class C defined by compiler and thus Where only returns an instance of C?

2)

Only when you iterate over the enumeration (i.e. using foreach or
ToList()) will the sequence be created “for real”. At that point – you
can even see this if you set a break point – all the Linq extension
methods are executed in turn – the Where() extension method will ask
the input IEnumerable for its first item, which will cause the
Select() operator in turn will get the first item from the underlying
collection and spit out a FooType item

a) You’re saying that from within results object Select and Where are called for each item I in a collection. Assuming I doesn’t implement IEnumerable<>, how then can Select and Where be called on I if they can only operate on IEnumerable<> types?

Answer 1

Think of it like this, because this is what happens at compile time anyway:

var results = someCollection.Select(item => item.Foo).Where(foo => foo < 3);

is translated into

var results = Enumerable.Where(
                  Enumerable.Select(
                      someCollection, item => item.Foo
                  ),
                  foo => foo < 3
              );

So now it’s clear that the Where operates on the result of Select. Then Where will pull from its source (in this case, the result of Enumerable.Select) and yield one at a time the items from the source that match the predicate (in this case foo < 3).

The implementation will look something like this:

public static IEnumerable<T> Where<T>(
    IEnumerable<T> source,
    Func<T, bool> predicate
) {
    foreach(var item in source) {
        if(predicate(item)) {
            yield return item;
        }
    }
}

public static IEnumerable<U> Select<T, U>(
    IEnumerable<T> source,
    Func<T, U> project
) {

    foreach(var item in source) {
        yield project(item);
    }
}

So what happens is that when you want to pull an item from results, Where will pull from Select until it find an item that matches the predicate. It might have to pull a lot of items until it finds one to yield back to you. Meanwhile, every time that it pulls from Select, Select pulls another item from someCollection and yield backs the projection (item.Foo). When you try to pull another item from Where, Where will pull the next however many items it needs from Select until it finds one to yield back to you. If Select exhausts someCollection at any point, Where will know it has exhausted the supply of items as well and will stop yielding back to you.