Question

Has anyone found/coded extension method that query data (using linq to sql) in batches? I’ve seen IEnumerable extensions but I’m looking for something I might use like this:

IQueryable<Order> orders = from i in db.Orders select i;
foreach(var batch in orders.InBatches(100))
{
   //batch of 100 products
   foreach(var order in batch)
   {
      //do something
   }
}

Answer 1

What you can do is this:

public static IEnumerable<IQueryable<T>> InBatches(
    this IQueryable<T> collection, int size)
{  
    int totalSize = collection.Count();

    for (int start = 0; start < totalSize; start += size)
    {
        yield return collection.Skip(start).Take(size);
    }
}

This extension method allows you to do extra filters over the return IQueryables. However, the usefulness is pretty limited. I can’t think of any good scenario for this :-). In most scenario’s you just want to stream the results and returning an IEnumerable<IEnumerable<T>> would just do fine, and is even better, since this will result in a single SQL query, while the shown approach will result in N + 1 queries.