Question

I am working with an interface which sits on top of (for example) a StreamReader or SqlDataReader. The interface exposes a method, GetNext(), which returns an object if there are any left, or null if there are none left.

public interface ICollectionWidget<T>
{
    T GetNext(); // Returns a T if there are any left, or null if there aren't
}

I need to process each T returned by GetNext() in parallel, and stop processing when GetNext() returns null. I’m not quite sure how this is done (using TPL or whatever). I need a kind of parallel while! Obviously, I don’t want any threads still processing to finish when I get a null, I just don’t want to add any new processing – and then to drop out of the ‘loop’ when all the threads finish what they’re doing.

Can anyone help? Please let me know if my question doesn’t make sense.

Answer 1

Note that “collections” like the one you’re showing are typically expose via IEnumerable<T>. If you have control over the API itself, I’d use IEnumerable<T> instead of the GetNext()-based iteration approach. If you don’t, however, it’s simple to perform the conversion…

I would wrap this API to expose it as an IEnumerable<T>. You could then use Parallel.ForEach:

private IEnumerable<T> EnumerateWidgets<T>(ICollectionWidget<T> widgets)
{
    T element = widgets.GetNext();
    while (element != null)
    {
        yield return element;
        element = widgets.GetNext();
    }
}

You could then use:

Parallel.ForEach(EnumerateWidgets(widgetCollection), widget =>
{
     // Process widget here
});

This will prevent threading issues while enumerating your widgets (as the enumerator will be single threaded), but allow you to process your collection in parallel.