Question

How to create a true function pipeline using C#? I got some idea like as follows, but it is not a true pipeline

public static IEnumerable<T> ForEachPipeline<T>(this IEnumerable<T> source, params Func<T, T>[] pipeline)
{
 foreach (var element in source) {
  yield return ExecutePipeline(element, pipeline);
 }
}

private static T ExecutePipeline<T>(T element, IEnumerable<Func<T, T>> jobs)
{
 var arg = element;
 T result = default(T);
 foreach (var job in jobs) {
  result = job.Invoke(arg);
  arg = result;
 }
 return result;
}

In the above code each element of IEnumerable<T> would able to get into the pipeline only after the previous element finishes executing all the functions (i.e. exits the pipeline), but according to the definition if element1 finishes executing func1 and start executing func2, by that time element2 should start executing func1 and so on thus maintaining continues flow of data in the pipeline.

Is this kind of scenario possible to implement in C#? If possible please give me some example code.

Answer 1

From comment: There is only a single execution context unless threads are introduced (the alternative approach with a single-thread is just the non-lazy result building at each step). With threads, each stage is just a FIFO queue passing messages around a “pump”. Threads (actually, concurrency) also greatly increase complexity, perhaps see the .NET4 “Parallel” methods.

An “easy” method is just to configure N “starts” using Parallel.ForEach — if and only if you can guarantee that the computations are side-effect free.

Edit: See comment(s).