Question

I have a set of nicely structured objects set up along these lines:

abstract class Data 
{
  public abstract void GetData();
  public abstract void Process();
}
abstract class DataSource1 : Data
{
  public void GetData() { ... }
}
class DataSource1ProcessorA : DataSource1
{
  public void Process() { ... }
}
abstract class DataSource2 : Data
{
  public void GetData() { ... }
}
class DataSource2ProcessorA : DataSource2
{
  public void Process() { ... }
}
class DataSource2ProcessorB : DataSource2
{
  public void Process() { ... }
}

Currently, I’m processing each DataSourceProcessor sequentially, which is starting to get long as we add more sources and processors. Each source is completely unrelated, but each processor needs to be run in order. So I need a way to turn

new List<List<Data>> tasks = new List<List<Data>>()
{
  new List<Data>() { new DataSource1ProcessorA() },
  new List<Data>() { new DataSource2ProcessorA(), new DataSource2ProcessorB() }
}
foreach (List<Data> source in tasks)
{
  foreach(Data data in source)
  {
    data.GetData();
    data.Process();
  }
}

into a series of Tasks. I’ve worked with Tasks before, although I haven’t done anything really complicated with them, and I just can’t figure out how to fire off an arbitrary number of tasks with an arbitrary sequence of sequential calls.

I don’t need to access the Tasks in any way, other than kicking them off and reporting any errors they throw. Preferably an error in DataSource2ProcessorA shouldn’t prevent DataSource2ProcessorB from running, but if the coding is significantly easier to allow one error to kill the whole DataSource’s thread, I can live with that.

Answer 1

You want to use a parallel foreach, and an action that processes the list in order for each item. That is:

new List<List<Data>> tasks = new List<List<Data>>()
{
  new List<Data>() { new DataSource1ProcessorA() },
  new List<Data>() { new DataSource2ProcessorA(), new DataSource2ProcessorB() }
}

Parallel.ForEach(tasks, (items) =>
{
    foreach (var item in items)
    {
        item.GetData();
        item.Process();
    }
});

That that, although this ensures that DataSource2ProcessorA is processed before DataSource2ProcessorB, there’s no guarantee about the order in which tasks are run.