Question

I have a class with a method that is running a nested ParallelFor loop. Basically I’m iterating over a list of objects and then a list contained in the properties of each object.

Based on a condition that is calculated for each object in the inner loop, I want to add to a queue. I’m using a “syncRoot” object in an attempt to maintain concurrency when adding to the queue.

public class ParallelTest
{
    private static object syncRoot = new object();

    public void Test() {
       List<MyLog> queue = new List<MyLog>();
       ...
       Parallel.For(0, set.Count(), delegate(int i)
       {
           var obj = set[i];
           List<Connection> conns = obj.GetConnections();
           ...      
           Parallel.For(0, conns.Count(), delegate(int j)
           {    
               Connection c = conns[j];
               MyLog log = new MyLog();             
               ...                  
               if (condition)
               {                    
                   lock (syncRoot)
                   {
                       queue.Add(log);
                   }
               }
           }
       }

       Debug.WriteLine(queue.Count);
   }
}

The problem I have is that it seems not all of my objects are getting added to the queue. I’m testing for a set of 200 objects and replacing the condition with true, so I would expect queue.Count to be 200. However, I get strange results … sometimes 200, sometimes 198 or 199.

What am I doing wrong here? How do I ensure each thread is accounted for?

Answer 1

This seems like an excellent candidate for Linq.

Assuming the code represented by ... could be encapsulated in a method with signature:

void Initialize(MyLog log, Connection conn, SomeUnknownType obj)

Your code could reduce to the following linq statement:

    var logs = set  
        .AsParallel()
        .SelectMany(
            obj =>
                obj.GetConnections()
                    .Select(conn => new{obj, conn}))
        .Select(x => { 
            var o = new{x.obj, x.conn, log = new MyLog()};
            Initialize(o.log, o.conn, o.obj); //or just do work inline
            return o;
        })
        .Where(x => x.obj... && x.conn...) //someCondition

    queue = logs.ToList();

Seeing as set.Count() is relatively high, parallelizing over set will ensure that the work is reasonably well divided over available cores. There’s no need to parallelize again later on.