Question

I have a thread, call it the “Parsing thread”.

Thread parsingThread = new Thread(myMethod);

I perform some computations on this thread, of which the last involves more parallel computations.

    public void ReadCityFiles(BlockingCollection<GeonamesFileInfo> files)
    {
        Parallel.ForEach<GeonamesFileInfo>(
            files.GetConsumingPartitioner<GeonamesFileInfo>(),
            new ParallelOptions { MaxDegreeOfParallelism = _maxParallelism },
            (inputFile, args) =>
            {
                RaiseFileParsing(inputFile);
                using (var input = new System.IO.StreamReader(inputFile.FullName))
                {
                    while (!input.EndOfStream)
                    {
                        RaiseEntryParsed(ParseCity(input.ReadLine()));
                        Interlocked.Increment(ref _parsedEntries);
                    }
                }
                RaiseFileParsed(inputFile);
            });
        RaiseDirectoryParsed(Directory);
    }

The problem is that when these very long and computationally expensive async foreach operations finish (~30 mins), the “Parsing Thread” doesn’t resume. My GUI is still responsive, but the RaiseDirectoryParsed function that is supposed to continue to run on the “Parsing Thread” is never called. I debugged the program up to this point, and am pretty baffled as to what to do in this situation.

Answer 1

The point of BlockingCollection is that when an operation cannot be performed now, but might be in the future (e.g. Take() or Add() on a collection with bounded capacity), it will block. The same applies to GetConsumingEnumerable() and thus also to GetConsumingPartitioner(): if the collection is currently empty, the enumerable will block until you add more items to the collection.

But there is also a way to tell the collection that you’re not going to add new items anymore and that it shouldn’t block when empty from now on: the CompleteAdding() method. If you call this when you know you won’t be adding any more new items to the collection, your Parallel.ForEach() won’t block anymore and your thread will continue executing.