Question

During adding key to dictionary, it is crashing if I don’t lock on it and giving NullReferenceException which is reasonable

Rarely sometimes it is also crashing when I am adding element to the Dictionary Value (list reference), which is weird…

I have another question as well. The files are in text format. Sometimes reading them takes 1890 ms, and other times it is taking 10 times as much as that. The runs are made consecutive. Is there a possiblity that something is busying the I/O buffer suddenly

Any recommendation to at least stablize this…

        private static void ParallelReadAndCalculate(FileInfo[] files)
    {
        Stopwatch sw1 = Stopwatch.StartNew();
        while (!Parallel.ForEach(files, (FileInfo file) => ReadFileToEnd(file)).IsCompleted) ;
        Console.WriteLine(sw1.ElapsedMilliseconds);
    }

        private static void ReadFileToEnd(FileInfo file)
    {
        string key = file.Name.Split('.')[0];
        lock (ListOfCompanyData)
            if (!ListOfCompanyData.ContainsKey(key))
            {
                ListOfCompanyData.Add(key, new List<string>(19800));
            }
        string line = "";

        using (StreamReader streamReader = (new StreamReader(file.FullName)))
        {                
            while ((line = streamReader.ReadLine()) != null) {
              // this is giving KeyNotFoundException sometimes and others, do I need to lock here given the fact that I am accessing different references synchronously
                    ListOfCompanyData[key].Add(line);                
            }                                                       
        }

    }

Answer 1

I would move the adding to the dictionary outside the parallel loop. This saves you a lock (at the cost of an extra loop), and makes it at least more practical to inspect the contents of the dictionary before the parallel processing.

Besides, it’s the reading that you want done in parallel, not the adding to the dictionary.