Question

I have an advanced program that retrieves data from a database and creates objects from the data via ExpandoObjects in C#.

I’m now in the process of optimizing my entire process but came upon a piece of code that was the bottleneck performance-wise. I am quite curious how far I can furhter optimize this piece of code and already managed to run it 3 times as fast by doing the following:

• Seperated ‘the finding ‘types’ section’ in a seperate for loop and only iterate over it when it hasn’t been initialized yet.

• Added a case for when the value is null and create an empty string instead, since for some reason Dictionary.Add slows down quite a lot when adding Null values.

      // Holds all objects that are created inside the object.
      Dictionary<string, IDictionary<string, dynamic>> objects = new Dictionary<string, IDictionary<string, dynamic>>();
  
      // This foreach loop is the slowest part!
      foreach (KeyValuePair<string, dynamic> pair in fields)
      {
          string type = pair.Key.Split('_')[0];
  
          IDictionary<string, dynamic> obj;
          if (!objects.TryGetValue(type, out obj))
          {
              obj = new ExpandoObject();
              objects.Add(type, obj);
          }
          int location = pair.Key.IndexOf(type + "_");
          string key = pair.Key.Remove(location, type.Length + 1);
          if (pair.Value == null)         // If Value is null, replace it with an empty string (dictionary slows quite alot when passing it null values)
              obj.Add(key, "");
          else
              obj.Add(key, pair.Value);         
      }
  
      if (types == null)
          types = objects.Select(x => x.Key).ToList();
  

I am wondering, how is it that dictionary slows down that much when adding Null values, is it that in the underlying structure it does special operations when encountering null values? And is there something I’m missing to further optimize the code?.

Any help is again greatly appreciated.

---

UPDATE

• Edited the code with the most recent changes I’ve gathered from SO.

Answer 1

You can avoid the multiple lookups in the dictionary by using TryGetValue instead of ContainsKey:

foreach (KeyValuePair<string, dynamic> pair in fields) 
{ 
    string type = pair.Key.Split('_')[0]; 

    IDictionary<string, dynamic> obj;
    if (!objects.TryGetValue(type, out obj)) 
    { 
        obj = new ExpandoObject(); 
        objects.Add(type, obj); 
    } 
    int location = pair.Key.IndexOf(type + "_"); 
    string key = pair.Key.Remove(location, type.Length + 1); 
    if (pair.Value == null)
        obj.Add(key, ""); 
    else 
        obj.Add(key, pair.Value);            
} 

BTW: I don’t see you using types in that second foreach loop. Therefore you can replace the first foreach loop with this simple code:

types = objects.Select(x => x.Key).ToList();

This should save quite a lot of time as it removes the slow search in the types list and the double parsing of the types.

Obviously this needs to be put after the foreach loop I showed above.