Question

Hopefully the comments explain the question. I have a large static (gets built with fresh data on program load) dictionary that is used a lot and I want to reference it as efficiently as possible rather than create copies of the data and eat memory. Have a hashset the represents a subset of that dictionary that I need to represent as a sorted list of values. New HashSet or changes to the HashSet then I build a new class that wraps the code below. Is there a better way to go about this? I could not figure out how to use an external HashSet in LINQ.

// FTSwordIDs is a hashset and is also used elsewhere - a lot - hashset for lookup speed
// dlFTSword is Dictionary<Int32, string> and is static and sorted by value (on load) 
// dlFTSword can contain over a million entries and is used a lot of places 
// need to refence it rather than build a new list and eat memory
words = new List<string>();
foreach(Int32 id in dlFTSword.Keys)
{
    if (FTSwordIDs.Contains(id)) words.Add(dlFTSword[id]);
}
return words;

The time is 12 ms

try
{
    sqlConRO1.Open();
    sqlCMDRO1.CommandText = "SELECT [ID], [word] FROM [FTSwordDef] WITH (NOLOCK); "; // ORDER BY [word]; ";
    SqlDataReader rdr = sqlCMDRO1.ExecuteReader();
    while (rdr.Read())
    {
        dlFTSword.Add(rdr.GetInt32(0), rdr.GetString(1));
    }
    rdr.Close();
    Debug.WriteLine("dlFTSword.Count = " + dlFTSword.Count.ToString());

}
catch (Exception Ex) { throw new Exception("InitializeData Failed " + Ex.Message); }
finally { sqlConRO1.Close(); }

HashSet<Int32> wordIDs = new HashSet<int>() { 1, 100000, 200000, 300000, 400000, 500000, 600000 };
Stopwatch stopWatch = new Stopwatch();
stopWatch.Start();

//List<string> words = dlFTSword.Where(pair => wordIDs.Contains(pair.Key))
//         .Select(pair => pair.Value)
//         .OrderBy(x => x)
//         .ToList();
//List<string> words = wordIDs.Where(key => dlFTSword.ContainsKey(key))
//          .Select(key => dlFTSword[key])
//          .OrderBy(value => value)
//          .ToList();
IEnumerable<string> words = wordIDs.Where(key => dlFTSword.ContainsKey(key))
          .Select(key => dlFTSword[key])
          .OrderBy(value => value);
DateTime dtEnd = DateTime.Now;
stopWatch.Stop();
TimeSpan ts = stopWatch.Elapsed;
Debug.WriteLine(ts.Milliseconds.ToString());

Answer 1

Sounds like you want:

var words = dlFTSword.Where(pair => FTSwordIDs.Contains(pair.Key))
                     .Select(pair => pair.Value)
                     .ToList();

Note that this avoids performing a lookup on each key after finding that it’s present. We don’t actually look anything up in the dictionary. It also doesn’t involve creating any other sets, which Intersect would.

This is worth checking though:

dlFTSword is Dictionary and is static and sorted by value (on load)

If it’s a Dictionary<TKey, TValue> then it isn’t sorted. There’s no such concept as a “sorted” Dictionary<TKey, TValue>. You could use a SortedDictionary<,> or SortedList<,>, but those are both different to a Dictionary<,>.

EDIT: If the hash set is really small, it makes more sense to iterate over that than over every pair in the dictionary:

var words = FTSwordIDs.Where(key => dlFTSword.ContainsKey(key))
                      .Select(key => dlFTSword[key])
                      .OrderBy(value => value)
                      .ToList();

It’s slightly ugly to have to do the lookup, but it’ll be significantly faster.