I need to store a list of processed network file shares (keeping the full UNC path) in memory so I do not process these folders again.
I was going to use an array but now I am thinking that a hashtable might be a faster option. I will need to check in a few places if that folder path (string value) exists already.
What what be the fastest (to check for an existence of that value) and less memory consuming option?
- Store folder paths in a hashtable with an key=folder path and an empty value
- Store folder paths in a one dimensional array
- Something else
UPDATE:
I just ran a few tests like in that thread pointed below by generating 50000 strings and doing 3 lookups. I also looked at memory consumption and HashSet is a winner. HashTable is still a good option but Array is the worst one.
Array – 125048 ms to create, 37 ms to lookup, 18Mb in RAM
Hash – 2167 ms to create, 0.74 ms to lookup, 2.4Mb in RAM
HashSet – 1966 ms to create, 0.02 ms to lookup, 1.4Mb in RAM
A hastable is preferable over an array in this situation as you are going to be querying for the keys after populating it and the number of items is relatively large. I would start with a hashtable. If the performance is not suitable, then try something else, like utilizing .net’s Dictionary(T,V) or .net 3.5+’s HashSet(T)
Also, here is a link that will be of interest to you:
Powershell 2 and .NET: Optimize for extremely large hash tables?