If I have, say, 100 items that’ll be stored in a dictionary, should I initialise it thus?
var myDictionary = new Dictionary<Key, Value>(100);
My understanding is that the .NET dictionary internally resizes itself when it reaches a given loading, and that the loading threshold is defined as a ratio of the capacity.
That would suggest that if 100 items were added to the above dictionary, then it would resize itself when one of the items was added. Resizing a dictionary is something I’d like to avoid as it has a performance hit and is wasteful of memory.
The probability of hashing collisions is proportional to the loading in a dictionary. Therefore, even if the dictionary does not resize itself (and uses all of its slots) then the performance must degrade due to these collisions.
How should one best decide what capacity to initialise the dictionary to, assuming you know how many items will be inside the dictionary?
What you should initialize the dictionary capacity to depends on two factors:
(1) The distribution of the gethashcode function, and
(2) How many items you have to insert.
Your hash function should either be randomly distributed, or it is should be specially formulated for your set of input. Let’s assume the first, but if you are interested in the second look up perfect hash functions.
If you have 100 items to insert into the dictionary, a randomly distributed hash function, and you set the capacity to 100, then when you insert the ith item into the hash table you have a (i-1) / 100 probability that the ith item will collide with another item upon insertion. If you want to lower this probability of collision, increase the capacity. Doubling the expected capacity halves the chance of collision.
Furthermore, if you know how frequently you are going to be accessing each item in the dictionary you may want to insert the items in order of decreasing frequency since the items that you insert first will be on average faster to access.