Question

The data structures I know such as b-tree or rb-tree do not change when I keep looking for the same keys. I’m looking for a data structure which optimizes itself in the runtime for faster lookup for frequently used keys. Now I have a naive implementation like this:

ValueType MyNaiveMap::get(KeyType key) {
  innerMap.addFreq(key);
  if (key == mostFreqUsedKey1) {
    return val1;
  } else if (key == mostFreqUsedKey2) {
    return val2;
  }
  else {
    return innerMap.get(key);
  }
}

The keys are always int in my case.

UPDATE:

I forgot to mention hash map. Because the map may grow to a very large size, I was trying to avoid resizing which is O(n) complex in the runtime.

Answer 1

If you know the total size ahead of time, or if O(n) insertions due to resizing (but still O(1) amortized in the average case) are okay for you, then I think hash table is the best option. The one-time hit of copying the whole table when resizing can also be distributed over several operations by using incremental resizing.

You have to keep the hash table partially empty to avoid collisions, which may feel like wasting memory. But with tree-based maps, you usually have at least one additional pointer per item, which will also “waste” a significant amount of memory. Also, with a hash table, you can tweak how much of it to keep empty, partially trading performance for memory (or vice versa).

But there also is a tree-based data structure optimized for looking up recently accessed keys: splay tree. It uses rotations to keep recently used keys near the root, while still maintaining amortized worst case of O(log n) for insertion and searching. (That “amortized” bit is important, some operations can still be O(n)).