Home/ Questions/Q 506493
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-13T06:42:25+00:00

We have a C++ application for which we try to improve performance. We identified

  • 0

We have a C++ application for which we try to improve performance. We identified that data retrieval takes a lot of time, and want to cache data. We can’t store all data in memory as it is huge. We want to store up to 1000 items in memory. This items can be indexed by a long key. However, when the cache size goes over 1000, we want to remove the item that was not accessed for the longest time, as we assume some sort of “locality of reference”, that is we assume that items in the cache that was recently accessed will probably be accessed again.

Can you suggest a way to implement it?

My initial implementation was to have a map<long, CacheEntry> to store the cache, and add an accessStamp member to CacheEntry which will be set to an increasing counter whenever an entry is created or accessed. When the cache is full and a new entry is needed, the code will scan the entire cache map and find the entry with the lowest accessStamp, and remove it.
The problem with this is that once the cache is full, every insertion requires a full scan of the cache.

Another idea was to hold a list of CacheEntries in addition to the cache map, and on each access move the accessed entry to the top of the list, but the problem was how to quickly find that entry in the list.

Can you suggest a better approach?

Thanks
splintor

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Have your map<long,CacheEntry> but instead of having an access timestamp in CacheEntry, put in two links to other CacheEntry objects to make the entries form a doubly-linked list. Whenever an entry is accessed, move it to the head of the list (this is a constant-time operation). This way you will both find the cache entry easily, since it’s accessed from the map, and are able to remove the least-recently used entry, since it’s at the end of the list (my preference is to make doubly-linked lists circular, so a pointer to the head suffices to get fast access to the tail as well). Also remember to put in the key that you used in the map into the CacheEntry so that you can delete the entry from the map when it gets evicted from the cache.

    • 0