I need to make a big cache of things — a (key,value) dictionary. I want to be able to save/load this cache to/from disk. The problem is, the dictionary might be too big to fit in memory, so only a part of it can be loaded in RAM at given point in time.
Is there a standard way of dealing with this problem. Am I forced to used a database? I was thinking about sqlite, but it’s not thread safe (?).
If you just need a big dictionary you may want to look at using a database that provides a Key-Value store, rather than a relational database like SQLite.
Popular Key-Value stores include: