Question

I’m using Django to power a site where I pull in tweets from twitter timelines for use (for about 50 different people). I want to keep a large dictionary of all the tweets in a cache so I don’t have to poll twitter every page-refresh. Right now I have it so when it retrieves tweets (30) from twitter, it saves it in the default cache with the key being that user’s ID. However, I want it to save these in the long-term so the list of tweets for a user grows over time.
My question is, if I save them using the file-system cache instead, will the files themselves (pickled dictionaries) get deleted after the timeout value, or will it just re-read them into the cache from the file? That way, I could still add to the file over time. Thanks!

Answer 1

The filesystem cache in Django works like any of the other caches, when the timeout value expires, the cache is “invalidated”. In the case of files, that means it will be deleted/overwritten.

If you want long-term storage, you need to use a a long-term storage solution (Django’s cache framework is specifically not a long-term storage solution). Just save the tweets to your DB or manually to a file. You can still implement caching in addition to this, but you need to handle the long-term storage end.