I’m hoping to write a custom data model for my web application that exists in memory as a collection of custom python objects. The web application could manipulate data on these objects and run their methods on user requests, and after a request thread dies the objects persist in memory.
The model objects would have methods to write themselves to disk (not sure how I will do this yet but that’s not what I’m interested in here), but only for backup purposes. Ideally they will persist in memory as long as the server runs.
Access to the model would have to be managed by a session in the case that multiple users are manipulating data on the objects at the same time, though this occurrence will be relatively rare.
It is not an option to initialise the objects with the web application, as they must persist even on a reset of the web app.
Are there any sort of 3rd party components that would make setting up this sort of model easier? Any advice for going down this path? Any help would be appreciated, as I don’t have much experience with programming independent processes to communicate in python or otherwise.
My server will be Linux and I’m planning on using Pyramid as a framework for the web application.
Thanks in advance,
Will
I think this mostly depends on how much work you want to put into what looks like building your own ORM.
If you’re feeling up to the task of serializing your python objects, and depending on the complexity of your models, you could give a shot at Redis.
Redis is a non-relational database, based on a key-value model. All your data is stored in memory, and is periodically (frequency depends on number of writes and is customizable) dumped to the disk. It’s also pretty easy to pick up and use.
There’s a wrapper around redis that is available for python: redis_py.
Of course, Redis is not the only of its kind, and you could also look at similar alternatives such as Tokyo Tyrant.