Question

I have the need to create a system to store python data structures on a linux system but have concurrent read and write access to the data from multiple programs/daemons/scripts. My first thought is I would create a unix socket that would listen for connections and serve up requested data as pickled python data structures. Any writes by the clients would get synced to disk (maybe in batch, though I don’t expect it to be high throughput so just Linux vfs caching would likely be fine). This ensures only a single process reads and writes to the data.

The other idea is to just keep the pickled data structure on disk and only allow a single process access through a lockfile or token… This requires all accessing clients to respect the locking mechanism / use the access module.

What am I over looking? SQLite is available, but I’d like to keep this as simple as possible.

What would you do?

Answer 1

You could serialize the data structures and store them as values using ConfigParser. If you created your own access lib/module to the access the data, you could do the serialization in the lib so the client code would just send and receive python objects. You could also handle concurrency in the lib.