I’m developing a client-server app to distribute and execute code in multiple clients. This is how it must to work:
- Clients connect to the server sending their status each minute.
- Server store clients IPs and status.
- Someone send a code to be executed to the server.
- The server searches for a free and active client and sends the code.
My problem with this is the amount of clients that I can handle. I was thinking in use lists to store clients objects or something similar but I could have thousands of clients so maybe is not a good idea because I’ll run out of resources. Which could be the best option to store all clients data efficiently without hang the server?
Sorry for not to show any code but I’m in the designing step currently and I’m stuck with this. The project will be developed using Python but I don’t need code.
If you are just worried about not having enough memory to keep all the Python data, you could pickle it and keep track of eg just the ID and the pickled file, Python Docs Pickle.
Another good example is Shelve.
The upside to both of these is not having to worry about serializing or modelling your data on eg an SQL database.
Of course, you could go for a combination of the two. Pickle the data and save the resulting string in a field in a database. This way, you can simply add columns with meta data if you need in the future. When it comes to the speed consideration, you either keep it in a database so you can query it nicely, or keep it as files in order to get rid of that “overhead”. Be careful about premature optimization.