I’m writing a python application to manage a cluster of linux machines and I’m looking for an efficient way to send commands to all the hosts.
The general architecture of the application is:
- Management interface
- Command dispatcher
- Nodes
I started to write the app using a simple python script on local nodes and then invoking it from SSH (thus using shell commands to dispatch commands) but I’m looking for a more efficient and native solution.
It would be great if I could execute python code remotely without using SSH and pass (pickled) python objects around.
Consider that it should be able to communicate to several hundred of hosts over the network and support SSL/TLS.
As a reference consider the VmWare vCenter architecture, as my goal is to create something very similar, so what kind of approach/technology you would use?
Like @malangi, I was going to suggest Salt as well. If you really want to roll your own solution rather than reuse Salt, you could use ZeroMQ as the comms backend (this is what Salt does). ZeroMQ is super fast and hides all the network comms from you behind a really nice and simple, yet powerful socket API.
It looks like ZeroMQ doesn’t do SSL/TLS connections yet: see this SO question. But have a look at this page on the ZeroMQ wiki for alternatives.