I have a multi-stage process that needs to be run at some intervals.
I also have a Controller program which starts the process at the right times, chains together the stages of the process, and checks that each stage has executed correctly.
The Controller accesses a database which stores information about past runs of the process, parameters for future executions of the process, etc.
Now, I want to use Pyramid to build a web interface to the Controller, so that I can view information about the process and affect the operation of the Controller.
This will mean that actions in the web interface must effect changes in the controller database.
Naturally, the web interface will use the exact same data models as the Controller.
What’s the best way for the Controller and Web Server to interact?
I’ve considered two possibilities:
- Combine the controller and web server by calling sched in Pyramid’s initialisation routine
- Have the web server make RPCs to the controller, e.g. using Pyro.
How should I proceed here? And how can I avoid code duplication (of the data models) when using the second option?
I would avoid running your Controller in the same process as the web application – it is a common practice to run web-applications with lowered permissions, for example; in some multi-threaded/multi-process environment which may spawn multiple workers and then possibly kill/recycle them whenever it feels like doing so. So having your controller running in a separate process with some kind of RPC mechanism seems like a much better idea.
Regarding code duplication – there are 2 options:
you can extract the common code (models) into a separate module/egg which is used by both applications
if you’re finding that you need to share a lot of code – nothing forces you to have separate projects for those applications at all. You can have a single code base with two or more “entry points” – one of which would start a Pyramid WSGI application and another would start your Controller process.