Question

I am running a Django-based webservice with Gunicorn behind nginx as a reverse proxy.

My webservice provides a Django view which performs calculations using an external instance of MATLAB. Because the MATLAB startup takes some seconds on its own, even requests incurring only very simple MATLAB calculations require this amount of time to be answered.

Moreover, due to the MATLAB sandboxing done in my code, it is important that only one MATLAB instance is run at the same time for a webserver process. (Therefore, currently I am using the Gunicorn sync worker model at the moment which implements a pre-forking webserver but does not utilize any multithreading.)

To improve user experience, I now want to eliminate the waiting time for MATLAB startup by keeping some (e.g. 3-5) “ready” MATLAB instances running and using them as requests come in. After a request has been serviced, the MATLAB process would be terminated and a new one would be started immediately, to be ready for another request.

I have been evaluationg two ways to do this:

1. Continue using Gunicorn sync worker model and keep one MATLAB instance per webserver process.

   The problem with this seems to be that incoming requests are not distributed to the webserver worker processes in a round-robin fashion. Therefore, it could happen that all computationally-intensive requests hit the same process and the users still have to wait because that single MATLAB instance cannot be restarted as fast as necessary.

2. Outsource the MATLAB computation to a backend server which does the actual work and is queried by the webserver processes via RPC.

   In my conception, there would be a number of RPC server processes running, each hosting a running MATLAB process. After a request has been processed, the MATLAB process would be restarted. Because the RPC server processes are queried round-robin, a user would never have to wait for MATLAB to start (except when there are too many requests overall, but that is inevitable).

Because of the issues described with the first approach, I think the RPC server (approach 2) would be the better solution to my problem.

I have already looked at some RPC solutions for Python (especially Pyro and RPyC), however I cannot find an implementation that uses a pre-forking server model for the RPC server. Remember, due to the sandbox, multithreading is not possible and if the server only forks after a connection has been accepted, I would still need to start MATLAB after that which would thwart the whole idea.

Does anybody know a better solution to my problem? Or is the RPC server actually the best solution? But then I would need a pre-forking RPC server (= fork some processes and let them all spin on accept() on the same socket) or at least a RPC framework that can be easily modified (monkey-patched?) to be pre-forking.

Thanks in advance.

Answer 1

I have solved the problem by making my sandbox threadsafe. Now I can use any single-process webserver and use a Queue to get spare MATLAB instances that are spawned in a helper thread.