The project I am working on at the moment basically takes in an image and then renders a video using blender from the command line. At the moment I am using Twisted to deal with the requests but there is certainly something that I am doing wrong as it is not working how I would like it to. You can see the jist of the program here (I have stripped out anything unnecessary).
The blender render is done by spawning a subprocess (I am aware Twisted can handle processes) along with a python script to configure the render and use the image provided as a texture.
The program needs to be able to handle as many connections as possible. At the moment the subprocess does one render at a time but ideally it would check CPU/number of parallel renders and adjust the number to the optimum. Each render is custom to the user so once a users render is complete they should get their render back (an avi file).
My question is: Is Twisted the right choice for this? Are there any other options? If not, is my implementation of the system flawed? I would appreciate any thoughts or opinions on this!
Is Twisted the right choice for this?– Perhaps.Are there other options?– Yes.If not, is my implementation of the system flawed?– Yes. It looks to me that your subprocess call is blocking:p.wait()It is possible to do what it sounds like you’re trying to do in Twisted, but you are a very long way from it.
You need a rate-controlled, asynchronous task queue with a web frontend.
What you’ve got is single page on a single threaded ‘site’ that doesn’t return any html until the submitted job is finished.
This is possible in twisted. However it’s probably easier to implement using django + celery.
Django: http://www.djangoproject.com/
Celery: http://celery.github.com/celery/getting-started/introduction.html
And a tutorial for a similar purpose:
http://webcookies.org/blog/2009/09/10/rabbitmq-celery-and-django/