Question

Original problem

I want to run queries on database from within Twisted async app. In example to add new record, update something or just get some data.

My proposal

I think about making special views in django that only Twisted app can access, which will do all sort of operations on database I need.

In example url domain.com/server/getuserdata/{{ user_id }}/ would return data about user in some convenient format (json).

GET/POST request would be deffered in Twisted so it won’t block my Twisted app (correct me if I’m wrong).

My Twisted app would only run those requests once in a while. It’s not realtime updating data. Realtime data will be stored in Redis and once I’m ready to store it Django database I run request to Django using data stored in redis.

But still there can be lot of requests from server per second (lets say from 100 to 1000).

Another approaches I read about

• twisted nadbapi, but it’s not updated since 2007 so I dont want to use it honestly
• storing requestes in Redis and run them by another deamon/app or cron.

Questions

• Is using deffered objects for GET/POST requests to Django a good approach?
• Can I somehow tell Django that Twisted app is trusted, so it won’t treat it like a DDOS?
• Will this be fast enough with so many requests (using cache of course)?
• Is there any better approach of this task?

I’m new to Twisted and intergrating async app with sync app, so I’m sorry if I typed something stupid

Answer 1

Is using deffered objects for GET/POST requests to Django a good
approach?

I wish i had the experience to answer this one in depth. I believe that as long as the request is asynchronous you are going to be fine on the twisted side. I am basing this on my basic experirience with tornado. It is relativly common for an app to have to request a webservice and tornado provides an async way of doing this built in to the framework.

Can I somehow tell Django that Twisted app is trusted, so it won’t
treat it like a DDOS?

I don’t think django inherently knows what service is requesting it. It doesn’t throttle speed or requests out of the box. It just responds to whatever requests are made to it. If these are more requests that it can handle it will crash. HOw many request django can handle is variant on your project and how you deploy it. You could run it using gunicorn easily and have nginx sit in front of it. If it is not performing as it shoudl you can start another gunicorn instance.

Will this be fast enough with so many requests (using cache of
course)?

I don’t think anyone can say. YOu might have to benchmark it yourself. There are a number of blog posts/articles that benchmark various django server configurations.

Is there any better approach of this task?

I think this is a relatively common approach. Expose your methods via django through a RESTful interface. Make async reqeusts from twisted.