Question

I have the nginx upload module handling site uploads, but still need to transfer files (let’s say 3-20mb each) to our cdn, and would rather not delegate that to a background job.

What is the best way to do this with tornado without blocking other requests? Can i do this in an async callback?

Answer 1

advice on the tornado google group points to using an async callback (documented at http://www.tornadoweb.org/documentation#non-blocking-asynchronous-requests) to move the file to the cdn.

the nginx upload module writes the file to disk and then passes parameters describing the upload(s) back to the view. therefore, the file isn’t in memory, but the time it takes to read from disk–which would cause the request process to block itself, but not other tornado processes, afaik–is negligible.

that said, anything that doesn’t need to be processed online shouldn’t be, and should be deferred to a task queue like celeryd or similar.