Question

In a django application I am working on, I have just added the ability to archive a number of files (starting 50mb in total) to a zip file. Currently, i am doing it something like this:

get files to zip
zip all files
send HTML response

Obviously, this causes a big wait on line two where the files are being compressed. What can i do to make this processes a whole lot better for the user? Although having a progress bar would be the best, even if it just returned a static page saying ‘please wait’ or whatever.

Any thoughts and ideas would be loved.

Answer 1

You should keep in mind showing the progress bar may not be a good idea, since you can get timeouts or get your server suffer from submitting lot of simultaneous requests.

Put the zipping task in the queue and have it callback to notify the user somehow – by e-mail for instance – that the process has finished.

Take a look at django-lineup

Your code will look pretty much like:

from lineup import registry
from lineup import _debug

def create_archive(queue_id, queue):
    queue.set_param("zip_link", _create_archive(resource = queue.context_object, user = queue.user))
    return queue


def create_archive_callback(queue_id, queue):
    _send_email_notification(subject = queue.get_param("zip_link"), user = queue.user)
    return queue

registry.register_job('create_archive', create_archive, callback = create_archive_callback)

In your views, create queued tasks by:

    from lineup.factory import JobFactory
    j = JobFactory()
    j.create_job(self, 'create_archive', request.user, your_resource_object_containing_files_to_zip, { 'extra_param': 'value' })

Then run your queue processor (probably inside of a screen session):

./manage.py run_queue

Oh, and on the subject you might be also interested in estimating zip file creation time. I got pretty slick answers there.