Home/ Questions/Q 7035501
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-28T01:18:42+00:00

I am new to web development, and I am working on a basic image

  • 0

I am new to web development, and I am working on a basic image gallery app (learning exercise) using Django. I have it set up so I can upload a zip full of images all at once to create a new album. This all seems to work OK, but I am getting an HTTP 504 error when the uploaded file is especially large.

I gather (please correct me if I am wrong) this error means my app is too slow to return an HTTP response. I am guessing this is because it takes a long time to unzip and process (create a Pic object in the DB and create thumbnails) all the images.

Is there a way to return a response (say to some intermediate page) while still performing the processing in the background – maybe using threads? What is the proper way to handle this? Is it time for me to start learning Javascript/AJAX?

Thank you!

Models:

from django.db import models
from blog.models import Post

class Album(models.Model):
    title = models.CharField(max_length=128)
    slug = models.SlugField()
    description = models.TextField()
    parent = models.ForeignKey('self', null=True, blank=True)

    pub = models.BooleanField()
    date_created = models.DateTimeField(auto_now_add=True)
    date_published = models.DateTimeField(null=True, blank=True)
    date_modified = models.DateTimeField(auto_now=True)

    def __unicode__(self):
        return self.title

class Pic(models.Model):
    image = models.ImageField(upload_to='pics/%Y/%m') 
    title = models.CharField(max_length=128)
    caption = models.TextField(blank=True, null=True)
    albums = models.ManyToManyField('Album', null=True, blank=True)
    posts = models.ManyToManyField(Post, blank=True, null=True)

    date_taken = models.DateTimeField(null=True, blank=True) 
    date_uploaded = models.DateTimeField(auto_now_add=True) 
    date_modified = models.DateTimeField(auto_now=True)

    def __unicode__(self):
        return self.title

View:

I’m doing this manually because I didn’t grok the Django admin when I started. I think it might be better to use admin customization here.

def new_album(request):
    if request.method == "POST":
        form = AlbumForm(request.POST, request.FILES)
        if form.is_valid():
            from gallery.pic_handlers import handle_uploaded_album
            pics = handle_uploaded_album(request.FILES['pic_archive'])
            a = form.save()
            a.slug = slugify(a.title)
            a.save()
            for pic in pics:
                pic.albums.add(a)
            return HttpResponseRedirect('/gallery/album/%s/' % a.slug)
    else:
        form = AlbumForm()

    return render_to_response('new_album.html', {
        'form' : form,
    }, context_instance = RequestContext(request))

Additional processing:

def handle_uploaded_album(pic_archive):
    destination = open(join(settings.MEDIA_ROOT,pic_archive.name), 'wb+')
    for chunk in pic_archive.chunks():
        destination.write(chunk)
    destination.close()

    today = datetime.date.today()
    save_path = 'pics/{0}/{1:02}/'.format(today.year, today.month)
    tmp_path = 'tmp/'
    z = zipfile.ZipFile(join(settings.MEDIA_ROOT,pic_archive.name), 'r')
    pics = []
    for member in z.namelist():
        if '/' in member or '\\' in member: 
            # don't deal with any directories inside the zip
            # this also solves the '__MACOSX' issue
            continue
        if splitext(member)[1] in IMG_EXT:
            z.extract(member,join(settings.MEDIA_ROOT,tmp_path))
            im = File(open(join(settings.MEDIA_ROOT,tmp_path,member), 'rb'))
            # create a Pic from this file
            pic = Pic()
            pic.title = member
            pic.image.save(
                join(save_path, member),
                im,
                True)
            create_thumbnails(pic)
            im.close()
            # remove extracted images
            remove(join(settings.MEDIA_ROOT,tmp_path,member))

            # TODO: save date taken if available
            pics.append(pic)

    z.close()
    remove(join(settings.MEDIA_ROOT,pic_archive.name))

    return pics

def create_thumbnails(pic):
    fname, ext = splitext(pic.image.path)
    img = Image.open(pic.image.path)

    img.thumbnail((512,512), Image.ANTIALIAS)
    img.save(fname + '_m' + ext)

    img.thumbnail((128,128), Image.ANTIALIAS)
    img.save(fname + '_s' + ext)
Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Long tasks such as this processing take too much time, and your client and/or your proxy will timeout – which is the 504 error you see.

    you should not run long jobs this way!

    As you correctly ask at the end, you need a way to detach long executions – via an asynchronous queue system, such as celery. This way you can return an answer straight away to your clients, while the backend runs jobs asynchronously.

    you should look at one of the followings:

    Since django-celery is definitely the best option, next step is to learn about it; there are also a lot of SO questions around it

    If you want to be sure it’s processing that fails, and not upload, simply try disabling all processing and returning straight away to your client. If it’s still failing, you also need to tweak your webserver so that it’s not going in timeout!

    • 0