Question

I’m using this plugin to upload multiple files with jquery, and this project is for Django, but now I’m trying to use App Engine datastore.

I had success deploying and it works until I try to upload, an error message shows in server log:

ValueError: The App Engine storage backend only supports BlobstoreFile
instances or File instances whose file attribute is a BlobstoreFile.

I believe that the error is because in models.py it uses django’s model class, which is models.FileField, but AppEngine requires db.BlobProperty().

Here is the project link: https://github.com/sigurdga/django-jquery-file-upload

Here is my models.py file:

from django.db import models

class Picture(models.Model):

    # This is a small demo using FileField instead of ImageField, not
    # depending on PIL. You will probably want ImageField in your app.
    file = models.FileField(upload_to="pictures")
    slug = models.SlugField(max_length=50, blank=True)

    def __unicode__(self):
        return self.file

    @models.permalink
    def get_absolute_url(self):
        return ('upload-new', )

    def save(self, *args, **kwargs):
        self.slug = self.file.name
        super(Picture, self).save(*args, **kwargs)

Any idea to fix it? Thanks.

Answer 1

Ok there are a number of things worth noting here, which go right to the core of how App Engine works.

Django’s Model layer is only designed to work with relational databases, like MySQL, SQLite, Postgres, Oracle, etc.

App Engine’s Datastore is non-relational, so doesn’t work with Django Models natively. You could, however, use Django-nonrel, which acts as a translation layer between Django models and non-relational databases, like the Datastore.

Unfortunately it’s still not quite as simple as that because the Django FileField doesn’t exactly fit the AppEngine BlobProperty. There are workarounds, but the Datastore isn’t great for serving images anyway.

Storing the image

Google recommend 2 ways of storing and serving files, such as images: the Blobstore and Google Cloud Storage.

Ultimately, the best way to store uploaded images is to store them in one or other of these and then record a link to the image on your model. So if you changed your model to:

class Picture(models.Model):

    file_url = models.URLField()
    slug = models.SlugField(max_length=50, blank=True)

    ...

In your view, use the techniques described here for uploading a file to the Blobstore, then save the upload_url on the model as the file_url field.

You could do the same with Google Cloud Storage, although as the Blobstore is actually part of App Engine (rather than than an API into another service) it gives you various benefits, described below.

Note that, as described above, this will only work if you use Django-nonrel, otherwise you won’t be able to save your Django model to the Datastore.

Storing the model

With Django-nonrel, you can just save the model above in the Datastore as-is. However, Django-nonrel has a number of caveats which can take getting used to and it can be quite slow. I personally wouldn’t recommend it for new projects, but others may disagree.

Alternatively, you could sidestep Django’s model layer and use App Engine’s own Model layer, which is designed to fit perfectly with the Datastore. It includes a specific property type for referencing objects in the Blobstore. Your model might look like this:

from google.appengine.ext.blobstore import blobstore
from google.appengine.ext import db

class Picture(db.Model):

    file = blobstore.BlobReferenceProperty()
    slug = db.StringProperty(required=False)

    ...

The BlobReferenceProperty gives you a lot more power than just storing a URL, because you can access the file data itself via this property, to get data about the file (size, etc) or process it. If you’re happy just storing the URL, there’s also a db.LinkProperty which is similar to Django’s models.URLField.

Of course using App Engine models instead of Django models loses a lot of the benefits of Django’s model layer and the close integration it offers with Django.

Google Cloud SQL

There is another option if you’d like to use Django models but don’t want to use Django-nonrel. If you’re happy to forgo the benefits of the non-relational Datastore, you could use Google Cloud SQL instead. Django supports Cloud SQL natively. However, Google Cloud SQL is currently in limited preview so you might find it hard to get access.

If you do decide to use Cloud SQL, you should still store your images in the Blobstore or Google Cloud Storage as discussed above if you wish to serve them publicly.