Question

I have a Java Google App Engine web application that allows user upload of images. Locally, it works great. However, once I deploy it to “the cloud”, and I upload an image, I get the following error:

java.lang.IllegalArgumentException: can’t operate on multiple entity groups in a single transaction.

I use the blobstore to store the images (Blobstore Reference). My method is below:

    @RequestMapping(value = "/ajax/uploadWelcomeImage", method = RequestMethod.POST)
@ResponseBody
public String uploadWelcomeImage(@RequestParam("id") long id,
        HttpServletRequest request) throws IOException, ServletException {

    byte[] bytes = IOUtils.toByteArray(request.getInputStream());
    Key parentKey = KeyFactory.createKey(ParentClass.class.getSimpleName(),
            id);
    String blobKey = imageService.saveImageToBlobStore(bytes);
    imageService.save(blobKey, parentKey);

    return "{success:true, id:\"" + blobKey + "\"}";
}

You’ll notice that this method first calls “imageService.saveImageToBlobStore”. This is what actually saves the bytes of the image. The method “imageService.save” takes the generated blobKey and wraps it in an ImageFile object, which is an object that contains a String blobKey. My website references imageFile.blobKey to get the correct image to display. The “saveImageToBlobStore” looks like this:

@Transactional
public String saveImageToBlobStore(byte[] bytes) {
    // Get a file service
    FileService fileService = FileServiceFactory.getFileService();

    // Create a new Blob file with mime-type "text/plain"
    AppEngineFile file = null;
    try {
        file = fileService.createNewBlobFile("image/jpeg");
    } catch (IOException e) {
        e.printStackTrace();
    }

    // Open a channel to write to it
    boolean lock = true;
    FileWriteChannel writeChannel = null;
    try {
        writeChannel = fileService.openWriteChannel(file, lock);
    } catch (FileNotFoundException e) {
        e.printStackTrace();
    } catch (FinalizationException e) {
        e.printStackTrace();
    } catch (LockException e) {
        e.printStackTrace();
    } catch (IOException e) {
        e.printStackTrace();
    }

    // This time we write to the channel using standard Java
    try {
        writeChannel.write(ByteBuffer.wrap(bytes));
    } catch (IOException e) {
        e.printStackTrace();
    }

    // Now finalize
    try {
        writeChannel.closeFinally();
    } catch (IllegalStateException e) {
        e.printStackTrace();
    } catch (IOException e) {
        e.printStackTrace();
    }

    // Now read from the file using the Blobstore API
    BlobKey blobKey = fileService.getBlobKey(file);
    while (blobKey == null) { //this is hacky, but necessary as sometimes the blobkey isn't available right away
        try {
            Thread.sleep(2000);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        blobKey = fileService.getBlobKey(file);
    }

    // return
    return blobKey.getKeyString();
}

My other save method looks like this:

public void save(String imageFileBlobKey, Key parentKey) {
    DatastoreService datastore = DatastoreServiceFactory
            .getDatastoreService();

    Entity imageFileEntity = new Entity("ImageFile", parentKey);
    imageFileEntity.setProperty("blobKey", imageFileBlobKey);

    datastore.put(imageFileEntity);
}

Like I said before, it works locally, but not deployed. The error is on the call to saveImageToBlobstore, specifically on the “fileservice.getBlobKey(file)”. Commenting out this line eliminates the error, but I need this line to save the bytes of the image to the blob store.

I also tried commenting other lines (see below), with no luck. Same error for this:

@RequestMapping(value = "/ajax/uploadWelcomeImage", method = RequestMethod.POST)
@ResponseBody
public String uploadWelcomeImage(@RequestParam("id") long id,
        HttpServletRequest request) throws IOException, ServletException {

    //byte[] bytes = IOUtils.toByteArray(request.getInputStream());
    //Key parentKey= KeyFactory.createKey(ParentClass.class.getSimpleName(),
            //id);
    byte[] bytes = {0,1,0};
    String blobKey = imageService.saveImageToBlobStore(bytes);
    //imageService.save(blobKey, parentKey);

    return "{success:true, id:\"" + blobKey + "\"}";
}

Any ideas? I am using GAE 1.5.2. Thank you!

UPDATE, SOLUTION FOUND:
I took some code out of the transactional “saveImageToBlobStore” and moved it up a level. See below:

@RequestMapping(value = "/ajax/uploadWelcomeImage", method = RequestMethod.POST)
@ResponseBody
public String uploadWelcomeImage(@RequestParam("id") long id,
        HttpServletRequest request) throws IOException, ServletException {

    byte[] bytes = IOUtils.toByteArray(request.getInputStream());
    Key parentKey = KeyFactory.createKey(ParentClass.class.getSimpleName(),
            id);

            //pulled the following out of transactional method:
    AppEngineFile file = imageService.saveImageToBlobStore(bytes);
    FileService fileService = FileServiceFactory.getFileService();
            //code below is similar to before//////////////
    BlobKey key = fileService.getBlobKey(file);
    String keyString = key.getKeyString();
    imageService.save(keyString, parentKey);

    return "{success:true, id:\"" + keyString + "\"}";

Answer 1

From the Docs:

When the application creates an entity, it can assign another entity as the parent of the new entity. Assigning a parent to a new entity puts the new entity in the same entity group as the parent entity.

Also:

All datastore operations in a transaction must operate on entities in the same entity group.

So, you have to choose:

• at one extreme, you can create a single ‘root’ entity (without any parent) set this as parent to everything else. You’ll be able to use transactions in any way you want; but there’s a limit on how many operations per second can happen on each entity group. This strategy limits your scalability.

• At the other extreme, you can put every entity alone on it’s own group. Simply make every entity a root entity, that is, without any parent. This gets you the maximum scalability, since the underlying system is free to distribute the load evenly among a great number of machines. Unfortunately, this means you wouldn’t be able to use transactions. You’ll have to think carefully about concurrent consistency and avoid race conditions.

• The middle point requires some planning: think what processes would benefit from being enclosed in transactions. Then define what entities would participate on the transaction. Then be sure to put all these into the same group when created.

A simple example is to use an entity group for each user; the main user record would be a root entity, and everything else that is related to this user would indicate that root entity as it’s parent. With this design, any intra-user operation can be enclosed into a transaction; but any inter-user operation wouldn’t.

That seems overly restricting, but you can design your way out. For example, you could define any user-user relation into a new entity group, not belonging to either of the user’s groups. Or use batch processing for everything that would cross between entity groups; if you’re out of the HTTP request/response, you can control concurrency and so avoid many kinds of race conditions.