Question

I’ve been following the app engine JDO documentation as closely as possible, but I’m having bizzare and inconsistent trouble loading a persistent collection that is contained by my Board object. The inconsistency occurs even in the local Development Webserver after I manually specify “eventual consistency” as being nonexistent.

Sometimes when I load my object/collection using the loading helper methods I’ve created, it loads no problem. Other times an empty collection is returned (please note that I am “touching” the collection with a getter method to ensure the data is not just a lazy loaded into a proxy object).

Initially I thought the problem was simply related due to the High Replication Storage engine’s “eventual consistency” drawbacks, but after making my own policy with 0% eventual consistency in the LocalServiceTestHelper, I’m fairly certain that’s not the case.

I’ve created a JUnit test that exemplifies this problem. Basically, I attempt to create and save a dummy User and Board object in the testInsertUser function. I attach a newly created ArrayList of PlayedTile objects to this board, and then execute a DataMaster.saveUser helper method which uses Google App Engine’s persistence manager to save the User (and thus the Board and PlayedTile collection) to the datastore. In the next method, we attempt to load that User (with its Board and PlayedTile collection) and display those saved results. Chaos ensues.

Here is the JUnit code:

package com.astar.wordswall.test.data;

import java.util.ArrayList;

import com.astar.wordswall.data.DataMaster;
import com.astar.wordswall.data.jdo.Board;
import com.astar.wordswall.data.jdo.User;
import com.astar.wordswall.data.jdo.PlayedTile;
import com.astar.wordswall.test.appengine.LocalCustomHighRepPolicy;
import com.google.appengine.api.datastore.Key;

import com.google.appengine.tools.development.testing.LocalDatastoreServiceTestConfig;
import com.google.appengine.tools.development.testing.LocalServiceTestHelper;
// import com.google.gwt.user.client.Random;

import org.junit.After;
import org.junit.Before;
import org.junit.Test;


public class SaveUsersBoardWithTilesTest {

    Key userKey;

    private final LocalServiceTestHelper helper =
        new LocalServiceTestHelper(new LocalDatastoreServiceTestConfig()
            .setAlternateHighRepJobPolicyClass(LocalCustomHighRepPolicy.class));

    @Before
    public void setUp() {
        helper.setUp();
    }

    @After
    public void tearDown() {
        helper.tearDown();
    }

    @Test
    public void testInsert1() {
        testInsertUser();
        testReadUser();
    }

    /**
     * Creation and insertion of a user, board, and linked set of tiles into BigTable.
     */
    private void testInsertUser() {
        User u = new User("Simon");
        Board b = new Board();
        ArrayList<PlayedTile> tiles = new ArrayList<PlayedTile>(7);

        u.setBoard(b);
        b.setPlayedTiles(tiles);

        for (int j = 0; j < 7; j++) tiles.add(new PlayedTile('T'));

        DataMaster.saveUser(u);
        // Retrieve the user's key so that we can read him from the database later
        userKey = u.getUserKey();

        // Display all of our saved tiles:
        System.out.println("Saved tiles:");
        // Note that "getTileString()" just iterates through each Played tile printing the letter
        System.out.println("\t" + u.getBoard().getTileString());
    }

    /**
     * A typical read of a user object from the Datastore.
     */
    private void testReadUser() {
        User u = DataMaster.getUserWithBoard(userKey);
        // Display all of our saved tiles:
        System.out.println("Loaded tiles:");
        System.out.println("\t" + u.getBoard().getTileString());
    }
}

And here’s the relevant DataMaster.getUserWithBoard static function that actually performs the JDO loading:

/**
 * Loads a uniquely specified User and their associated board from 
 * the Datastore.  It also loads the board's complete list of PlayedTiles.
 * @param userKey the unique key assigned to this user
 */
public static User getUserWithBoard(Key userKey){
    User u = null;
    PersistenceManager pm = PMF.get().getPersistenceManager();
    try{
        u = pm.getObjectById(User.class, userKey);
        // In order for the board and tile collection to load, we must "touch" it while PM is active
        if (u.getBoard().getPlayedTiles().size() != 0) u.getBoard().getPlayedTiles().get(0);
        if (u.getBoard().getPlayedWords().size() != 0) u.getBoard().getPlayedWords().get(0);
    } finally{
        pm.close();
    }
    return u;
}

Oddly enough, this code SOMETIMES works as expected: it prints out the exact same set of tiles it saves after it loads them from the datastore in testReadUser(). SOMETIMES it just loads an empty collection, although what’s particularly bizzare is that the u.getBoard().getPlayedWords().get(0) call doesn’t throw a null pointer exception.

The output oscillates between

Correct:

Saved tiles:
    T T T T T T T 
Loaded tiles:
    T T T T T T T 

And Incorrect:

Saved tiles:
    T T T T T T T 
Loaded tiles:

totally haphazardly.

Can anybody out there shine some light on this? It’s driving me completely crazy. 🙂

EDIT: Another bizzare clue/factoid is that if I make the whole test iterate by enclosing both the testSaveUser and testReadUser method calls in a for loop, either EVERY load operations executes correctly, or NONE of them do. Is this a bug in the Local Google App Engine testing environment?

Answer 1

Just to check: Did you mark your data classes as detachable? I.e.

@PersistenceCapable(detachable = "true")
public class Board { /* fun stuff here */ }

Also, it might be helpful to look at the datastore viewer to see if really worked or not: http://localhost:8888/_ah/admin/