Question

I have implemented an online leaderboard via Google App Engine for my Android app. But after 2 hours I reached 100% of my quotas in “Datastore Read Operations”. Can anybody help me to modify my code to reduce the read operations?
Here is my code:

public class The_Big_Bang_Theory_Quiz_HighscoreserverServlet extends HttpServlet {
public void doGet(HttpServletRequest req, HttpServletResponse resp) throws IOException {
    String game = req.getParameter("game");
    String name = req.getParameter("name");
    String pointsStr = req.getParameter("points");
    String behaviorStr = req.getParameter("behavior");
    int behavior = 0; // 0 = upload, 1 = download
    if (behaviorStr != null) {
        try {
            behavior = Integer.parseInt(behaviorStr);
        } catch (NumberFormatException e) {
            behavior = 0;
        }
    }
    if (behavior == 0) {
        int points = 0;
        if (pointsStr != null) {
            try {
                points = Integer.parseInt(pointsStr);
            } catch (NumberFormatException e) {
                points = 0;
            }
        }
        if (points > 0 && name != null) {
            addHighscore(game, name, points);
        }
    } else {
        String maxStr = req.getParameter("max");
        int max = 1000;
        if (maxStr != null) {
            try {
                max = Integer.parseInt(maxStr);
            } catch (NumberFormatException e) {
                max = 1000;
            }
        }
        returnHighscores(resp, game, max);
    }
}

private void returnHighscores(HttpServletResponse resp, String game, int max) {
    DatastoreService datastore = DatastoreServiceFactory.getDatastoreService();
    Key gameKey = KeyFactory.createKey("game", game);
    Query query = new Query("highscore", gameKey);
    query.addSort("points", Query.SortDirection.DESCENDING);
    List<Entity> highscores = datastore.prepare(query).asList(FetchOptions.Builder.withLimit(max));
    for(Entity e : highscores) {
        try {
            resp.getWriter().println(e.getProperty("name") + ";" +e.getProperty("points"));
        } catch (IOException exc) {
            exc.printStackTrace();
        }
    }
}

private void addHighscore(String game, String name, int points) {
    DatastoreService datastore = DatastoreServiceFactory.getDatastoreService();
    Key gameKey = KeyFactory.createKey("game", game);
    Entity highscore = new Entity("highscore", gameKey);
    highscore.setProperty("name", name);
    highscore.setProperty("points", points);
    datastore.put(highscore);
}
}

I read something about the BlobStore. Is it a better method?

Answer 1

I had the same problem, i used the cache mechanism from GEA to solve the problem.
Basicly the Cache is a Distributed HasMap

some code:
create the Map:

try {
            cache = CacheManager.getInstance().getCacheFactory().createCache(
                    new ConcurrentHashMap<String, Category>());
        } catch (CacheException e) {
            Logger
                    .getLogger(TipsDAO.class.getName())
                    .severe(
                            "unable to cretate cache using an internal ConcurrentHashMap");
            cache = new ConcurrentHashMap<String, Category>();
        }

For every read pop you check the Map first, if you find i there you return, if you don’t find it you read from the DB and put it in the Map before you return.

if (cache.containsKey(cat)) {
            return (Category) cache.get(cat);
        }
        try {
            Query query = entityManager
                    .createQuery("SELECT FROM Category WHERE name = ?1");
            query.setParameter(1, cat);
            Category temp = (Category) query.getSingleResult();
            cache.put(cat, temp);
            return temp;
        } catch (Exception e) {
            LOG.severe(e.getMessage());
            return null;
        }

For every write op to the DB you also write to the Map

cache.put(cat.getName(), cat);