Question

Suppose I have this kind of query

String sql = "SELECT s.team_id, s.team_name, s.gp, s.w, s.t, s.l, s.go, s.ga, s.score, s.p FROM "
           + "(SELECT team_id, team_name, SUM (gp) gp, SUM (w) w, SUM (t) t, SUM (l) l, SUM (GO) go, SUM (GA) ga, SUM (GO)- SUM (GA) score, SUM (2*w+t) p FROM "
           + "(SELECT t._id team_id, t.name team_name, COUNT(CASE WHEN score_home IS NOT NULL THEN 1 END) gp, COUNT (CASE WHEN score_home > score_away THEN 1 END) w,"
           + " COUNT (CASE WHEN score_home = score_away THEN 1 END) t, COUNT (CASE WHEN score_home < score_away THEN 1 END) l,"
           + " SUM (score_home) go, SUM (score_away) ga"
           + " FROM team_table t LEFT OUTER JOIN match_table m ON m.team_home = t._id"
           + " WHERE t.tournament_id = ? GROUP BY t._id, t.name"
           + " UNION ALL"
           + " SELECT t._id team_id, t.name team_name, COUNT(CASE WHEN score_away IS NOT NULL THEN 1 END) gp, COUNT (CASE WHEN score_home < score_away THEN 1 END) w,"
           + " COUNT (CASE WHEN score_home = score_away THEN 1 END) t, COUNT (CASE WHEN score_home > score_away THEN 1 END) l,"
           + " SUM (score_away) go, SUM (score_home) ga"
           + " FROM team_table t LEFT OUTER JOIN match_table m ON m.team_away = t._id"
           + " WHERE t.tournament_id = ? GROUP BY t._id, t.name)"
           + " GROUP BY team_id, team_name) s"
           + " ORDER BY s.p DESC, s.score DESC, s.go ASC";

which is then used like this

Cursor cursor = database.rawQuery(sql, args);

cursor.moveToFirst();
while (!cursor.isAfterLast()) {
    TeamStats stat = new TeamStats();

    stat.setTeamId(cursor.getLong(0));
    stat.setTeamName(cursor.getString(1));
    stat.setGamesPlayed(cursor.getInt(2));
    stat.setWins(cursor.getInt(3));
    stat.setTies(cursor.getInt(4));
    stat.setLoses(cursor.getInt(5));
    stat.setGoalsOwn(cursor.getInt(6));
    stat.setGoalsAgaist(cursor.getInt(7));
    stat.setScore(cursor.getInt(8));
    stat.setPoints(cursor.getInt(9));

    stats.add(stat);
    cursor.moveToNext();
}
cursor.close();

So it selects values from many tables, does some operations etc. As you can see the query is horribly complex (very difficult to debug) and the performance does not seem to be as good as I would expect. My questions are:

1. Can I improve performace using some kind of prepared statement?
2. Would it be faster to perform more simple queries and process them manually with some custom code?

Answer 1

If I were you, I would copy your sqlite database to host, then try to execute it manually in some SQLite GUI while replacing bound variables (?) with actual variable values you have. For GUI on Windows, I really like SQLite Expert Personal, and on Linux sqliteman is pretty good.

While debugging your SQL (in command line or GUI), be sure to analyze your SQL statements by running them under EXPLAIN and/or EXPLAIN QUERY PLAN. Watch out for table scans. You should try to eliminate expensive scans by adding indexes. But don’t index everything – it may make things worse.
Often, you can have big performance gains by using compound (multi-column) indexes. Note, that on any given table SQLite cannot make use of more than just one index (while running given SQL statement) – so, choose your indexes wisely. (See also the basic explanation in Query Planning.)

And to address your concerns about data processing in Java vs. SQLite – I think that fully optimized (with proper indexes, etc.) SQLite query against relational data will (almost) always going to be faster than manual processing of this data in Java. This must be especially true in your case – all your data is basically relational.

One small note though: Your Android APK using Java may have access to more memory than SQLite does by default – you may want to increase SQLite cache size for your database using setMaxSqlCacheSize() (equivalent of PRAGMA cache_size). Android default is 10 (max 100), try increasing it and see if makes any difference for your query. Note that desktop SQLite default for this setting is much higher – 2000.