Question

I currently have some code which needs to perform multiple updates per user for thousands of users, incrementing a counter depending on an action they’ve taken in order to track what actions are being performed. Each action consists of subactions which need to have the count updated too. These need to be tracked by day.

So I am storing “action”:”actionName”, “day”:day, “count”: count, for actions per day (e.g. incoming from outside web page, start game, stop game by exiting, concatenated with the game name for a lot of games).

Each day I get a few thousand rows (one per unique action) added which are updated a few hundred thousand times each day to increase the count.

The relevant code is as follows (creating array of actions not included).

$m = new Mongo();
$db = $m->actionsDB;
$collection = $db->action_count;

foreach ($arr as $action) {
    $collection->update(array("action" => $action, "day" => $day),array('$inc' => array("count" => 1)),array("upsert" => true));)
}
$collection->ensureIndex(array("action" => 1, "day" => -1));

An example of the series of updates made on an action and subactions would be:
startGame, 20110417;
startGameZork, 20110417;
startGameZorkWindows, 20110417

The problem seems to be that with this code running on the server, mongo commands in the shell get queued up.

Currently I’m unsure as to why, I guess there may be a performance issue with so many updates per second.

What I am wondering is how can I increase performance? I’m pretty new to mongo, so not entirely sure what options are available. I looked at PHP’s batchInsert but I can’t see any mention of doing batchUpdate (so instead of updating, creating an array holding all the data I currently update then doing a batchUpdate in a single trip to the DB).

Mongo driver version is 1.2.0, so persistent connections are by default.

Edit: db.serverStatus() before, during and after on ~1600 updates per second (30 seconds). Test Data

Answer 1

There is no built-in batching for updates/upserts. You can only limit the docs to be updated by adjusting your query expression and adding some further filter for “emulating” a batch somehow. MongoDB won’t help you here. Updates/Upserts are one or all.