Question

I’m connecting to MySQL db from two different PHP instances. The first thread adds a new row, passes the ‘id’ of new row to the second thread via a queue. Sometimes the second thread is unable to find the new row data, even though theoretically it should hit the db after first threads has finished its job.

The simplified pseudocode looks something like

Thread 1

$db = get_mysql_connection();
$db->beginTransaction();
$rowid = $db->query("insert data..");
$db->commit();

//For Debugging purposes only
$db->check_if_row_exists($rowid); //Always returns true

send_to_queue($rowid);

Thread 2

$rowid = fetch_from_queue();
$db = get_mysql_connection();
$db->check_if_row_exists($rowid); //Sometimes returns false;
usleep(1000000);
$db->check_if_row_exists($rowid); //Always returns true.

I cannot understand, why thread 1 shows that data has a valid entry, while thread 2, which certainly makes a query sometime after thread 1, is unable to find the data. I’m using transactions to commit the data, is that doing something weird?

I’m using Gearman as the queue. Thread 1 is run through Apache, while Thread 2 simply runs as a standalone process.

Edit 1:
This happens when thread 2 is running concurrently to thread 1. Apparently it reaches some race condition with Thread 1, but I can’t figure out why.

Edit 2:
As pointed out by N.B., Innodb delays writing the data to disk, and hence it is not visible to second thread.

How should I handle this scenario? Sleep/Usleep is nearly always a suboptimal solution, as under heavy load conditions, the disk i/o time may increase. Is there some way to ‘notify’ the second thread that Innodb has finished its disk i/o?

Answer 1

Pasting from my comment seeing that there’s no need to change anything:

There are no race conditions here. Thread 1 can see its transactions
and data. Thread 2 can’t because they haven’t reached the disk yet (no
fsync call has been made yet by InnoDB). So naturally, you’ll always
see data from thread 1, but if it’s not on the disk at the time of
calling – you won’t see it within thread 2. Threads 1 and 2 don’t
share the SAME mysql connection thread, both use a different one. So
to cut it short – if not on disk, no data available. It’s not on disk
because InnoDB will delay the write until the drive is ready to write.
That’s why you see it after usleep.