Question

I have two concurrent processes:

1.) Writer – inserts new rows into a MySQL database on a regular basis (10-20 rows/sec)

2.) Reader – reads from the same table being inserted into

I notice that the Reader process only seems to see a snapshot of the database at about the time of its startup. Inserts occuring before this startup are found, but inserts occuring after are not. If I shut the Reader process down and restart it (but leave the Writer running), it will sometimes (but not always) see more data, but again seems to get a point-in-time view of the database.

I’m running a commit after each insert (code snippet below). I investigated whether this was a function of change buffering/pooling, but doing a “set @@global.innodb_change_buffering=none;” had no effect. Also, if I go in through MySQL workbench, I can query the most current data being inserted by the Writer. So this seems to be a function of how the Python/MySQL connection is getting set up.

My environment is:

Windows 7

MySQL 5.5.9

Python 2.6.6 — EPD 6.3-1 (32-bit)

MySQL python connector

The insert code is:

    def insert(dbConnection, statement):
    cursor = dbConnection.cursor()
    cursor.execute(statement)   
    warnings = cursor.fetchwarnings()
    if warnings:
        print warnings
        rowid = []
    else:
        rowid = cursor.lastrowid
    cursor.close()
    dbConnection.commit()
    return rowid    

The reader code is:

def select(dbConnection, statement):
cursor = dbConnection.cursor()
cursor.execute(statement)   
warnings = cursor.fetchwarnings()
if warnings:
    print warnings
    values = []
else:
    values = np.asarray(cursor.fetchall())
cursor.close()
return values   

Answer 1

What’s the read side look like?

I bet this is a problem with the isolation level on the read side. Most likely your read connection is getting an implicit transaction and the default InnoDB isolation level is:

Repeatable Read

Try issuing:

cursor.execute("SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED")

on the read side.