Question

I am trying to do a simple fetch using MySQLDB in Python.

I have 2 tables(Accounts & Products). I have to look up Accounts table, get acc_id from it & query the Products table using it.

The Products tables has more than 10 rows. But when I run this code it randomly returns between 0 & 6 rows each time I run it.

Here’s the code snippet:

# Set up connection
con = mdb.connect('db.xxxxx.com', 'user', 'password', 'mydb')

# Create cursor
cur = con.cursor()

# Execute query 
cur.execute("SELECT acc_id FROM Accounts WHERE ext_acc = '%s'" % account_num ) # account_num is alpha-numberic and is got from preceding part of the program

# A tuple is returned, so get the 0th item from it
acc_id = cur.fetchone()[0] 
print "account_id = ", acc_id

# Close the cursor - I was not sure if I can reuse it
cur.close() 

# Reopen the cursor
cur = con.cursor() 

# Second query
cur.execute("SELECT * FROM Products WHERE account_id = %d" % acc_id)

keys = cur.fetchall()
print cur.rowcount # This prints incorrect row count

for key in keys: # Does not print all rows. Tried to directly print keys instead of iterating - same result :(
    print key

# Closing the cursor & connection
cur.close()
con.close()

The weird part is, I tried to step through the code using a debugger(PyDev on Eclipse) and it correctly gets all rows(both the value stored in the variable ‘keys’ as well as console output are correct).

I am sure my DB has correct data since I ran the same SQL on MySQL console & got the correct result.

Just to be sure I was not improperly closing the connection, I tried using with con instead of manually closing the connection and it’s the same result.

I did RTM but I couldn’t find much in it to help me with this issue.

Where am I going wrong?

Thank you.

EDIT: I noticed another weird thing now. In the line
cur.execute("SELECT * FROM Products WHERE account_id = %d" % acc_id), I hard-coded the acc_id value, i.e made it
cur.execute("SELECT * FROM Products WHERE account_id = %d" % 322) and it returns all rows

Answer 1

I sort of figured out the problem. It was silly at the end. It was a race condition!

This is how my actual code was organized :

---

 Code Block 1
 {code which calls an API which creates an entry in Accounts table &
 Creates corresponding entries in Product table(10 entries)}

……

Code Block2
{The code I had posted in my question}

---

The problem was that the API(called in Code Block 1) took a few seconds to add 10 entries into the Product table.

When my code(Code Block 2) ran a fetch query, all the 10 rows were not added and hence fetched somewhere between 0 to 6 rows(how much ever was added at that time).

What I did to solve this was made the code sleep for 5 seconds before I did the SQL queries:

Code Block 1
time.sleep(5)
Code Block 2

The reason why it worked when I hard coded the acc_id was that, the acc_id which I hard-coded was from a precious execution(each run returns a new acc_id).
And the reason why it worked while stepping through a debugger was that manually stepping acted like giving it a sleep time.

It is a lesson for me to know a little about the inside working of APIs(even though they are supposed to be like a black box) and think about race conditions like this, the next time I come across similar issues.