Question

I want to select one record at a time from a MySQL table. I found a similar post on SO here –

How to select records one by one without repeating

However, in my case, the table size is not fixed, The data is continuously been added to the table and I want to select one record at a time from this table. Also, I’m using python to connect to the mysql database and do processing over each record. Any pointers?

P.S. : The size of the table is very large, hence everytime, I can not compute the number of records in the table

Answer 1

This functionality isn’t built into SQL.

If you have a dense, incrementing, indexed column on the table, then you just do this:

i = 0
while True:
    # your usual select here, but with "and MyIndexColumn = %d' % (i,)

With some databases, there’s such a column built in, whether you want it or not, usually called either “RowID” or “Row ID”; if not, you can always add such a column explicitly.

If the values can be skipped (usually they can, e.g., if for no other reason than because someone can delete a row from the middle), you have to be able to handle a select that returns 0 rows, of course.

The only issue is that there’s no way to tell when you’re done. But that’s inherent in your initial problem statement: data is being continuously added, and you don’t know the size of it in advance, so how could you? If you need to know when the producer is done, it has to somehow (possibly through another table in the database) give you the highest rowid it created.