Question

I have a table with roughly 1 million rows. I’m doing a simple program that prints out one field from each row. However, when I started using mysql_pconnect and mysql_query the query would take a long time, I am assuming the query needs to finish before I can print out even the first row. Is there a way to process the data a bit at a time?

–Edited–
I am not looking to retrieve a small set of the data, I’m looking for a way to process the data a chunk at a time (say fetch 10 rows, print 10 rows, fetch 10 rows, print 10 rows etc etc) rather than wait for the query to retrieve 1 million rows (who knows how long) and then start the printing.

Answer 1

Printing one million fields will take some time. Retrieving one million records will take some time. Time adds up.

Have you profiled your code? I’m not sure using limit would make such a drastic difference in this case.

Doing something like this

while ($row = mysql_fetch_object($res)) {
   echo $row->field."\n";
}

outputs one record at a time. It does not wait for the whole resultset to be returned.

If you are dealing with a browser you will need something more.

Such as this

ob_start();
$i = 0;
while ($row = mysql_fetch_object($res)) {
   echo $row->field."\n";
   if (($i++ % 1000) == 0) {
       ob_flush();
   }
}
ob_end_flush();