Question

I’m having problems debugging a failing mysql 5.1 insert under PHP 5.3.4. I can’t seem to see anything in the mysql error log or php error logs.

Based on a Yahoo presentation on efficient pagination, I was adding order numbers to posters on my site (order rank, not order sales).

I wrote a quick test app and asked it to create the order numbers on one category. There are 32,233 rows in that category and each and very time I run it I get 23,304 rows updated. Each and every time. I’ve increased memory usage, I’ve put ini setting in the script, I’ve run it from the PHP CLI and PHP-FPM. Each time it doesn’t get past 23,304 rows updated.

Here’s my script, which I’ve added massive timeouts to.

include 'common.inc'; //database connection stuff
ini_set("memory_limit","300M");
ini_set("max_execution_time","3600");
ini_set('mysql.connect_timeout','3600');
ini_set('mysql.trace_mode','On');
ini_set('max_input_time','3600');
$sql1="SELECT apcatnum FROM poster_categories_inno LIMIT 1";
$result1 = mysql_query($sql1);
while ($cats = mysql_fetch_array ($result1)) {
$sql2="SELECT poster_data_inno.apnumber,poster_data_inno.aptitle FROM poster_prodcat_inno, poster_data_inno WHERE poster_prodcat_inno.apcatnum ='$cats[apcatnum]' AND poster_data_inno.apnumber = poster_prodcat_inno.apnumber ORDER BY aptitle ASC";
$result2 = mysql_query($sql2);
$ordernum=1;
while ($order = mysql_fetch_array ($result2)) {
$sql3="UPDATE poster_prodcat_inno SET catorder='$ordernum' WHERE apnumber='$order[apnumber]' AND apcatnum='$cats[apcatnum]'";
$result3 = mysql_query($sql3);
$ordernum++;
} // end of 2nd while
}

I’m at a head-scratching loss. Just did a test on a smaller category and only 13,199 out of 17,662 rows were updated. For the two experiments only 72-74% of the rows are getting updated.

Answer 1

I’d say your problem lies with your 2nd query. Have you done an EXPLAIN on it? Because of the ORDER BY clause a filesort will be required. If you don’t have appropriate indices that can slow things down further. Try this syntax and sub in a valid integer for your apcatnum variable during testing.

SELECT d.apnumber, d.aptitle 
FROM poster_prodcat_inno p JOIN poster_data_inno d
  ON poster_data_inno.apnumber = poster_prodcat_inno.apnumber 
WHERE p.apcatnum ='{$cats['apcatnum']}' 
ORDER BY aptitle ASC; 

Secondly, since catorder is just an integer version of the combination of apcatnum and aptitle, it’s a denormalization for convenience sake. This isn’t necessarily bad, but it does mean that you have to update it every time you add a new title or category. Perhaps it might be better to partition your poster_prodcat_inno table by apcatnum and just do the JOIN with poster_data_inno when you need the actually need the catorder.