I have a table with 17.6m rows
'CREATE TABLE `tmp_hist` (
`ti` int(11) DEFAULT NULL,
`cip6` varchar(15) DEFAULT NULL,
`date` varchar(20) DEFAULT NULL,
`fact` int(11) DEFAULT NULL,
`se` char(1) DEFAULT NULL,
`oper` int(11) DEFAULT NULL,
`qte` int(11) DEFAULT NULL,
`prix` double DEFAULT NULL,
`cip` int(11) DEFAULT NULL,
`fl` int(11) DEFAULT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1'
And it takes around 10 minutes to run a very simple update
update tmp_hist set cip=100
There’re the times:
- 39sec — repair table tmp_hist (this one is particularly interesting because in myisam, repairing a table copies it to a different file and replaces original, this indicates disk speed)
- 531sec — update tmp_hist set cip=100
- 400sec — CREATE TABLE tmp_inno SELECT * FROM tmp_hist (here I try to convert the table to InnoDB)
- 317sec — update tmp_inno set cip=999 (this one in InnoDb)
By all assumptions I expect the update time to be very comparable to repair time, that is, 40 seconds. But it takes 10 minutes! What can be done to speed it up? The code in question converts some data from format A to format B. It is guaranteed to run in a single thread without anyone else ever accessing same data and no recovery will be needed should anything go wrong, it can be just started over.
PS: The UPDATE statement is simplified for testing (and all times are for this simplified update). The real code sets different values for each row, but I found that the execution time for real update is almost the same as for simplified update, so i narrowed the problem to simplified one.
Current progress
Using the knowledge from the answer, I applied ROW_FORMAT=FIXED (which appears to be equivalent to changing all column types to fixed) and it reduced the update time to 145sec, which is almost 4 times faster.
Still, it’s around 2.5x slower then repair time, which I deem to be the time that can be reached.
Since you have
varchars in your table, the update must read the row, look for the correct offset and then update the cip field. Furthermore, since the rows are variable sized, the engine cannot easily determine offsets of a single record. So, you might try changing yourvarcharfields to fixedcharand test, if this makes a difference.An interesting answer, covering this topic too, is over at dba SE https://dba.stackexchange.com/a/2643