Question

I’m trying to alter 2000 tables by adding a column to each. I achieve this by using a for loop in C# and executing a query on each table name for a list of strings I have. It starts out fast but then gets slower and slower. The memory for mysql.exe increases from 50k to 375k in about 30 seconds. Why is it doing this? Once I do one alter table the memory should drop down again. Is some resource not being disposed of after each alter? A cache, or buffer? Garbage collection? I can’t finish this for loop because it gets incredible slow after the 20th-25th alter table.

Answer 1

When you issue a alter table statement to MySQL, it starts rebuilding the table.

It does the following steps:

1. Copy the old table to a new file.
2. Add a column to the new file definition .frm-file
3. Add the column in the datafile in the new .myd-datafile, this requires rewriting the entire datafile.
4. Go through all the rows in the new datafile to fill the new column with the default value
5. If the new column has an index MySQL has to do steps 2,3,4 for the .MYI-indexfile as well.
6. Rename the old datafile
7. Rename the new datafile so it has the same name the old file had.
8. Delete the old datafile

For performance reasons it will try to keep the new data and index file in memory.
This is the memory increase you are seeing.
MySQL will keep the table data in memory for some time and will only clear out the cache if the memory assigned to cache space in My.ini has been exhausted.

You can clear the cache by issuing:

RESET QUERY CACHE;

After every alter table statement.

See: http://dev.mysql.com/doc/refman/5.0/en/query-cache-status-and-maintenance.html
Read up on MySQL cache issues here: http://www.docplanet.org/mysql/mysql-query-cache-in-depth/