Question

I have a very large table (800GB) which has a DATETIME field which is part of a partition schema. This field is named tran_date. The problem I’m having is that the indexes are not properly aligned with the partition and I can’t include the tran_date field in the PRIMARY KEY because it’s set to nullable.

I can drop all foreign key relationships, statistics, and indexes, but I can’t modify the column because the partition schema is still dependent on the tran_date column.

In my research I’ve located one way to move the table off of the partition which is to drop the clustered index and then re-write the clustered index onto the PRIMARY filegroup which will then allow me to modify the column, but this operation takes several hours to drop, 13 hours to write the temporary CLUSTERED INDEX on PRIMARY and then I have to drop that, alter the table, and re-write the CLUSTERED INDEX properly which takes another 13 hours. Additionally I have more than one table.

When I tested this deployment in my development environment with a similarly sized data set it took several days to complete, so I’m trying to look for ways to chop down this time.

If I can move the table off the partition without having to write a CLUSTERED INDEX on PRIMARY it would significantly reduce the time required to alter the column.

Answer 1

No matter what, you are going to end up moving data from “point A” (stored in table partitions within the database) to “point B” (not stored within table partitions within the database. The goal is to minimize the number of times you have to work through all that data. Simplest way to do this might be:

• Create a new non-partitioned table
• Copy the data over to that table
• Drop the original table
• Rename the new table to the proper name

One problem to deal with is the clustered index. You could either create the new table without the clustered index, copy the data over, and then reindex (extra time and pain), or you could create the table with the clustered index, and copy the data over “in order” (say, low Ids to high). This would be slower than copying it over to a non-clustered table, but it might be faster overall since you wouldn’t then have to build the clustered index.

Of course there’s the problem of “what if users change the data while you’re copying it”… but table partitioning implies warehousing, so I’m guessing you don’t have to worry about that.

A last point, when copying gobs of data, it is best to break the insert into several inserts, so as to not bloat the transaction log.