Question

I am working with a big table (~100.000.000 rows) in SQL Server 2008. Frequently, I need to add and remove batches of ~30.000.000 rows to and from this table. Currently, before loading a large batch into the table, I disable indexes, I insert the data, then I rebuild the index. I have measured this to be the fastest approach.

Since recently, I am considering implementing table partitioning on this table to increase speed. I will partition the table according to my batches.

My question, will it be possible to disable the index of one particular partition, and load the data into that one before enabling it again? In that case, the rest of my table will not have to suffer a complete index rebuild, and my loading can be even faster?

Answer 1

Indexes are typically on the Partition Scheme. For the scenario you are talking about you can actually load up a new table with the batch (identical structure, different name) and then use the SWITCH command to add this table as a new partition into your existing table.

I have included code that I use to perform this, you will need to modify it based on your table names:

DECLARE @importPart int
DECLARE @hourlyPart int

SET @importPart = 2 -- always, so long as the Import table is only made up of 1 partition

-- get the Hourly partition
SELECT 
    @hourlyPart = MAX(V.boundary_id) + 1
FROM 
    sys.partition_range_values V
JOIN    sys.partition_functions F
    ON  V.function_id = F.function_id
    AND F.name = 'pfHourly'

ALTER TABLE Import
SWITCH PARTITION @importPart
TO Hourly PARTITION @hourlyPart;