Question

I’ve developed a couple of T-SQL stored procedures that iterate over a fair bit of data. The first one takes a couple of minutes to run over a year’s worth of data which is fine for my purposes. The second one, which uses the same structure/algorithm, albeit over more data, takes two hours, which is unbearable.

I’m using SQL-Server and Query-Analyzer. Are there any profiling tools, and, if so, how do they work?

Alternatively, any thoughts on how improve the speed, based on the pseudo-code below? In short, I use a cursor to iterate over the data from a straight-forward SELECT (from a few joined tables). Then I build an INSERT statement based on the values and INSERT the result into another table. Some of the SELECTed variables require a bit of manipulation before INSERTion. The includes extracting some date parts from a date value, some basic float operations and some string concatenation.

— Rough algorithm / pseudo-code

DECLARE <necessary variables>
DECLARE @cmd varchar(1000)
DECLARE @insert varchar(100) = 'INSERT INTO MyTable COL1, COL2, ... COLN, VALUES('

DECLARE MyCursor Cursor FOR
    SELECT <columns> FROM TABLE_1 t1
    INNER JOIN TABLE_2 t2 on t1.key = t2.foreignKey
    INNER JOIN TABLE_3 t3 on t2.key = t3.foreignKey

OPEN MyCursor
FETCH NEXT FROM MyCursor INTO @VAL1, @VAL2, ..., @VALn
WHILE @@FETCH_STATUS = 0
BEGIN
   @F = @VAL2 / 1.1  --- float op
   @S = @VAL3 + ' ' + @VAL1
   SET @cmd = @insert
   SET @cmd = @cmd + DATEPART(@VAL1) + ', '
   SET @cmd = @cmd + STR(@F) + ', '
   SET @cmd = @cmd + @S + ', '
   SET @cmd = @cmd + ')'
   EXEC (@cmd)
   FETCH NEXT FROM MyCursor @VAL1, @VAL2, ..., @VALn
END
CLOSE MyCursor
DEALLOCATE MyCursor

Answer 1

The first thing to do – get rid of the cursor…

INSERT INTO MyTable COL1, COL2, ... , COLN
SELECT ...cols and manipulations...
FROM TABLE_1 t1
INNER JOIN TABLE_2 t2 on t1.key = t2.foreignKey
INNER JOIN TABLE_3 t3 on t2.key = t3.foreignKey

Most things should be possible direct in TSQL (it is hard to be definite without an example) – and you could consider a UDF for more complex operations.