Question

I know how to accomplish my task with .NET, but I wanted to do this just in SQL.

I need to loop through all of the rows where the primary key is somewhat arbitrary. It can be a number or a series of letters, and probably any number of unusual things.

I know I could do something like this…

DECLARE @numRows INT
SET @numRows = (SELECT COUNT(pkField) FROM myTable)


DECLARE @I INT
SET @I = 1
WHILE (@I <= @numRows)
BEGIN
        --Do what I need to here
        SET @I = @I  + 1
END

...if my rows were indexed in a contiguous fashion, but I don't know enough about SQL to do that if they're not. I keep coming across the use of "cursors," but I come across just as much reading about avoiding cursors.

I found this SO solution but I'm not sure if that's what I'm needing?

I appreciate any ideas.

Answer 1

If you are going to loop through rows, use a cursor. it is much less efficient to do it the way you have shown. Basically what you are doing is a cursor without using SQL Server’s cursor facilities. Just because you don’t use the keyword CURSOR doesn’t mean it isn’t a cursor. Logically that’s what you have. Each of your selects is basically going to be a query on the underlying table. Especially if the table is not indexed, it may be much less efficient than a cursor. There are things you can do to make a cursor less bad…FORWARD_ONLY, etc. Also sometimes making a static cursor will make things faster, other times it will make things slower. Read up on the CURSOR statement.

The reality is to avoid cursors you don’t just invent a cursor using select statements, that is stupidity and the people who do that should be fired. People who claim that this is not a cursor are deluding themselves… You need to re-think your entire problem. There are some things that cannot be done without cursors and for those, there are cursors. Basically I would define a cursor as any mechanism that performs an operation row by row on a table.

Anyway the missing piece for what you are trying to do is that you can turn a non numeric key into a numeric key by creating a mapping table:

SELECT IDENTITY(BIGINT, 1, 1) AS numKey, pkField INTO #keymap
FROM MyTable

DECLARE @I INT
SET @I = 1
WHILE (@I <= @numRows)
BEGIN
    SELECT mt.*
    FROM  myTable mt inner join #keymap km on km.pkfield = mt.pkfield
    WHERE numkey = @I
    --Do what I need to here
    SET @I = @I  + 1
END

This is still very bad because now you are doing a join on every record…it may be slightly better to:

A) SELECT IDENTITY(BIGINT, 1, 1) AS numKey, * INTO #keymap
FROM MyTable
Now #keymap is the same table as your original table but with a numeric key, so the solution you have will work without a join…however if the table is big or you want to update it, then this is not good

B) instead of doing a join

SELECT IDENTITY(BIGINT, 1, 1) AS numKey, pkField INTO #keymap
FROM MyTable

DECLARE @I INT, @myKey VARCHAR(255) -- assume varchar
SET @I = 1
WHILE (@I <= @numRows)
BEGIN
    SELECT @myKey = pkfield FROM #keymap WHERE numKey = @i
    SELECT mt.*
    FROM  myTable
    WHERE pkfield = @myKey
    --Do what I need to here
    SET @I = @I  + 1
END

This way there will be no join, but you are now doing two queries…you may even need to index #keymap….

The best thing is to re-evaluate your problem. Sometimes you can use CASE WHEN or an identity column to enable you to avoid the cursor. It really depends on the problem. But for those cases when you need to use a cursor, using an actual CURSOR is usually much faster than the other method. Even faster is to write a program in .NET, Java, even Perl/Python. I had a function with SQL Server 2000 that used two nested cursors and took over an hour before I gave up. When I wrote a Perl program to do the same thing it only took 10 minutes. And Perl, like any interpreted language, is much slower than a compiled language. Java/C/C++/C# would probably be 10/20/30 times quicker…. So if you really need a cursor, it is probably better to do it in another programming language. Especially if you are using SQL Server 2005 which has the CLR (Common Language Runtime) embedded in it…