Question

I’m trying to come up with a way to use a relative stdev range function in SQL Server 2005/8 without any looping. Assuming a test dataset (test_data) with 10 records and columns:

column_id identity(1,1), column_data, column_stdev

I would need my stdev() to work as follows:

stdev(column_data where id between i and i + 3) with i = column_id, autoincrementing.

So far I’m only able to acoomplish it via the following query, which is extra slow:

declare @i int
set @i=1
while @i < 11
 begin
 update test_data
 set column_stdev = (select stdev(column_data) from test_data 
                     where column_id between @i and @i+2)
 set @i=@i+1
end

Could you, please, let me know if there is a way to avoid looping in order to set column_stdev to get stdev of the past 3 records in column_data.

Answer 1

One obvious improvement would be to get rid of the WHILE loop and the row by row processing. If you are not doing it in an explicit transaction you will need to wait for 10 individual commits to complete and the total amount of work required is greater anyway as shown by the stats below.

NB: I’m not claiming that my query can’t be improved upon however. You may be able to adjust the “quirky update” approach for your needs and do it with a single scan through the data for example.

create table t (
column_id int identity(1,1) primary key, 
column_data float, 
column_stdev float)

insert into t (column_data)
select top 10 CHECKSUM(newid()) from sys.objects

SET STATISTICS IO ON

UPDATE t 
SET  column_stdev = (SELECT stdev(t2.column_data)
                     FROM t t2 
                     WHERE t2.column_id BETWEEN t.column_id AND t.column_id + 2)

/*Table 't'. Scan count 11, logical reads 42, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
Table 'Worktable'. Scan count 1, logical reads 23, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.*/

declare @i int
set @i=1
while @i < 11
 begin
 update t
 set column_stdev = (select stdev(column_data) from t 
                     where column_id between @i and @i+2)
 set @i=@i+1
end


/* (Aggregated the 10 results)
Table 't'. Scan count 20, logical reads 240, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
Table 'Worktable'. Scan count 10, logical reads 230, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.*/