Question

I have this big table called StateChanges (1.9 million rows) from a third party application that contains this data:

[ComponentID] : integer : Id of what changed
[NewStatus] : integer : How it changed (0: OK, 1: Error, 2:Warning)
[ConsoleTimeStamp] : timestamp, index : When it changed.

I need to display the various changes in state for a bunch of components between two dates. This is fairly trivial. However, I display my data this way:

[OldStatus] --> [New Status] | [ConsoleTimeStamp]

The problem I have is that, in order to get the ‘initial’ status of any given component, I need to get the rows that immediately before the initial date for all the components. There isn’t any range we can be sure of, there could have been no changes between 2003 and now.

For a single component, i could get this query to work fast enough:

SELECT TOP 1 [NewStatus], [ConsoleTimeStamp] FROM [StateChanges] 
WHERE [ComponentID] = ? AND [ConsoleTimeStamp] < ?
ORDER BY [ConsoleTimeStamp] DESC

Now, is there a way to get all the “previous states” for all of my components (the ids are in an array) in an effective manner? I’ve tried:

SELECT ComponentId, NewStatus, MAX(ConsoleTimeStamp) As LastDate FROM StateChanges
WHERE ComponentId IN ({0}) AND ConsoleTimeStamp <= ?
GROUP BY ComponentId,NewStatus
ORDER BY ComponentId ASC, LastDate ASC

This solutions just happens to be real slow (and give extra results i don’t need) compared to the other query that fetches the entire set of changes between the two dates.

Thank you.

Answer 1

I’m not entirely sure I totally understand your question – but one approach would be to use a CTE (Common Table Expression) if you’re on SQL Server 2005 and newer (you aren’t specific enough in that regard).

With this CTE, you can partition your data by some criteria – i.e. your ComponentId – and have SQL Server number all your rows starting at 1 for each of those partitions, ordered by some other criteria – i.e. probably ConsoleTimeStamp.

So try something like this:

;WITH PartitionedComponents AS
(
   SELECT 
       ComponentId, NewStatus, ConsoleTimeStamp,
       ROW_NUMBER() OVER(PARTITION BY ComponentId ORDER BY ConsoleTimeStamp DESC) AS 'RowNum'
   FROM 
       dbo.StateChanges
   WHERE
       ComponentId IN (.....) 
       AND ConsoleTimeStamp <= (threshold)
)
SELECT 
   ComponentId, NewStatus, ConsoleTimeStamp, RowNum
FROM 
   PartitionedComponents
WHERE
   RowNum <= 2

Here, I am selecting only the last two entries for each “partition” (i.e. for each ComponentId) – ordered in a descending fashion by the ConsoleTimeStamp.

Does that approach what you’re looking for??