Question

I have data in the following format:

DATE                 DATA1                 DATA2
-------------------------------------------------
20121010             ABC                   DEF
20121010             DEF                   ABC
20121010             HIJ                   KLM
20121010             KLM                   HIJ
20121212             ABC                   DEF
20121212             DEF                   ABC
20121212             HIJ                   KLM
20121212             KLM                   HIJ

What I want to do is select rows 1 and 3. I don’t care about rows 2 and 4 because they are essentially “duplicates” in my eyes.

Seems simple but I’m just trying to put the query together to accomplish this.

Answer 1

You can use the row_number() function for this, assuming you are using version 2005 or higher:

select date, data1, data2
from (select t.*,
             row_number() over (partition by date order by date) as seqnum
      from t
     ) t
where seqnum = 1

The expression order by date should produce an arbitrary ordering in any database that supports row_number. In SQL Server, you can also use order by (select NULL).

Or, I realize that your question may be about eliminate duplicates, regardless of order. For that, you can do:

select distinct date, minData, maxData
from (select t.date,
             (case when data1 > data2 then data1 else data2 end) as minData,
             (case when data1 > data2 then data2 else data1 end) as maxData
      from t
     ) t

This might, however, rearrange the two values, when only one row appears.

The more complicated solution to maintain the original ordering of the columns and eliminate the additional rows combines the two approaches:

select date, data1, data2
from (select t.*,
             row_number() over (partition by date order by minData, maxData) as seqnum
      from (select t.*
                   (case when data1 > data2 then data1 else data2 end) as minData,
                   (case when data1 > data2 then data2 else data1 end) as maxData
            from t
           ) t
     ) t
where seqnum = 1