Question

I am attempting to normalize data using SSIS in the following format:

SerialNumber  Date       R01  R02  R03  R04 
-------------------------------------------
1             9/25/2011  9    6    1    2
1             9/26/2011  4    1    3    5
2             9/25/2011  7    3    2    1
2             9/26/2011  2    4    10   6

Each “R” column represents a reading for an hour. R01 is 12:00 AM, R02 is 1:00 AM, R03 is 2:00 AM and R04 is 3:00 AM. I would like to transform the data and store it in another table in this format (line breaks for readability):

SerialNumber  Date                Reading
-----------------------------------------
1             9/25/2011 12:00 AM  9    
1             9/25/2011 1:00 AM   6    
1             9/25/2011 2:00 AM   1   
1             9/25/2011 3:00 AM   2   

1             9/26/2011 12:00 AM  4    
1             9/26/2011 1:00 AM   1    
1             9/26/2011 2:00 AM   3   
1             9/26/2011 3:00 AM   5   

2             9/25/2011 12:00 AM  7    
2             9/25/2011 1:00 AM   3    
2             9/25/2011 2:00 AM   2   
2             9/25/2011 3:00 AM   1

2             9/26/2011 12:00 AM  2    
2             9/26/2011 1:00 AM   4    
2             9/26/2011 2:00 AM   10   
2             9/26/2011 3:00 AM   6

I am using the unpivot transformation in an SSIS 2008 package to accomplish most of this but the issue I am having is adding the hour to the date based on the column of the value I am working with. Is there a way to accomplish this in SSIS? Keep in mind that this is a small subset of data of around 30 million records so performance is an issue.

Thanks for the help.

Answer 1

1. Create a SSIS package and add a new Data Flow Task and configure this DFT (Edit...)
2. Add a new data source
3. Add UNPIVOT component and configure it thus:
   
4. Add DATA CONVERSION component:
   
   Temporary results:
   

5. Add DERIVED COLUMN component:
   
   For NewData derived column you can use this expression: DATEADD("HOUR",(Type == "R01" ? 0 : (Type == "R02" ? 1 : (Type == "R03" ? 2 : 3))),Date). «boolean_expression» ? «when_true» : «when_false» operator is like IIF() function (from VBA/VB) and is used to calculate number of hours to add: for “R01” -> 0 hours, for “R02” -> 1 hour, for “R03” -> 2 hours or else 3 hours (for “R04”).

   Results: