Question

I am using conditional splits to validate data and send bad data down an alternate path in my package. My conditional split code is similar to this:

(Gender != “M” ||Gender != “F” ||Gender != “U”) == FALSE
(Gender == “M” ||Gender == “F” ||Gender == “U”) == TRUE

If the value is true I pass control to another conditional split or if false I will redirect that row to an error table.

I am using 25 conditional splits in a single package and I want to know if this is a good thing to do from a performance perspective. How is the performance of using conditional splits as opposed to using another method to validate data?

I have added another scenario but its not working in conditional split. Please find the below information

I have allowed date values only this format and this period ’01/01/1753′ to ’12/31/9000′

My Conditions

(Dob Varchar(10))

SUBSTRING(Dob,1,2) <= “12” && SUBSTRING(Dob,4,2) <= “31” && (SUBSTRING(Dob,7,4) >= “1753” || SUBSTRING(Dob,7,4) <= “9000”) == TRUE

SUBSTRING(Dob,1,2) > “12” || SUBSTRING(Dob,4,2) > “31” || SUBSTRING(Dob,7,4) < “1753” || SUBSTRING(Dob,7,4) > “9000” == FALSE

My Input – 12/32/1990, 13/15/2000, 12/31/2010,01/01/1753,12/31/9000,12/31/9001,01/01/9001
Expected Output – 12/31/2010,01/01/1753,12/31/9000

But all the records considered as false in my conditions.
Kindly give the solution for this scenario.

Answer 1

As William Todd Salzman has indicated, testing is the only way you will determine what is the optimal setup for your package. Generally speaking however, as counter-intuitive as it may seem, the less you do in a particular component, the faster SSIS can make it. The reason for this is the way the SSIS engine can determine parallelism – if it can determine column B is not manipulated in Derived column 1, then it can safely start making changes to that value in Derived Column 2.

• Can different combinations of components affect Dataflow performance?
• Increasing Throughput of Pipelines by Splitting Synchronous Transformations into Multiple Tasks

Given your scenario, I would look to create multiple Derived Column Transformations that define your various boolean checks. For example, I’d create one field that indicates whether we have a valid gender like this.

Add a Derived Column Transformation named DFT IsGenderValid and I’d configure it thus

• IsGenderValid
• “add as new column”
• Gender != “M” || Gender != “F” || Gender != “U”
• DT_BOOL

I would then have a Conditional Split transformation operating on boolean value if for no other reason than I have one place to test, correct and maintain logic.

Repeat this pattern for as many validations as your business logic requires.