I have an excel file with duplicate values..In SQL table the fields are Primary key..I created a ssis package for import the data from excel. If there have any duplicate values or any corrupted data ,i put an error table to store the corrupted or duplicate data.If there have any error data its redirects to the error table…I tried with an excel without any duplicate data it will insert whole data to the Target table and the corrupted data to the error table..If there have any duplicate data,it will insert some data to the target table and all other data to the error table,not only the duplicate data.And the number of data moving to error table are unpredictable..same happening with a .csv file…What will be the reason for this?
If anyone knows please help me to rectify…
Thanks.
SSIS batches records for the insert, so if one of the records in the batch gets a SQL error because of the primary key constraint the whole batch would be moved to the error table. Since the records are not inserted one at a time it cannot move only the record that fails the constraint, because it does not know which record failed.
You can set the batch size to 1 so that individual records are moved, but I would definitely affect performance.
http://www.sqlservercentral.com/Forums/Topic981919-148-1.aspx#bm981944