Question

My SSIS program reads as input from a .csv file.

The file has about 60,000 rows. And my SSIS package fails during the read, saying cannot convert a certain column due to potential loss of data.

Now, I am certain that the majority of the rows are correct. As I have tried pasting random subset of the file and the SSIS reads fine.

But I can’t figure out a way to determine exactly on what line did my package fail.

I have spent 2 months on this problem, any advice?

Answer 1

You could find out the first culprit with 16 iterations. Here is a brain + brawn method:

First: Back everything up. Make copies of backups in safe places. Sorry to state the obvious, but I’ve recently been bitten, and I know better.

The file with 60K records – let’s call this your base file.

1. Split base file into two files (FileA, FileB)
2. Use one of them as input.
3. Run SSIS – if SSIS fails, use FileA as your base file – else use FileB as base file
4. Go to step 1

You will have the offending record at the 16 iteration. (60k, 30k, 15k, 7500, 3750, 1875, 937, 468, 234, 117, 58, 29, 14, 7, 3, 1)

Turn logging on for everything and rerun the SSIS package. You should have the offending record in the base file and the exact data point in the log.