Question

I have a requirement to download multiple CSV files from a remote FTP site. I am leveraging SSIS because that is the only tool available on site. I do have FTP script which downloads all files and a for each loop which allows me to merge all files.

I would like to convert files to a TAB delimited format to avoid commas in the data from splitting fields (or if someone has another solution I am willing to listen). I have a VB script which does convert the files but I’d like to leverage a task such as ActiveX Script Task or Script Task to run the script within SSIS. How can I insert/convert the script to use one of these tasks? Below is the code I am using to convert the files.

Dim objFSO, objFile, objFileTSV
Dim strLine, strNewLine, strNewText
Dim FileNameLength, LineLength, NewFileName, Linepos, Quote, QuoteCount, TotalFilesConverted

Set objFSO = CreateObject("scripting.filesystemobject")
strCurPath = objFSO.GetAbsolutePathName(".")
TotalFilesConverted = 0

For Each objFile In objFSO.getfolder(strCurPath).Files
    If UCase(Right(objFile.Name, 4)) = ".CSV" Then
        FileNameLength = Len(objFile.Name)-4
        NewFileName = Left(objFile.Name,FileNameLength) & ".tsv"
        Set objFile = objFSO.OpenTextFile(objFile, 1)

        Do Until objFile.AtEndOfStream
            strLine = objFile.ReadLine
            LineLength = Len(strLine)
            Linepos =1
            strNewLine =""
            Quote = False
            QuoteCount = 0

            Do While Linepos <= LineLength
                If mid(strLine, Linepos, 1) = "," and Not Quote Then 
                    strNewLine = strNewLine + vbTab
                    Quote = False
                Elseif mid(strLine, Linepos, 1) = Chr(34) Then
                    QuoteCount = QuoteCount +1
                    If QuoteCount =2 and Linepos <> LineLength Then
                        If mid(strLine, Linepos, 2) = Chr(34) & Chr(34) Then
                            strNewLine = strNewLine + Chr(34)
                            Linepos = Linepos +1
                            Quote = True
                            QuoteCount = 1
                        Else
                            Quote = False
                            QuoteCount = 0
                        End If
                    Else 
                        Quote = True
                    End If
                Else
                    strNewLine = strNewLine + Mid(strLine, Linepos, 1)
                End If
                Linepos = Linepos +1
            Loop
            strNewText = strNewText & strNewLine & vbCrLF
        Loop
        objFile.Close

        Set objFileTSV = objFSO.CreateTextFile(NewFileName)
        objFileTSV.WriteLine strNewText
        TotalFilesConverted = TotalFilesConverted +1
        strNewText = ""
        objFileTSV.Close

    End If
Next

MsgBox CStr(TotalFilesConverted) + " Files Converted from CSV to TSV."

Answer 1

Since the SSIS Script Task object gives you the choice of either C# or VB.NET, you can track down plenty of code ideas on how to parse a CSV file (see Parse Delimited CSV in .NET for instance).

Also, looping through the file system in .NET is very easily done:

For Each dirItem As String In System.IO.Directory.EnumerateFileSystemEntries(DirPath)
    ' Insert code here ...
Next

Hope that helps!