Question

I have one folder called ‘Data’ that contains all files with current date in filename like for example ‘myfile 20-08-2011’. Now I want to create an SSIS package which collects all files which are from the month 08, that is, I want to sort out files by month wise and copy those files into a new folder called “august”. How can I do that?

Answer 1

Here is one possible solution to achieve this with the help of Foreach loop container, Script Task and File System Task. You can do this without File System Task. However, I have used it to make use of the in-built control flow task to move the files. The example was created using SSIS 2005.

The example assumes that the files will be named uniformly. So, the example uses the format File DD-MM-YYYY. For example, the files will be named File 29-07-2011, File 15-08-2011 etc.

On the SSIS package, create the following variables. In this example, the source files are stored in the folder location F:\Temp\ and the files should be moved to the location *F:\Temp\Monthwise*. Within the destination folder, there will be folders for each month like July, August etc.

• DestinationFolder variable will hold the final destination folder value like F:\Temp\Monthwise\August but this variable will be assigned with the actual value inside the Script task. For now, let’s assign the value F:\Temp\Monthwise\. This temporary value is to avoid File System Task from throwing error messages at design time.

• DestinationRoot will contain the actual root folder under which the folders like July, August should be created based on the month names.

• SourceFolder denotes the folder in which all the files are initially stored. Here in this example, the source folder will be F:\Temp\

• SourceFilePath denotes the actual file path. This variable will be assigned with the individual file values when the Foreach loop container loops through each variable. To avoid the File System Task from throwing error messages at design time, let’s assign it with some dummy value F:\Temp\1.txt.

• FilePattern defines the file pattern that should be looped through in the given source folder path. Let’s assign *.*, which means all the files will be looped through. You can also specify *.txt or File*.txt or My*.xls etc. It is upto your requirements.

• MonthStartPosition denotes the position where the month value starts in the file name. So, in the file name format File 29-07-2011, the month 07 starts at 9th character. Hence the value 9.

• MonthLength specifies the number of character to extract. This will anyways be 2 characters but I didn’t want to hard code. So, I created a variable.

• MonthNameFormat specifies how the folders should be created. Value MMMM denotes that it will create the folders with full month names like January, February etc. If we use the value MMM, the folders will be created as Jan, Feb etc. The folders will be created only if they didn’t exist.

On the SSIS package’s Control Flow tab, place a Foreach loop container and configure it to loop through the folder specified in the variable SourceFolder using the file pattern variable FilePattern. As the Foreach loop container loops through the files, the file names will be assigned to the variable SourceFilePath. We will use this variable to fetch the month value in Script Task.

Within the Foreach loop container, place a Script Task and on the script task’s Script section click the Design script… button to open the VSTA editor and paste the code provided after these screenshots. Since the example was created in VS 2005, the code is written in VB.NET because that is the only supported language in SSIS 2005.

Script Task Code: The code gets the full file path value from the variable SourceFilePath and extracts only the file name to store it in the local variable FileName.

Then checks to see if the MonthStartPosition and MonthLength variables are assigned with proper non-zero values. It then extracts the month value to store it in the local variable MonthValue.

Uisng the MonthValue, it fetches the full month name value using the DateTime function. The values 1 are assigned to day and year because we only want the Month name.

The month name in the local variable FolderName is combined with the DestinationRoot value to check if the folder exists or not. If the folder doesn’t exist, the folder will be created so that the File System Task doesn’t fail.

Finally the full destination folder value is assigned to the package variable DestinationFolder. This variable will be used in the File System Task.

VB.NET code for SSIS 2005

Imports System
Imports System.Data
Imports System.Math
Imports Microsoft.SqlServer.Dts.Runtime

Public Class ScriptMain

    Public Sub Main()

        Dim varCollection As Variables = Nothing
        Dts.VariableDispenser.LockForRead("User::SourceFilePath")
        Dts.VariableDispenser.LockForRead("User::DestinationRoot")
        Dts.VariableDispenser.LockForRead("User::MonthStartPosition")
        Dts.VariableDispenser.LockForRead("User::MonthLength")
        Dts.VariableDispenser.LockForRead("User::MonthNameFormat")
        Dts.VariableDispenser.LockForWrite("User::DestinationFolder")
        Dts.VariableDispenser.GetVariables(varCollection)

        Dim SourceFilePath As String = varCollection("User::SourceFilePath").Value.ToString()
        Dim FileName As String = SourceFilePath.Substring(SourceFilePath.LastIndexOf("\") + 1)
        Dim DestinationRoot As String = varCollection("User::DestinationRoot").Value.ToString()
        Dim MonthStartPosition As Integer = Convert.ToInt32(varCollection("User::MonthStartPosition").Value)
        Dim MonthLength As Integer = Convert.ToInt32(varCollection("User::MonthLength").Value)
        Dim MonthValue As Integer = 0
        Dim MonthNameFormat As String = varCollection("User::MonthNameFormat").Value.ToString()
        Dim FolderName As String = String.Empty
        Dim MonthwiseDirectory As String = String.Empty

        If MonthStartPosition > 0 AndAlso MonthLength > 0 Then
            MonthValue = Convert.ToInt32(FileName.Substring(MonthStartPosition - 1, MonthLength))
        End If

        If FileName.Length > 0 AndAlso MonthValue > 0 Then
            FolderName = New DateTime(1, MonthValue, 1).ToString(MonthNameFormat)
        End If

        MonthwiseDirectory = System.IO.Path.Combine(DestinationRoot, FolderName)

        If Not System.IO.Directory.Exists(MonthwiseDirectory) Then
            System.IO.Directory.CreateDirectory(MonthwiseDirectory)
        End If

        varCollection("User::DestinationFolder").Value = MonthwiseDirectory

        Dts.TaskResult = Dts.Results.Success
    End Sub

End Class

C# code for SSIS 2008 and above

public void Main()
{
    Variables varCollection = null;
    Dts.VariableDispenser.LockForRead("User::SourceFilePath");
    Dts.VariableDispenser.LockForRead("User::DestinationRoot");
    Dts.VariableDispenser.LockForRead("User::MonthStartPosition");
    Dts.VariableDispenser.LockForRead("User::MonthLength");
    Dts.VariableDispenser.LockForRead("User::MonthNameFormat");
    Dts.VariableDispenser.LockForWrite("User::DestinationFolder");
    Dts.VariableDispenser.GetVariables(ref varCollection);

    string SourceFilePath = varCollection["User::SourceFilePath"].Value.ToString();
    string FileName = SourceFilePath.Substring(SourceFilePath.LastIndexOf('\\') + 1);
    string DestinationRoot = varCollection["User::DestinationRoot"].Value.ToString();
    int MonthStartPosition = Convert.ToInt32(varCollection["User::MonthStartPosition"].Value);
    int MonthLength = Convert.ToInt32(varCollection["User::MonthLength"].Value);
    int MonthValue = 0;
    string MonthNameFormat = varCollection["User::MonthNameFormat"].Value.ToString();
    string FolderName = string.Empty;
    string MonthwiseDirectory = string.Empty;

    if (MonthStartPosition > 0 && MonthLength > 0)
    {
        MonthValue = Convert.ToInt32(FileName.Substring(MonthStartPosition - 1, MonthLength));
    }

    if (FileName.Length > 0 && MonthValue > 0)
    {
        FolderName = new DateTime(1, MonthValue, 1).ToString(MonthNameFormat);
    }

    MonthwiseDirectory = System.IO.Path.Combine(DestinationRoot, FolderName);

    if (!System.IO.Directory.Exists(MonthwiseDirectory))
    {
        System.IO.Directory.CreateDirectory(MonthwiseDirectory);
    }

    varCollection["User::DestinationFolder"].Value = MonthwiseDirectory;

    Dts.TaskResult = (int)ScriptResults.Success;
}

Within the Foreach loop container, place File System Task after the Script task. Configure the File System Task as shown in the screenshot.

Once the package tasks are configured, the Control Flow tab should look like as shown below.

Let’s test the package. Before that, the contents of the source folder F:\Temp are shown below. The files are dummy. Hence, the size 0 KB.

Below screenshot shows the successful execution of the package.

Below screenshots show how the files have been moved to respective destination folder that have been created based on the month names. Contents of the individual folders are shown below.

Hope that helps.