Question

Currently, I have a control flow that connects to more than 20 dbs (same structure)

In this controlf flow everything is the same except Data Flows which are different only in connection to db.
This is example of a dataflow:

OLE DB Source is actually what’s different.

Is it possible to use a Foreach loop in this case so that I avoid these repeatings?
If possible, how to do it? I couldn’t find any good article which describes how to connect to different dbs.

Thanks,
Ilija

EDIT:
I found an article which describes most of the thing I need: http://sql-bi-dev.blogspot.com/2010/07/dynamic-database-connection-using-ssis.html

I followed all steps form here but i don’t understand a part with query. This is a query from that article:

SELECT '' TableName, N'' ConnString, 0 RecordCount, GETDATE() ActionTime

I don’t understand this syntax and I guess this is where the errors come from:

[OLE DB Source 1] Error: There was an error with output column
“ConnString” (37) on output “OLE DB Source Output” (11). The column
status returned was: “Text was truncated or one or more characters had
no match in the target code page.”.

[OLE DB Source 1] Error: The “output column “ConnString” (37)”
failed because truncation occurred, and the truncation row disposition
on “output column “ConnString” (37)” specifies failure on truncation.
A truncation error occurred on the specified object of the specified
component.

[SSIS.Pipeline] Error: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED. The
PrimeOutput method on component “OLE DB Source” (1) returned error
code 0xC020902A. The component returned a failure code when the
pipeline engine called PrimeOutput(). The meaning of the failure code
is defined by the component, but the error is fatal and the pipeline
stopped executing. There may be error messages posted before this
with more information about the failure.

I would like to get data from table called Agency (which I defined in variable SourceTable = ‘Agency’ – if I set non-existing table then I get error, so i guess the connection to Source DB is ok)… and it confuses me that I haven’t defined anywhere which columns I need, so I guess an error is in this query, but I don’t understand its syntax. Any suggestion?

EDIT 2:
Another problem I ran into is that Message box always returns me the same value in Connection string – Variable ConnString doesn’t change. How can I assign value from SourceList variable to it?

Answer 1

I think it is possible. I would make this way:

1. Foreach Loop’s Collection should be “Foreach Item Enumerator”. At this point in the Enumerator Configuration panel you can make a little table. Add a column with a type string. Here will come the connection strings of the databases.
2. Create a string variable and associate it with this column: Go to Variable Mappings in Foreach Loop editor, choose the string variable (e.g. User::ConnString), give index 0. The loop will iterate through the entries you set in first step, and you can access the values through this User::ConnString variable.
3. Add a Data Flow task in to the Foreach Loop container. Inside set up an OLE DB Source. It needs a connection manager. Select the connection manager associated with OLE DB Connection, go to Propertes, Expressions. Select the ConnectionString property, and type @[User::ConnString] to expression column (or use your variable name).
4. Locate all connection string you currently use and type them to Enumeration Items.

I think with these steps you can simplify your data flow.

This process assumes that you have the same table names in all databases. In other case you can also iterate the table names to connect to. (Second column in the Enumeration configuration and fill the cells with the apropriate table names. You also need a second variable.)

Of course you not have to copy the whole connection string but the changing part of it. I this case you should form your expression appropriately.

EDIT

You are right. There are some problem with this example. (I think it’s a bit complicated for your solution, but it can be used.)

The first one is (as you mentioned) it did not pass the values to the variables. I would complete the 6th step with these: After you set User::SourceList as ADO object source variable, select Variable Mappings on the left side. In Variables column choose User::ConnString first (set index to 0), then User::SourceTable second (set index to 1). This will solve your second edit with the MessageBox. From now on it should show the right values.

The second problem is what you wrote, with this truncation error. If you look at the error messages carefully they say that some string columns have length of 1. And that’s why some truncation occures. To solve this, right click OLE DB source, choose Show Advanced editor. Go to the last tab (Input and Output Properties), expand OLE DB Source output/Output columns. Choose TableName first. On the right side, in Data Type Properties you should see Lenght = 1. (SSIS could not correctly determine the length of this column, because you choose SQL command variable as Data access mode, I guess). According to the Result table definition, set this value to 128. Also change the length of ConnString column from 1 to 256.

After these changes the example should work. (In my case it worked.) This example uses much the same concept that I wrote, just it gets the connection string from a data store but not from inside the package itself. Former can be a more robust solution, I accept. I hope you can make your package work. Please, let me know if not.