Question

I have been tasked to write a module for importing data into a client’s system.
I thought to break the process into 4 parts:

1. Connect to the data source (SQL, Excel, Access, CSV, ActiveDirectory, Sharepoint and Oracle) – DONE
2. Get the available tables/data groups from the source – DONE
    i. Get the available fields form the selected table/data group – DONE
    ii. Get all data from the selected fields – DONE
3. Transform data to the user’s requirements
4. Write the transformed data the the MSSQL target

I am trying to plan how to handle complex data transformations like:
Get column A from Table tblA, inner joined to column FA from table tblB, and concatenate these two with a semicolon in between.

OR

Get column C from table tblC on source where column tblC.D is not in table tblG column G on target database.

My worry is not the visual, but the representation in code of this operation.

I am NOT asking for sample code, but rather for some creative ideas.

The data transformation will not be with free text, but drag and drop objects that represent actions.

I am a bit lost, and need some fresh input.

Answer 1

See my answer: Manipulate values in a datatable?