Home/ Questions/Q 6121579
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-23T15:48:00+00:00

A generic data import module: I am reading data from any of 6 data

  • 0

A generic data import module:
I am reading data from any of 6 data source types (CSV, Active Directory, SQL, Access, Oracle, Sharepoint) into a datatable.

This data is then possibly changed by the users by casting, and calculation per column and written to a SQL table (any table selected by the user).

Doing this seems easy except that the user must also be able to replace some fields in the datatable with values from fields in the target SQL database (lookups)

I would really like to do all of the above to the datatable before sending to the target databes, but cannot, repeat NOT use Linq since the table structures (both source and target are unknown and do not represent a specific business object.

tl;dr I need to do data transformations on any datatable. Which is a good way (No Linq!)

EDIT: The source and target tables are different in structure.

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    I ended up writing a class for each database type all part of one interface, and using a GenericConnection based on DbConnection for the differenct types of sources.

    I broke the process up into:

    1. Import
    2. Transform
    3. Write
      stages that can be saved and reopened for re-use or editing.

    The Transform part consists of:

    1. Casting
    2. Calculations (string,int,decimal,date,bool) as Add, Subtract, Divide, Multiply, AND, OR, Substring, Replace
    3. Lookups against other tables
    4. Direct copying

    The transformations can be queued so that one column of data can go through any amount to match the target before being written to target.

    • 0