Home/ Questions/Q 8570243
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-11T18:30:51+00:00

I want to convert data from old database to new database with new structure.

  • 0

I want to convert data from old database to new database with new structure.
in old database I have attachment table that must be convert to attachment table in new database.

old database attachment table structure is below:

Attachment (ID int, Image Image, ...)

and new database attachment table structure is below :

Attachment (ID int, Image Image, OldID Int, ...)

each time I execute convert package copy only not exists data (new data) from old database to new database.

I use below format for do it :

enter image description here

lookup between old table and new table (ID --> OldID) for check exists record.

When I run SSIS Packages; SSIS, first cache all lookups and source component data in memory then execute package. my source data in this package is very huge and when I run this package it will be run very slowly. I want to get Image column data from old database for each new record after lookup for check exists component. if I use new lookup component for get image column data from old database, SSIS cache this new lookup data and execution time of run this package not change. what must I do?

thanks in advance.

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Are you sure you’re thinking this through correctly? SSIS should not be slow even if the amount of data you are loading is huge.

    Your LOOKUP component needs to make sure it’s not doing anything it doesn’t need to. If you are pointing it to the table in the new database, change it to a SQL Query at once. In this query you only need to SELECT OldId FROM tbl and point the incoming ID from old database to this. Your data flow should contain ID and Image from Old database, which is mapped ID -> OldIdand “Image -> Image` in your OLE DB Destination. No more is needed for “Insert new rows only” operation like you are doing here.

    For this job, there is no need for any custom code or dynamic SQL. You -do- want to get the ID and Image from your source system in the data flow (unless you have major network bottlenecks to sort out) – doing a RBAR lookup to get the image data from the old system is a very backwards way of thinking your ETL.

    • 0