Question

well this is my problem

i use 2 source

first query (select * from servera.databasea.tablea)
secund query(select id, modifiedon from serverb.databaseb.tableb)
sort first query, sort second query
    merge join at left join
condition split is.. isnull(idtableb) then i do insert (insert ont serverb)
               !isnull(idtableb) && modifiedontableb<modifiedontablea then update(on server b)

it works ok with a few of rows but i work with more of 50000 and it take more than 2 hours on sort and it get error

well my another way was doing

sort on oledbsource on right-click at show advancededitor and on
input and output properties on ole db source output i choose issorted changed to true

on output columns to id i changed to
sortkeyposition to 1
(i didnt put nothing to modifiedon)
so i did these steps for 2 oledbsource
(oledb for server1, and server 2)

it work a lot of faster it finished at 5 minuts and do insert (always)
condition split doesn’t work now :s cuz always going to insert
so i on condition split added parse (DT_DBDATE) and it continues being equal (only inserts)
never going to update after i chended mofidiedon parse (DT_DATE) and it continue being equal. then my question is
(i dont want to use sort) how can i do condition split works?

Answer 1

The Sort step takes long because you are running low on memory for your sort operation. This means it will start sorting on disk, and that is horribly slow. Options to this is to use some 3rd party sorting components like NSort.

Otherwise you can do the following:

In order for your MERGE to work your inputs need to be sorted, both in the query, and using the SortKeyPosition. Also they need to be sorted the same.

Your queries should read:

SELECT * FROM servera.databasea.tablea ORDER BY id, modifiedon

SELECT id, modifiedon FROM serverb.databaseb.tableb ORDER BY id, modifiedon

Now set the IsSorted to TRUE, set SortKeyPosition 1 to id

In your MERGE step, use id for join key.

Now in your conditional split, you can use your two output cases.

Please note, if you have MULTIPLE rows per id, you need something more to sort/join on, so that you don’t get stuff in the wrong order.