Question

I’ve got a source database (Sybase), which is read-only and you can write to the database with a import file. The other side is my own database (MSSQL) which has no limitations.

The main problem is that there are no timestamps on the first database and I don’t have any access to change the source database. So is there a engine/solution to get this sync. done?

Answer 1

A diff algorithm might work, but it wouldn’t be fast, in the sense that you would have to scan the whole source database for each synchronization.

Basically you would do a full data extract, in an agreed upon, and stable, manner (ie. two such extracts with no changes between would produce identical output.)

Then you compare that to the previous extract you did, and then you can find all the changes. Something slightly more intelligent than a pure text diff would be needed, to help determine that rows weren’t just deleted+inserted, but in fact updated.

Unfortunately, if there is no way to ask the source database what the latest changes are, through, as you’ve pointed out, lack of timestamps, or similar mechanisms, then I don’t see how you can get any better than a full extract each time.

Now, I don’t know Sybase that much, but in MS SQL Server you could potentially create another database that mirrors the first, and in this second database you could make whatever changes you need.

However, if you can make such a database in Sybase, and use SQL to access both at the same time, you might be able to run queries that produce the differences.

For instance, something along the lines of:

SELECT S.*
FROM sourcedb..sourcetable1 AS S
    FULL JOIN clonedb..sourcetable1 AS C
    ON S.pkvalue = C.pkvalue
WHERE S.pkvalue IS NULL OR C.pkvalue IS NULL

This would produce rows that are inserted or deleted.

To find those that changed, you would need this WHERE-clause:

WHERE S.column1 <> C.column1
   OR S.column2 <> C.column2
   OR ....

Since the tables are joined, the WHERE-clause would filter out any rows where the previous extract and the current state is different.

Now, this might not run fast either, you would have to test to make sure.