We have 3 SQL Servers (A, B and C). Server A has linked server connections to B and C. We have a number of Views on A that join data from B and C. No surprise that these views are horribly slow, taking hours to query. We are talking 1000s of records in a number of tables in B and C.
Our idea is to move data from B and C into local tables on A and then point the views to the local tables. We came up with a number of options for data transfer:
- Each night, replicate data from B and C to A, moving only the deltas across to the tables on A (rather than the whole dataset)
- Each night, restore the B and C databases to local databases on A and point all views to the local databases
- Write a custom stored procedure on A that, each night, DROPs local tables and fills them with data from B and C using SELECT * INTO. Point all views on A to the local tables.
- Similar to option 3, write a custom stored procedure on A that, each night, calculates the deltas between the local tables on A and the equivalent tables on B and C. Point all views on A to the local tables.
We did not want to go with option 1 because (according to our senior DBA) the replication would modify the tables on B and C (e.g. adding columns called msrepl_tran_version). The systems on B and C are 3rd party applications that we can’t modify.
We did not want to go with option 2 because we don’t want to have EVERY table from B and C. We only want a subset of the tables. These databases are huge and restoring them would take up way too much space.
We are currently exploring option 3. However, it doesn’t sit well with me because, every night, we are moving a large amount of data and 90-something percent of the data we bring across from B and C is the same as the previous night.
The senior DBA suggested option 4 as the only other way besides option 1 to move deltas across each night. However, this seems like a lot of work. Surely this is a common problem and can be solved without writing custom code?
Does anyone have any other suggestions?
You will never be able to detects the deltas correctly. Do not reinvent the wheel, the problem you describe has a simple straight forward solution:
Transactional replication does not modify the tables in any fashion. The replication agent only mines the log for changes and transforms them into updates that are staged into distribution db and then applied to subscribers. No changes occur to any of them 3rd party application, the table schema is not modified. You may have been misled into thinking replication does not work because you only considered merge replication, which is not appropriate for your problem description (and, unlike transactional replication, does indeed require structural changes to the published articles).