Home/ Questions/Q 111009
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-11T02:19:01+00:00

We have a bit of a messy database situation. Our main back-office system is

  • 0

We have a bit of a messy database situation.

Our main back-office system is written in Visual Fox Pro with local data (yes, I know!)

In order to effectively work with the data in our websites, we have chosen to regularly export data to a SQL database. However the process that does this basically clears out the tables each time and does a re-insert.

This means we have two SQL databases – one that our FoxPro export process writes to, and another that our websites read from.

This question is concerned with the transform from one SQL database to the other (SqlFoxProData -> SqlWebData).

For a particular table (one of our main application tables), because various data transformations take places during this process, it’s not a straightforward UPDATE, INSERT and DELETE statements using self-joins, but we’re having to use cursors instead (I know!)

This has been working fine for many months but now we are starting to hit upon performance problems when an update is taking place (this can happen regularly during the day)

Basically when we are updating SqlWebData.ImportantTable from SqlFoxProData.ImportantTable, it’s causing occasional connection timeouts/deadlocks/other problems on the live websites.

I’ve worked hard at optimising queries, caching etc etc, but it’s come to a point where I’m looking for another strategy to update the data.

One idea that has come to mind is to have two copies of ImportantTable (A and B), some concept of which table is currently ‘active’, updating the non-active table, then switching the currenly actice table

i.e. websites read from ImportantTableA whilst we’re updating ImportantTableB, then we switch websites to read from ImportantTableB.

Question is, is this feasible and a good idea? I have done something like it before but I’m not convinced it’s necessarily good for optimisation/indexing etc.

Any suggestions welcome, I know this is a messy situation… and the long term goal would be to get our FoxPro application pointing to SQL.

(We’re using SQL 2005 if it helps)

I should add that data consistency isn’t particularly important in the instance, seeing as the data is always slightly out of date

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. There are a lot of ways to skin this cat.

    I would attack the locking issues first. It is extremely rare that I would use CURSORS, and I think improving the performance and locking behavior there might resolve a lot of your issues.

    I expect that I would solve it by using two separate staging tables. One for the FoxPro export in SQL and one transformed into the final format in SQL side-by-side. Then either swapping the final for production using sp_rename, or simply using 3 INSERT/UPDATE/DELETE transactions to apply all changes from the final table to production. Either way, there is going to be some locking there, but how big are we talking about?

    • 0