Question

I have 200,000 records in a database with the PK as a varchar(50)

Every 5 minutes I do a SELECT COUNT(*) FROM TABLE

If that result is greater than the List.Count I then execute

"SELECT * FROM TABLE WHERE PRIMARYKEY NOT IN ( " + myList.ToCSVString() + ")"

The reason I do this is because records are being added to the table via another process.

This query takes a long time to run and I also believe its throwing an OutOfMemoryException

Is there a better way to implement this?

Thanks

Answer 1

SQL Server has a solution for this, add a timestamp column, every time you touch any row in the table the timestamp will grow.

Add an index for the timestamp column.

Instead of just storing ids in memory, store ids and last timestamp.

To update:

• select max timestamp
• select all the rows between old max timestamp and current max timestamp
• merge that into the list

Handling deletions is a bit more tricky, but can be achieved if you tombstone as opposed to delete.