i have a sql query with multiple joins & it pulls data from a database for processing. This is supposed to be running on some scheduled basis. So day 1, it might pull 500, day 2 say 400.
Now, if the service is stopped for some reason & the data not processed, then on day3 there could be as much as 1000 records to process. This is causing timeout on the sql query.
How best to handle this situation without causing timeout & gradually reducing workload to process?
TIA
create a batch process. Allow no more than say n records to process. Lets say n = 100 …
then make your select queries to only select top 100 until there are no more records to process.
This will allow your command to run forever.
Note this could cause database locks and other issues. If you use the batch process I described above and determine the longest running query you can set your connect timeout to what is necessary.