Question

I’m using MySQL for storing the data and sending requests to URLs each few minutes.

The URLs table is queried once and each request is re-sent by running the sendRequest function each minute with setInterval().

The thing is, if a URL is removed from the database, it still keeps sending the requests as I’m not querying the database before each request (and my function doesn’t know that the record is deleted).

And, I don’t want to query the related database record before each request as that’s an overkill (with lots of records).

So, I’m willing to query the database once each minute to find all deleted records and clearInterval the deleted ones. So that I won’t need to query each minute for each record but only once each minute for updated records (if I had a million URLs, this would end up in 1 query each minute rather than a million query each minute).

But, let’s say that my query told me that the database records 1,3 and 6 are deleted, how can I clearInterval only them?

Answer 1

You could store the timeout IDs linked to the record IDs in an object:

var recordIntervals = {};

recordIntervals[recordId] = setInterval(queryRecordId, time);

However that’s probably not a good idea. What I’d do it keep only one interval running every minute, query the database and then send the requests to whatever URLs the database gives me.

setInterval(function () {
  var urls = db.query('SELECT * from URLs');
  urls.forEach(function (url) {
    sendRequest(url);
  });
}, oneMinute);