Question

I’ve spent a few hours playing around with this one, without success so far.

I’m outputting a very large query, and trying to split it into chunks before processing the data. This query will basically run every day, and one of the fields (‘last_checked’) will be used to ensure the same data isn’t processed more than once a day.

Here’s my existing query;

<cfquery name="getprice" maxrows="100">
  SELECT ID, source, last_checked, price
  FROM product_prices
  WHERE source='api'
  ORDER BY ID ASC
</cfquery> 

I then run a cfoutput query on the results to do various updates. The table currently holds just over 100,000 records and is starting to struggle to process everything in one hit, hence the need to split it into chunks.

My intention is to cfschedule it to run every so often (I’ll increase the maxrows and probably have it run every 15 minutes, for example). However, I need it to only return results that haven’t been updated within the last 24 hours – this is where I’m getting stuck.

I know MySQL has it’s own DateDiff and TimeDiff functions, but I don’t seem to be able to grasp the syntax for that – if indeed its applicable for my use (docs seem to contradict themselves in that regard – or, at the least the ones I’ve read).

Any pointers very much appreciated!

Answer 1

Try this with MySQL first:

SELECT ID, source, last_checked, price
  FROM product_prices
  WHERE source='api'
      AND last_checked >= current_timestamp - INTERVAL 24 HOUR
  ORDER BY ID ASC