Question

I have user subscriptions in my DB, and the SUBSCRIPTIONS table has UID (that I use with the CC company to charge the Card), and DATE_EXPIRED, DATE_PAID, VALID and LENGTH.

I am going to need to use some kind of a function to process a batch of subscriptions from the DB and send to the CC company for processing, and upon return, I should treat the subscription accordingly.

in the case the charge went ok, I mark VALID = 1, set DATE_PAID = NOW(), SET DATE_EXPIRED = NOW + INTERVAL LENGTH MONTH
but if the result is not ok, I need to mark VALID = ERROR_NO and do some actions, like sending an email, bring up messages etc.

My question is about the approach to the situation,
I will be getting a list of subscriptions that need to be updated, so:

1. What do you think is the best way to process a batch like that?
   My internal processing function is using cURL to contact the CC server, and gets a cURL response. upon which I know what to do next.
   Is it a file I should write all necessary subscriptions to ? Is that a cURL to my own server to batch independently ? How do I send a batch of result for processing to myself ? What do you think is the best approach ?

2. How do I keep the SELECT result to process in one time ?
   I think I should clarify. Say I theoretically have 100,000 results, I think I should seperate my SELECT to portions with LIMIT X,Y then put in memory and save on memory, but on the next SELECT I will be casting upon a perhaps different table (as it might have been updated by now). I’d like to run through all 100,000 results having the same 100,000 I had in the first SELECT. Doing all the process in the page would be uneffective of course as the whole process would drop as soon as you close the page.

Answer 1

Use LOCK TABLES subscriptions WRITE to solve your second problem, and UNLOCK the table when you are done. Although really, unless you’re running this on a server with no RAM at all, keeping these 100,000 rows in memory and going through them in a while loop using something like mysql_fetch_assoc or mysqli_fetch_assoc shouldn’t be a problem.

As to your first question, why don’t you simply process the cURL response when you get it, and save the results in the database on a per-subscription basis? I don’t really see why you would want to put this into a separate file first, and certainly not why you would then need to use cURL to contact your own server?