Question

I’m building a small PHP/Javascript app which will do some processing for all cities in all US states. This rounds up to a total of (52 x 25583) = 1330316 or less items that will need to be processed.

The processing of each item will take about 2-3 seconds, so its possible that the user could have to stare at this page for 1-2 hours (or at least keep it minimized while he did other stuff).

In order to give the user the maximum feedback, I was thinking of controlling the processing of the page via javascript, basically something like this:

var current = 1;
var max = userItems.length; // 1330316 or less

process();

function process()
{
  if (current >= max)
  {
     alert('done');
     return;
  }

  $.post("http://example.com/process", {id: current}, function()
     {
        $("#current").html(current);
        current ++;
        process();
     }
  );
}

In the html i will have the following status message which will be updated whenever the process() function is called:

<div id="progress">
   Please wait while items are processed.
   <span id="current">0</span> / <span id="max">1330316</span> items have been processed.
</div>

Hopefully you can all see how I want this to work.

My only concern is that, if those 1330316 requests are made simultaneously to the server, is there a possibility that this crashes/brings down the server? If so, if I put in an extra wait of 2 seconds per request using sleep(3); in the server-side PHP code, will that make things better?

Or is there a different mechanism for showing the user the rapid feedback such as polling which doesn’t require me to mess with apache or the server?

Answer 1

If you can place a cronjob in the server, I believe it’d work much better. What about using a cronjob to do the actual processing and use Javascript to update periodically the status (say, every 10 seconds)?

Then, the first step would be to trigger some flag that the cronjob PHP will check. If it’s active, then the task must be performed (you could use some temporary file to tell the script which records must be processsed).

The cronjob would do the task and then, when its iteration is complete, turn off the flag.

This way, the user can even close your application and check it back later, and the server will handle all the processing, uninterrupted by client activity.