Question

I have some code that looks like this (I left out the parts that are unimportant for the question):

$.ajax({  
    type: "POST",  
    url:  "prepXML.php",  
    data: "method=getartists&user=" + userName + "&amount=" + amount,  
    dataType: "xml",
    success: function(xml) { 
        $("artist", xml).each(function(){
            // calculate and output some stuff and then call getTracks()
            getTracks(artistName, artistNamePOST, artistPlaycount, divId, artistId);
        }); 
    }
});  

function getTracks(artistName, artistNamePost, artistPlaycount, divId, artistId){
    $.ajax({  
        type: "POST",  
        url:  "prepXML.php",  
        data: "method=gettracks&user=" + userName + "&artist=" + artistNamePOST + "&playcount=" + artistPlaycount,  
        dataType: "xml",
        success: function(xml){
            // calculate and output some stuff
        });                     
    }
});  

When this is run, it calls getTracks() fifty times and makes quite some (server) CPU load in very short time until it all gets done. What I would like to do is to group AJAX getTrack() queries by for example 5 at a time, wait until those five are done, then call the next five, wait until the next five are done, call the next five etc. The purpose of this would be to make fewer queries at basically the same time (less and more evenly spread CPU load).

I am unsure how or even if this can be done since it kind of partially beats the point of AJAX, but I would still like to make this work if possible. Could someone please point me into the right direction? Thank you.

To better understand what I need this for and what the app does, here is a link to the app. it can be tried out with any lastfm nick (if you don’t have one, you can use mine – “pootzko”). I hope I am allowed to put the link in the post(?), if not, feel free to remove it..

Answer 1

I’d consider only retrieving track info for artists that a user has clicked on. Or possibly retrieving all of the data via a single request (perhaps then after it is retrieved process it in batches via setTimeout()).

But something along the lines of the following might work to do only five requests at a time:

$.ajax({  
    type: "POST",  
    url:  "prepXML.php",  
    data: "method=getartists&user=" + userName + "&amount=" + amount,  
    dataType: "xml",
    success: function(xml) { 
      var artists = $("artist", xml),
          i = 0,
          c = 0;

      function getTracksComplete() {
         if (--c === 0)
            nextBatch();
      }

      function nextBatch() {
         for(; c < 5 && i < artists.length; i++, c++) {
            // artists[i] is the current artist record    
            // calculate and output some stuff and then call getTracks()
            getTracks(artistName, artistNamePOST, artistPlaycount, divId, artistId,
                      getTracksComplete); 
         }
      }

      // Optional - if you need to calculate statistics on all the artists
      // then do that here before starting the batches of getTracks calls
      artists.each(function() { /* do something */ });

      // Kick of the first batch of 5
      nextBatch();
   } 
});

function getTracks(artistName, artistNamePost, artistPlaycount, divId, artistId,
                   callback){
    $.ajax({  
        type: "POST",  
        url:  "prepXML.php",  
        data: "method=gettracks&user=" + userName + "&artist=" + artistNamePOST + "&playcount=" + artistPlaycount,  
        dataType: "xml",
        success: function(xml){
            // calculate and output some stuff
        },
        complete : function() {
            if (callback) callback();
        });                     
    }
});

The above was just off the top of my head (so I didn’t have time to build it to scale or to paint it), but the idea is that instead of using .each() to loop through all of the artists at once we will cache the jQuery object and do a few at a time. Calling the function nextBatch() (which is local to the success handler and thus has access to the local variables) will run a loop that calls getTracks() only 5 times, but starting processing from where we left off the previous time. Meanwhile getTracks() has been updated slightly to accept a callback function so when its ajax call completes (and note we do this on complete rather than success in case of errors) it can let the main process know that it has finished. Within the callback we keep track of how many have completed and when they all have call nextBatch() again.