Question

Is it possible for a perl cgi script to segment its AJAX responses into numerous individual HTTP responses?

Say I have this code:

xmlhttp=new XMLHttpRequest();
xmlhttp.onreadystatechange=function()
{
    if (xmlhttp.readyState==4 && xmlhttp.status==200)
    {
        onDataReceived(xmlhttp.responseText);
    }
    else if(xmlhttp.status!=200 && xmlhttp.status!=0) {    }
}
xmlhttp.open("POST","script.cgi",true);
xmlhttp.setRequestHeader("Content-type","application/x-www-form-urlencoded");
xmlhttp.send(toURLString(options));

as javascript (dont tell me about xml object compatibility issues with ie, I know, and don’t care).

and this:

print "Content-type: text/html\n\n";

my %form = Vars();
if($ENV{REQUEST_METHOD} eq "POST" )
{
    $|=1;
    for(my $i, (1..100000000))
    {
        print "1\n";
    }
}

as perl cgi. Is it possible to print out this result in numerous individual packets of 1s, instead of generating 100000000 1s before finally having an output?

Answer 1

Please see this SO question for possible approaches, though it’s not Perl specific:

Dealing with incremental server response in AJAX (in JavaScript)

From the linked Wiki article, this link seems most relevant: http://en.wikipedia.org/wiki/Comet_%28programming%29#XMLHttpRequest

However, I would strongly suggest considering a polling approach instead of the “server push” one you are considering:

The server stores the chunks of data as accessible files (with some ordering meta info)

print "Location: xxxx"; 
# Sorry, forgot the exact form of Location HTTP response.
# Location points to URL mapped to /home/htdocs/webdocs/tmp/chunk_0.html
my %form = Vars();
if($ENV{REQUEST_METHOD} eq "POST" )
{
    $|=1;
    $file_num = 0;
    my $fh;
    for(my $i, (1..100000000))
    {
        if ($i % 1000 == 0) {
            close $fh if $fh;
            open $fh, ">", "/home/htdocs/webdocs/tmp/chunk_${file_num}.html";
            # Add the usual error handling on open/close i'm too lazy to type
            $file_num++;
        }
        print $fh "1\n";
    }
    print $fh "\n##############END_TRANSMISSION__LAST_FILE####################\n";
    # This was a singularly dumb way of marking EOF but you get the drift
    close $fh;
}

The AJAX poller retrieves them in a loop one by one, processing the response containing the next chunk and looking for meta-info to know what (and if) the next piece to poll for is.