Question

This post discusses how to inform the user via a progress bar by calculating the percentage of the file being downloaded by a www::mechanize Perl script.

I need to inform my user of a www::mechanize script’s progress but not necessarily in percentage remaining. The script is migrating through a user’s account and picking up data, which can vary significantly in size so the percentage factor is unknown.

If I can show “progress” via some js DOM div writing (in the dialog that shows a “loader” image while the perl script is running) then that would suffice.

Can I inject js script like:

<script>
$('#formboxtext').html('Logging into account ...');
</script>

into the Perl script to show progression to my user? Or does the Perl script have to return before the DOM will get updated? (The answer appears to be “no”.)

Are JavaScript::SpiderMonkey and WWW::Scripter modules I need to make this happen, or is the solution more simple?

EDIT:

I am expanding a PHP-based CMS. I have written a screen-scraping script using Perl and the module www::Mechanize which traverses several pages of a user’s account on another site for retrieval into mySQL databases. I will display the collected content in a php form for the user to save once the Perl script is completed. The collection process will range from 10 seconds to a minute. I would like to display progression as the script navigates the user’s account pages collecting information.

To begin, I supply the user a jQuery modal dialog (calling a php file to fill it’s contents with a form of username and password inputs) used to login to their personal account. I show a loader image in this dialog and a sentence asking for patience, but I would like to use jQuery to rewrite this sentence (div) as the Perl script navigates through pages, sending back progress to display; such as, “I’m here now. Now I’m over here.” If an error occurs (i.e. bad login) then I can rewrite the modal dialog and offer solutions per usual. If success, then I would close the dialog and display the collected information in form inputs ready for saving into my database – on the php page that spawned the modal dialog.

If all of this entails multiple DOMs, forking processes and returning control from one script execution to another… then I am definitely over my head. BUT, I would like to learn it. 🙂 I would appreciate an overview on what to read and learn. If it’s actually much simpler than I realize then I would appreciate that answer too.

Thanks to daxim and reinierpost for their patience and advice. Thanks for any help.

The Answer:
Recap: For me, I decided to fake the progress bar showing progression by estimating the time it would take. That has worked nicely. This post shows how you can dup output from a perl script back to the calling php script, but then feeding that information back to the original DOM was becoming too complex for it’s worth. It was made more complex by various parameters passed to the perl script which changed the progression and output. “Faking it” proved a nice solution. Hey, now I see why my girlfriend does it! 🙂

PS. I gave the green checkmark to daxim because he has answered other questions of mine and been a big help to me, even though he was sort of shooting in the dark on this one.

Answer 1

Instead of printing progress to STDOUT as in https://stackoverflow.com/a/1938448, expose the number as a Web service. This should suffice:

sub {
    return [200, [Content_Type => 'text/plain'], [$PROGRESS]]
}

On the client side, use jQuery get to poll the Web service every half second or so and jQuery Progressbar to display it.

---

Edit: code example

use 5.010;
use strictures;
use DBI qw();
use Plack::Request qw();
use POSIX qw(floor);
use Forks::Super qw(fork);
use WWW::Mechanize qw();

sub db_connect {
    return DBI->connect('dbi:SQLite:dbname=/var/tmp/progress.db');
}

sub {
    my ($env) = @_;
    my $req = Plack::Request->new($env);
    if ('/progress' eq $req->path_info) {
        return [200,
            [Content_Type => 'text/html'],
            [db_connect->selectrow_array('select progress from progress')]
        ]
    } else {
        fork(sub => sub {
            state $total = 0;
            my $dbh = db_connect;
            $dbh->do('delete from progress');
            $dbh->do('insert into progress (progress) values (0)');
            WWW::Mechanize->new->get(
                'http://localhost:5000/VBoxGuestAdditions_4.0.4.iso', # large-ish file
                ':read_size_hint' => 1024**2,
                ':content_cb' => sub {
                    my ($data, $response, $proto) = @_;
                    $total += length($data);
                    my $size = $response->header('Content-Length');
                    $dbh->do(
                        'update progress set progress = ?', {}, floor(($total/$size)*100)
                    );
sleep 1;
                },
            );
        }) unless defined db_connect->selectrow_array('select progress from progress');
        return [200,
            [Content_Type => 'text/html'],
            [q~<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title></title>
<link rel="stylesheet" href="http://code.jquery.com/ui/1.8.18/themes/base/jquery-ui.css" />
<style>
#progress { width: 80em; height: 5em; border: 1px solid black; }
</style>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js"></script>
<script src="https://ajax.googleapis.com/ajax/libs/jqueryui/1.8.18/jquery-ui.min.js"></script>
<script>
jQuery(document).ready(function() {
    // TODO: stop the timer if progress == 100 or no response
    var timer = setInterval(function() {
        jQuery.ajax({ async: false, cache: false, dataType: 'html', url: 'http://localhost:5001/progress' }).done(function(progress) {
            jQuery('#progress').progressbar({ value: parseInt(progress) });
        });
    }, 500);
});
</script>
</head>
<body>
<h1>downloading your stuff</h1>
<div id="progress"><div>
</body>
</html>~]
        ]
    }
};