Question

I have written a Perl CGI script which fetches an html page from a site using POST, then does some processing on the html data and prints output html content.

My script url: http://mysite.com/cgi-bin/p.pl

Site URL: http://site.com/interesting.asp

#!/usr/bin/perl
# p.pl
...

sub post_url {
    my( $url, $formref ) = @_;
    my $ua = new LWP::UserAgent(timeout => 300);
    my $response = $ua->post($url, $formref );

if( $response->is_success ){
    return $response->content;
} else {
    return undef;
}
}

my $url = 'url: http://site.com/interesting.asp';
my %param = ('eno' => '1234', 'submit' => 'Submit');
my $htmlcontent = post_url( $url, \%param );    # <--- Not working

... do some processing on $htmlcontent ...

... print out html page ...

The script runs fine when run from the command line but fails to fetch the HTML page from the site when run from a Webserver. Is it because the script it trying to access a page from a different domain/IP address? Can anybody suggest a workaround?

Answer 1

There could be many causes for this. Assuming that the cgi script is actually called (i.e., that it is installed in the proper location for your web server), I would ensure that LWP is actually installed on your server. Indeed, in various occasions I had to install it manually, YMMV.

It would be really helpful to output some diagnostics where the script fails… Does new LWP::UserAgent succeed? if so, what is $response->status_line after the post?

EDIT:

Since your error message is “500 permission denied”, it is likely that your web server has blocked all outgoing connection. You should check whether your provider allows outgoing http connection through some proxy and configure that into LWP::UserAgent. If there is no proxy available, then possibly there is no way to go out. We cannot help here without knowing your hosting provider settings, though; maybe you can ask your hosting provider support…