Home/ Questions/Q 610689
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-13T17:42:08+00:00

I have a Perl script that uses LWP::UserAgent to download a webpage which it

  • 0

I have a Perl script that uses LWP::UserAgent to download a webpage which it then processes using regular expressions. The problem is that portions of the webpage which are regular HTML aren’t being returned to LWP::UserAgent since the site recognizes that the browser doesn’t have Flash installed and instead returns HTML prompting us to download Flash instead of the appropriate HTML that we need to parse.

How can I make LWP::UserAgent appear to have flash installed to the web server we’re requesting the page from? I’m using the following code to initialize LWP::UserAgent:

use LWP::UserAgent;
my $ua = LWP::UserAgent->new(cookie_jar => { },requests_redirectable => [ ]);
$ua->agent('Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:9.9.9.9) Gecko/20079999 Firefox/2.0.0.1');
$ua->timeout(10);

Thanks in advance for your help!

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Both @Michael & @dma_k were correct. The server was not checking if LWP::UserAgent had flash installed. Instead, for some reason the returned content was not being dumped correctly while we were trying to debug the script. Unfortunately we didn’t figure out a way to fix this but after some trial and error we figured out how to pull out the appropriate fields from the page. Sorry that there isn’t really a correct answer for this one.

    • 0