Question

I am pulling Nintendo DS prices from this website using lynx -dump.

For example, let’s say I am going to pull from the webpage for the game Yoshi Touch and Go:

/usr/bin/lynx -dump -width=150 http://videogames.pricecharting.com/game/nintendo-ds/Yoshi-Touch-and-Go

Everything works fine and I can use Regex to pull the prices easily. The problem comes from when the URL contain’s an apostrophe (‘) or an ampersand(&) as that brings up an error. So let’s say I try and find the page for the game Yoshi’s Island DS, I would use this line of code:

/usr/bin/lynx -dump -width=150 http://videogames.pricecharting.com/game/nintendo-ds/Yoshi's-Island-DS

which would give me these little errors:

sh: -c: line 0: unexpected EOF while looking for matching `''
sh: -c: line 1: syntax error: unexpected end of file

Here is the code I use to call the -dump with $fullURL being the string containing: “http://videogames.pricecharting.com/game/nintendo-ds/Yoshi’s-Island-DS”.

$command     = "/usr/bin/lynx -dump -width=150 $fullURL";
@pageFile = `$command`;

Could anyone help me find a solution that will turn the $fullURL string into a URL compatible string?

Answer 1

You need to escape the ' in your URL before it is passed to the shell. Perl provides to quotemeta function to perform the needed escapes for most shells.

my $quoted_URL = quotemeta($fullURL);
$command     = "/usr/bin/lynx -dump -width=150 $quoted_URL";
...

You can also use the \Q and \E escapes in the string for the same result.

$command     = "/usr/bin/lynx -dump -width=150 \Q$fullURL\E";
...