Question

I am trying to use LWP::Simple to make a GET request to a REST service. Here’s the simple code:

use LWP::Simple;
$uri = "http://api.stackoverflow.com/0.8/questions/tagged/php";
$jsonresponse= get $uri;
print $jsonresponse;

On my local machine, running Ubuntu 10.4, and Perl version 5.10.1:

farhan@farhan-lnx:~$ perl –version
This is perl, v5.10.1 (*) built for x86_64-linux-gnu-thread-multi

I can get the correct response and have it printed on the screen. E.g.:

farhan@farhan-lnx:~$ head -10 output.txt
{
“total”: 1000,
“page”: 1,
“pagesize”: 30,
“questions”: [
{
“tags”: [
“php”,
“arrays”,
“coding-style”
(… snipped …)

But on my host’s machine to which I SSH into, I get garbage printed on the screen for the same exact code. I am assuming it has something to do with the encoding, but the REST service does not return the character set type in the response, so how do I force LWP::Simple to use the correct encoding? Any ideas what may be going on here?

Here’s the version of Perl on my host’s machine:

[dredd]$ perl –version
This is perl, v5.8.8 built for x86_64-linux-gnu-thread-multi

Answer 1

I happen to have a 64 bit RHEL 5.4 box which has Perl 5.8.8 on it. I took your code and got the exact same result. I tried using Data::Dumper to dump the data, but that didn’t change anything. I then went to the command line and did this:

 wget -O jsonfile http://api.stackoverflow.com/0.8/questions/tagged/php
 --2010-05-26 11:42:41--  http://api.stackoverflow.com/0.8/questions/tagged/php
 Resolving api.stackoverflow.com... 69.59.196.211
 Connecting to api.stackoverflow.com|69.59.196.211|:80... connected.
 HTTP request sent, awaiting response... 200 OK
 Length: 5430 (5.3K) [application/json]
 Saving to: `jsonfile'
 2010-05-26 11:42:42 (56.9 KB/s) - `jsonfile' saved [5430/5430]

When I did this:

 file jsonfile

I got:

jsonfile: gzip compressed data, from FAT filesystem (MS-DOS, OS/2, NT), max speed

So, the JSON data was gzipped by the web server. I tried this:

gzip -dc jsonfile

and lo and behold the results are the JSON data as you would expect.

What you can do now is to either use another module to ungzip the data, or you can check out this other thread which shows how to accept gzip using LWP::UserAgent and handle the request that way