Question

I have been trying for about a day now to get an xml feed into my application. Not usually an issue and I have two other feeds coming into the application using the simpleXml method which I have attached below.

My issue is that this other page is an aspx page and it seems to have some sort of redirect or probably just using an aspx framework that is using a clean url.

This is the script I used for the other two data pulls that works fine.

$grb_feed_url = 'http://www.grb.uk.com/rss.php';
$grb_jobs = simplexml_load_file($grb_feed_url, 'SimpleXMLElement', LIBXML_NOCDATA);

That is great but when I try it for the url http://www.milkround.com/rss.aspx it returns nothing.

I then tried a cURL script, this one works fine for the godaddy example but return nothing for the Milkround url. Also strange is that if I remove the CURLOPT_FOLLOWLOCATION line or set it to 0 it returns with "object moved to here".

function get_data($url)
{
  $ch = curl_init();
  $timeout = 5;
  curl_setopt($ch,CURLOPT_URL,$url);
  curl_setopt($ch,CURLOPT_RETURNTRANSFER,1);
  curl_setopt($ch,CURLOPT_CONNECTTIMEOUT,$timeout);
  curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
  $data = curl_exec($ch);
  curl_close($ch);
  return $data;
}

$returned_content = get_data('http://www.milkround.com/rss');
print_r($returned_content);

/* example of a url that works using this script */
/* $returned_content = get_data('http://www.godaddy.com/hosting/website-builder.aspx'); */

Answer 1

You need to include a User-Agent header in the cURL request, otherwise the site produces a 501 error:

curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/4");