Question

I would like to grab other websites information, so, I have a function like this:

$doc = new DOMDocument();
@$doc->loadHTMLFile($aUrl);
$xpath = new DOMXPath($doc);  

And it will help me to get the url from the client, but I am worry that some sites may make my program crash, for example, they are timeout, or no response, or keep redirecting or return me a very big web site that may make my program no memory. How can I avoid this?

Answer 1

I would use cURL to fetch the contents of the website, since that allows for far more configuration, and you can set a couple of the options to address your concerns. This should do what you need:

<?php

$aUrl = "http://www.example.com";
$ch = curl_init($aUrl);

curl_setopt($ch, CURLOPT_TIMEOUT, 10);
curl_setopt($ch, CURLOPT_MAXREDIRS, 2);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);

$html = curl_exec($ch);
curl_close($ch);

$maxlength = 51200; // 50 kilobytes
if ($html && strlen($html) < $maxlength) {
    $doc = new DOMDocument();
    @$doc->loadHTML($html);
    $xpath = new DOMXPath($doc);

    // Do your processing here.
} else {
    die('There was an error fetching the HTML.');
}

?>

That takes care of timeouts and limits redirection to two. It also does not attempt to process pages larger than 50 kilobytes (you may want to adjust that based on how large you expect the page to be).