Question

I am attempting to load a webpage with file_get_contents(), and am running into problems.

Basically, my web app will use the youtube data api to search for videos based on a user’s query, and then read each video’s youtube page to find information not provided by the api. I use file_get_contents() to read the youtube pages and then load the text into a DOM parser. Most of the time, this works like a charm. However, on occasion, I’ll get warnings saying that file_get_contents() returned an empty string (not for all videos, only for some of them). I know that the url I am providing is correct, because I echo a link to that url and it works as expected. I’ll refresh the page, reopen my browser, switch to a different browser, etc. but nothing will work. Then, I’ll leave the thing for an hour or two, come back to it, and it will magically work again!

Here is a snippet of my code:

function processNext($int) {
    // this function processes the next $int videos from the youtube data api response ($xmlDoc)
    global $xmlDoc;
    $begin = count($_SESSION["results"]) - $_SESSION["start"] + 1;
        /* $_SESSION["results"] is the array of already-processed videos
           $_SESSION["start"] and $_SESSION["end"] are the indexes of the first and last videos in the $xmlDoc
        */
    $end = count($_SESSION["results"]) - $_SESSION["start"] + $int;

    for ($i = $begin; $i <= $end; $i++) {
        $video = $xmlDoc->entry[$i];
        $doc = new DOMDocument();
        $doc->strictErrorChecking = FALSE;
        libxml_use_internal_errors(true);

        // this is the line that is causing me problems
        $doc->loadHTML(file_get_contents(getWatchURL(getVidID($video->id)))); 

        $doc = $doc->documentElement;

               // then, do some processing on the $doc

    }
}

Is it possible that file_get_contents() is timing out? Is cURL a better tool for what I am doing?

UPDATE: I get the same results with cURL.

Answer 1

Look at $http_response_header. You can check the status code of the request. If it’s anything other than 200, something has likely gone wrong. More about status codes.