Question

I want to list 10.000 pages in the screen using php function file_get_contents to get information for a db.

This works until about page 500, then the script stops running (the page has loaded) and displays no error.

<?php

for ($nr=1; $nr<=10000; $nr++){

$url = "http://site.com/u$nr";
$string = file_get_contents($url);

echo '<textarea>'.$string.'</textarea>';

}

?>

@edit

I want to get this information for my computer to be manipulated via Javascript, I think that gives a lot of work to manipulate HTML using PHP.
Record these pages in db is a good idea.

Answer 1

I’d advice to set your time_limit inside the loop. Setting the time-limit will ‘extend’ timeout with the given amount of time;

<?php 
for ($nr=1; $nr<=10000; $nr++){
    $url = "http://site.com/u$nr";
    $string = file_get_contents($url);
    echo '<textarea>'.$string.'</textarea>';

    // extend the time-limit with this amount of time,
    // the amount of time this request is allowed to take
    set_time_limit(5);
}

?>

This way you won’t have to calculate the ‘total’ time limit for 10.000 records, and don’t allow a single request to take (for example) 1 hour to complete

[update]
New information posted by the OP states that he wants to process the inputs using javascript. Therefore, a better solution will be to fetch all pages 1 at a time using jQuery, process the information and submit the results to the database, using a Ajax post and server-side saving it to the database