Question

I need to fetch a list items available in a website that list the items in paging style. I will extract information of the items on each pages. I wrote a Javascript code using a settimeout(function(){...code...}, 5000), and run this code in the Firefox web console. The script fetches all the items in the page, once this is done, the script submits the page by executing click() method of the hyperlink "next". this will continue till the page has "Next" hyperlink in the page (in last page only "back" hyperlink if present). On every page submit I timeout it to 5 seconds.

I executed this code in the Firefox web console, this run successfully and fetches the first pages results, then submits the page, the next page too loads. But, the problem is, once the page is submitted the script stops. How to run this script continuously even after the next page load? Is there any other way to achieve this functionality?

var inc=0;
var ids;
var main=function(){
console.log("attempiting..............."+inc);
var i=0;
{
    ids[inc]=b();//function, fetch the items from the page
    console.log(ids);
    var more=document.getElementById("m_more_item");
    if(more){
        //next link is there
        var cli=more.getElementsByTagName("a");
        console.log("navigating to next page "+cli[0]);
        inc++;
        setTimeout(main,5000); //I tried placing this after click, but didnt work!
        cli[0].click(); //submitting...
    } else {
        //last page reached, exiting...
        console.log("exit1-> total items fetched are: "+ids);
        return ids;
    }
}
};
main();

Answer 1

setTimeout call make you function recursive and keep looping this should work.

setTimeout(cli[0].click, 5000);

Cheers.

EDIT:
It was long for a comment so I am adding it as a answer.

You can use AJAX calls in separated page first call the page with AJAX with start=1 and on success of this request parse the data and collect information as you need and do not forget the keep lastItemIndex for your next request and post the information you need with another AJAX call on success of the second AJAX call first one with start=lastItemIndex+1.

These calls should do what you want.

I suggest using jQuery for this as it will be easy to call AJAX and parse the response.