Question

I have a web application based on Django. I use the Scrapy Crawler to crawl webpages. My goal, at the moment, is to be able to control the crawler from within a webpage using jQuery and AJAX requests.

My theoretical setup is the following:

• On the webpage, I have a button. When I click the button, the crawler is started on the server-side.
• Once the crawler has started, I periodically send AJAX GET requests to the server using window.setInterval to find out how many webpages have been crawled so far.
• Once the crawler has finished, the GET requests should stop by using window.clearInterval.

These are the relevant lines from my current code:

$(document).ready(function() {

    // This variable will hold the ID returned by setInterval
    var monitorCrawlerId;

    $startCrawlerButton.on('click', function(event) {

        // This function should be run periodically using setInterval
        var monitorCrawler = function() {

            $.ajax({

                type: 'GET',
                url: '/monitor_crawler/',
                // ...
                success: function(response) {

                    // if the server sends the message that the crawler
                    // has stopped, use clearInterval to stop executing this function
                    if (response.crawler_status == 'finished') {

                        clearInterval(monitorCrawlerId);

                    }

                } 

            });

        };

        // Here I send an AJAX POST request to the server to start the crawler
        $.ajax({

            type: 'POST',
            url: '/start_crawler/',
            // ...
            success: function(response) {

                // If the form that the button belongs to validates correctly,
                // call setInterval with the function monitorCrawler defined above
                if (response.validation_status == 'success') {

                    monitorCrawlerId = setInterval('monitorCrawler()', 10000);

                }

            }

        });

    });
});

The problem: When I execute this code, I get this in Firefox’s web console:

ReferenceError: monitorCrawler is not defined

The strange thing, however, is that the function monitorCrawler gets periodically executed anyway. But with every execution, I get the same error message again. If I put monitorCrawler outside of $startCrawlerButton.on() I still get the same errors. How can I resolve this? Since I’m a JavaScript newbie, any help is appreciated. Thank you very much!

Answer 1

setInterval, when first parameter is string, resolves in the global (window) context. You can give it a variable pointing to the function to be called or even:

setInterval(function(){monitorCrawler();}, 10000);

this will create a closure where local variable monitorCrawler will still exist when interval fires.