Question

I’m using PhantomJS page.evaluate() to do some scraping. My problem is that the code I pass to the webkit page is sandboxed, and so has no access to the variables of my main phantom script. This makes it hard make the scraping code generic.

page.open(url, function() {
  var foo = 42;

  page.evaluate(function() {
    // this code has no access to foo
    console.log(foo);
  });
}

How could I push arguments into the page?

Answer 1

I’ve had that exact problem. It can be done with a little trickery, because page.evaluate also can accept a string.

There are several ways to do it, but I use a wrapper called evaluate, which accepts additional parameters to pass to the function that must be evaluated on the webkit side. You would use it like this:

page.open(url, function() {
  var foo = 42;

  evaluate(page, function(foo) {
    // this code has now has access to foo
    console.log(foo);
  }, foo);
});

And here is the evaluate() function:

/*
 * This function wraps WebPage.evaluate, and offers the possibility to pass
 * parameters into the webpage function. The PhantomJS issue is here:
 * 
 *   http://code.google.com/p/phantomjs/issues/detail?id=132
 * 
 * This is from comment #43.
 */
function evaluate(page, func) {
    var args = [].slice.call(arguments, 2);
    var fn = "function() { return (" + func.toString() + ").apply(this, " + JSON.stringify(args) + ");}";
    return page.evaluate(fn);
}