Question

I am trying to navigate to urls that are created from the script itself.

This sample code does not work as (I had) expected. Can’t figure out why 🙁

var casper = require('casper').create({
    viewportSize:{
        width:1024, height:768
    },
    pageSettings:{
        userAgent:'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_2) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.79 Safari/535.11'
    },
    verbose:true
});

casper.on('open', function (location) {
    console.log(location + ' loaded');
});

casper.start('http://www.google.com', function() {
    this.test.assertTitle('Google', 'Google homepage title is the one expected');
});

casper.mytest = '';

casper.then(function () {
    casper.mytest = 'http://www.yahoo.com';
});

casper.thenOpen(casper.mytest, function() {
    this.test.assertTitle('Yahoo', 'Yahoo homepage title is the one expected');
});

casper.run(function () {
        casper.exit();
    }
);

The result is that the second page does not load:

http://www.google.com loaded
PASS Google homepage title is the one expected
 loaded    
FAIL Yahoo homepage title is the one expected
#    type: assertTitle
#    subject: ""
#    expected: "Yahoo"

Answer 1

I think, the reason for your problem is that at the moment, when you register thenOpen step for Yahoo the variable casper.mytest is empty. This value gets into the CasperJS’s map of steps at this moment, and it does not matter that you change the source variable in the step before.

The blog post Webscraping with CasperJS and PhantomJS may be helpful as an example of fetching dynamically constructed urls.