Question

What’s the simplest way to launch Firefox, load a 3rd party website (which I’m authorised to “automate”), and run some “privileged” APIs against that site? (e.g: nsIProgressListener, nsIWindowMediator, etc).

I’ve tried a two approaches:

1. Create a tabbed browser using XULrunner, “plumbing” all the appropriate APIs required for the 3rd party site to open new windows, follow 302 redirects, etc. Doing it this way, it’s an aweful lot of code, and requires (afaict) that the user installs the app, or runs Firefox with -app. It’s also extremely fragile. :-/

2. Launch Firefox passing URL of the 3rd party site, with MozRepl already listening. Then shortly after startup, telnet from the “launch” script to MozRepl, use mozIJSSubScriptLoader::loadSubScript to load my code, then execute my code from MozRepl in the context of the 3rd party site — this is the way I’m currently doing it.

With the first approach, I’m getting lots of security issues (obviously) to work around, and it seems like I’m writing 10x more browser “plumbing” code then automation code.

With the second approach, I’m seeing lots of “timing issues”, i.e:

• the 3rd party site is somehow prevented from loading by MozRepl (or the execution of the privileged code I supply)???, or
• the 3rd party site loads, but code executed by MozRepl doesn’t see it load, or
• the 3rd party site loads, and MozRepl isn’t ready to take requests (despite other JavaScript running in the page, and port 4242 being bound by the Firefox process),
• etc.

I thought about maybe doing something like this:

Modify the MozRepl source in some way to load privileged JavaScript from a predictable place in the filesystem at start-up (or interact with Firefox command-line arguments) and execute it in the context of the 3rd party website.

… or even write another similar add-on which is more dedicated to the task.

Any simpler ideas?

---

Update:

After a lot of trial-and-error, answered my own question (below).

Answer 1

I found the easiest way was to write a purpose-built Firefox extension!

Step 1. I didn’t want to do a bunch of unnecessary XUL/addon related stuff that wasn’t necessary; A “Bootstrapped” (or re-startless) extension needs only an install.rdf file to identify the addon, and a bootstrap.js file to implement the bootstrap interface.

• Bootstrapped Extension: https://developer.mozilla.org/en-US/docs/Extensions/Bootstrapped_extensions

• Good example: http://blog.fpmurphy.com/2011/02/firefox-4-restartless-add-ons.html

The bootstrap interface can be implemented very simply:

const path = '/PATH/TO/EXTERNAL/CODE.js';
const Cc = Components.classes;
const Ci = Components.interfaces;
const Cu = Components.utils;
var loaderSvc = Cc["@mozilla.org/moz/jssubscript-loader;1"];
                    .getService(Ci.mozIJSSubScriptLoader);

function install() {}
function uninstall() {}
function shutdown(data, reason) {}
function startup(data, reason) { loaderSvc.loadSubScript("file://"+path); }

You compile the extension by putting install.rdf and bootstrap.js into the top-level of a new zip file, and rename the zip file extension to .xpi.

Step 2. To have a repeatable environment for production & testing, I found the easiest way was to launch Firefox with a profile dedicated to the automation task:

• Launch the Firefox profile manager: firefox -ProfileManager
• Create a new profile, specifying the location for easy re-use (I called mine testing-profile) and then exit the profile manager.
• Remove the new profile from profiles.ini in your user’s mozilla config (so that it won’t interfere with normal browsing).
• Launch Firefox with that profile: firefox -profile /path/to/testing-profile
• Install the extension from the file-system (rather than addons.mozilla.org).
• Do anything else needed to prepare the profile. (e.g: I needed to add 3rd party certificates and allow pop-up windows for the relevant domain.)
• Leave a single about:blank tab open, then exit Firefox.
• Snapshot the profile: tar cvf testing-profile-snapshot.tar /path/to/testing-profile

From that point onward, every time I run the automation, I unpack testing-profile-snapshot.tar over the existing testing-profile folder and run firefox -profile /path/to/testing-profile about:blank to use the “pristine” profile.

Step 3. So now when I launch Firefox with the testing-profile it will “include” the external code at /PATH/TO/EXTERNAL/CODE.js on each start-up.

NOTE: I found that I had to move the /PATH/TO/EXTERNAL/ folder elsewhere during step 2 above, as the external JavaScript code would be cached (!!! – undesirable during development) inside the profile (i.e: changes to the external code wouldn’t be seen on next launch).

The external code is privileged and can use any of the Mozilla platform APIs. There is however an issue of timing. The moment-in-time at which the external code is included (and hence executed) is one at which no Chrome window objects (and so no DOMWindow objects) yet exist.

So then we need to wait around until there’s a useful DOMWindow object:

// useful services.
Cu.import("resource://gre/modules/Services.jsm");    
var loader = Cc["@mozilla.org/moz/jssubscript-loader;1"]
                .getService(Ci.mozIJSSubScriptLoader);

var wmSvc = Cc["@mozilla.org/appshell/window-mediator;1"]
                .getService(Ci.nsIWindowMediator);

var logSvc = Cc["@mozilla.org/consoleservice;1"]
                .getService(Ci.nsIConsoleService);

// "user" code entry point.
function user_code() {   
   // your code here!
   // window, gBrowser, etc work as per MozRepl!
}

// get the gBrowser, first (about:blank) domWindow, 
// and set up common globals.
var done_startup = 0;
var windowListener;
function do_startup(win) {

    if (done_startup) return;
    done_startup = 1;
    wm.removeListener(windowListener);

    var browserEnum = wm.getEnumerator("navigator:browser");
    var browserWin = browserEnum.getNext();
    var tabbrowser = browserWin.gBrowser;
    var currentBrowser = tabbrowser.getBrowserAtIndex(0);
    var domWindow = currentBrowser.contentWindow;
    window = domWindow.QueryInterface(Ci.nsIInterfaceRequestor)
                 .getInterface(Ci.nsIWebNavigation)
                 .QueryInterface(Ci.nsIDocShellTreeItem)
                 .rootTreeItem.QueryInterface(Ci.nsIInterfaceRequestor)
                 .getInterface(Ci.nsIDOMWindow);
    gBrowser = window.gBrowser;

    setTimeout = window.setTimeout;
    setInterval = window.setInterval;
    alert = function(message) { 
        Services.prompt.alert(null, "alert", message); 
    };
    console = { 
        log: function(message) { 
            logSvc.logStringMessage(message); 
        } 
    };

    // the first domWindow will finish loading a little later than gBrowser...
    gBrowser.addEventListener('load', function() {
        gBrowser.removeEventListener('load', arguments.callee, true);
        user_code();
    }, true);
}

// window listener implementation
windowListener = {
    onWindowTitleChange: function(aWindow, aTitle) {},
    onCloseWindow:       function(aWindow) {},
    onOpenWindow:        function(aWindow) {
        var win = aWindow.QueryInterface(Ci.nsIInterfaceRequestor)
                     .getInterface(Ci.nsIDOMWindowInternal || Ci.nsIDOMWindow);
        win.addEventListener("load", function(aEvent) {
            win.removeEventListener("load", arguments.callee, false);
            if (aEvent.originalTarget.nodeName != "#document") return;
            do_startup();
        }
};

// CODE ENTRY POINT!
wm.addListener(windowListener);

Step 4. All of that code executes in the “global” scope. If you later need to load other JavaScript files (e.g: jQuery), call loadSubscript explicitly within the null (global!) scope

function some_user_code() {
    loader.loadSubScript.call(null,"file:///PATH/TO/SOME/CODE.js");
    loader.loadSubScript.call(null,"http://HOST/PATH/TO/jquery.js");
    $ = jQuery = window.$;
}

Now we can use jQuery on any DOMWindow by passing <DOMWindow>.document as the second parameter to the selector call!