Automate NW.js UI testing

NW.js, previously known as node-webkit is a technology that enables a whole new way of writing desktop applications using web technologies. If you are not familier with it, I strongly suggest you get acquainted, and after you do, get back to this post.

Recently I ran into this nifty module called chrome-remote-interface, a Remote Debugging Protocol interface to instrument Chrome.

Quite a few of the products we develop at ironSource are based on NW.js (which is based on Chromium) so first thing that popped into my mind was "hey lets try and see if this works with NW.js?"

Long story short, it does! and its a very interesting lightweight alternative for implementing UI testing automation, using javascript all around and without additonal tools like selenium, in situations where only chromium is involved.

TL;DR - the full source code of this post is available here. The most pertinent file is test.js.

The following is a step by step guide on how to make this happen...

1. Setting up an NW.js app

To quickly get started, clone nw-ninja, a minimalistic NW.js starter project I wrote for 2014 workshops

    $ git clone nwjs-ui-testing

Followed by

    $ npm install && cd nwapp && npm install

and finally

    $ gulp run

Sit back and relax while node-webkit-builder downloads NW.js for you and launch the application. Once launched, you should see a default bootstrap demo page. I changed my index.html to look like this:


2. Preparing the test suite

In this guide I will use mocha (npm install -g mocha if you dont have it already) as my test framework. You can probably use your favorite one instead if you so wish it.

To start, create this test file, named test.js (surprise, surprise!) in the root directory of the project:

// reuse the gulp task that runs our app
var run = require('./task/run.js')

describe('my app', function () {  
    var nw, cri

    it('sends a message to the server', function (done) {
        // soon we will replace this with the instrumentation code
        setTimeout(done, 5000)

    // before every test, launch nw.js
    beforeEach(function(done) {
        // invoke run and assign the instance of NwBuilder so we can use it later
        nw = run()
        done() // This function will be asynchronous shortly, hang tight :)

    // after every test shut down nw.js if it's running
    afterEach(function(done) {        
        if (nw.isAppRunning()) {            
            nw.getAppProcess().on('close', done).kill()
        } else {
            setTimeout(done, 1000)

This code will launch our app before each test case and close it afterwards. It reuses the gulp task code, but if you don't want it feel free to replace it.

Execute the following in the root directory of the project:

    $ mocha test.js --remote-debugging-port=9222

If everything went well, you should see a "hello world" app launching itself (it might not become the foreground window though). The app should stay on for a few seconds and exit. FYI, Mocha will only accept externeous command line arguments in the above format; mocha --something won't work.

In this test code, I'm only passing the debugging port parameter to nw.js (see the run task and ninjaConfig). In a real world scenario, this port should also be used when initializing chrome-remote-interface (see the options.port parameter).

3. Adding chrome-remote-interface

Now we will introduce the "nifty module" into the mix:

    $ npm install --save chrome-remote-interface

The way chrome-remote-interface instrument chrome is through a communication channel to the browser on port 9222 (the one we told mocha cli about, earlier).

We require it in our test:

var chromeRemoteInterface = require('chrome-remote-interface')  

And then change beforeEach to connect to our nw.js app after it starts:

beforeEach(function(done) {  
    // invoke run and assign an instance of NwBuilder
    // so we can use it later
    nw = run()
    setTimeout(function () {
        // connect to nw.js app
        chromeRemoteInterface(function(c) {
            cri = c
        }).once('error', done)
    }, 1000)

For the sake of simplicity I used a timeout before attempting a connection but more elaborate means can be employed in real world scenarions, like checking if the port is available before proceeding (or sending a PR to node-webkit-builder that turns NwBuilder into an event emitter with an event .on('app started') or something)

Next, we modify afterEach to disconnect:

afterEach(function(done) {  
    if (cri) {

    if (nw.isAppRunning()) {            
        nw.getAppProcess().on('close', done).kill()
    } else {
        setTimeout(done, 1000)

This section runs a test (its a weak test but sufficient for now) to see if chrome was successfully connected to in beforeEach, if it did, the connection is terminated.

Now that the stage is set we may proceed with implementing the actual test code.

4. Simulating a click

In order to simulate a click we will need to invoke the dispatchMouseEvent method using the remote interface. The method requires, among other things, the coordinates for the event, relative to the viewport's origin point.

Simulating a "click" on the send button should be as simple as dispatching the event through the remote interface with a set of coordinates that fall inside the area of the button but in reality it is slightly more complex.

I couldn't find any straight forward way to get the coordinates of an element using the remote interface (and if you do, please let me know) so I had to improvise.

To start, lets instruct nw.js to evaluate some javascript code that will determine the coordinates of the send button and then we'll retrieve the results so we can use them when we dispatch the mouse event.

We need to add some dependencies first:

    $ npm install --save async lodash

And in test.js require the following:

var async = require('async')  
var _ = require('lodash')

// more on that below
var findPositionInWindow = require('./findPositionInWindow.js')  

Now we'll add this method, which encapsulates all the gory details, to test.js:

function getElementPosition(cri, selector, callback) {  
        function evalFindPos (cb) {
                expression: findPositionInWindow.toString() + '\nfindPositionInWindow(\'a#send\')'
            }, cb)
        function getFindPosResults (remoteEval, cb) {
                objectId: remoteEval.result.objectId
            }, cb)
    ], function(err, findPosResults) {
        if (err) return callback(err)

        var left = _.find(findPosResults.result, {
            name: 'left'

        var top = _.find(findPosResults.result, {
            name: 'top'

        callback(null, {
            x: left.value.value,
            y: top.value.value

This method executes two calls using the remote interface and async.js.

Firstly, it asks nw.js to evaluate a script, composed of the function declaration of findPositionInWindow (from findPositionInWindow.js) and a call that function right away with a query selector targeting the send button. Nw.js will evaluate this code (roughly):

function findPositionInWindow() { ... implementation ... }  

Secondly, getElementPosition sends a request to nw.js to get the result of the findPositionInWindow invocation and lastly, it extracts the top and left properties returning them to the caller as { x: ..., y: ... } object via the callback.

Sending the click command

The click is actually composed of two dispatchMouseEvent calls, one with the type mousePressed and the other with the type mouseReleased.

Lets add a click function to test.js:

function click(cri, x, y, callback) {  
    var baseEventInfo = { x: x, y: y, button: 'left', clickCount: 1}

    var mousePressed = _.clone(baseEventInfo)
    mousePressed.type = 'mousePressed'

    var mouseReleased = _.clone(baseEventInfo)
    mouseReleased.type = 'mouseReleased'

        _.bind(cri.Input.dispatchMouseEvent, cri.Input, mousePressed),
        _.bind(cri.Input.dispatchMouseEvent, cri.Input, mouseReleased)
    ], callback)    

So now that we have a way to get the position of the send button and click it, lets combine these two in the actual test:

    it('sends a message to the server', function(done) {
        // get the position of the send button  
        getElementPosition(cri, 'a#send', function(err, pos) {
            if (err) return done(err)

            // click it
            click(cri, pos.x + 1, pos.y + 1, function(err, result) {
                if (err) return done(err)

                setTimeout(done, 3000)

Of course, in real life situations one would have a simpler API composed of such methods similar to nightmarejs for example...

5. Listening to network traffic

So we managed to click a button (or atleast we think we did) in the UI but we still need a way to verify that this button does its job, since the results of the dispatchMouseEvent doesn't tell us much.

The click handler for the send button (located at nwapp/index.js) looks like this:

$(function () {
    $('a#send').click(function () {
        $.get('http://localhost/api', function(result) {

Remember this is client side code not test or server code.

To verify that a request is made, we could start a local http server just for the test and then correlate between it and the simulated click. However, to demonstrate another capability of the remote interface and to achieve greater confidence in our assertion we'll use a different method: listening to outgoing requests.

Our test code will change again, to look like this:

    it('sends a message to the server', function(done) {
        var requests = []

        // enable network in cri

        // listen to outgoing requests
        cri.Network.requestWillBeSent(function(params) {

        // get the send button's position
        getElementPosition(cri, 'a#send', function(err, pos) {
            if (err) return done(err)

            // simulate the click
            click(cri, pos.x + 1, pos.y + 1, function(err, result) {
                if (err) return done(err)

                // verify that the right request was sent from the right source
                setTimeout(function () {

                    var requestParams = requests[0]

                    var indexJs = 'file://' + path.join(__dirname, 'nwapp', 'index.js') 
                    var clickHandler = _.find(requestParams.initiator.stackTrace, { url: indexJs })


                }, 1000)

For this code to work we need some more dependencies too:

var path = require('path')  
var should = require('should')  

Initially, we tell cri to listen to network events, and then we hook to the requestWillBeSent event. Every request that is sent from the nw.js client will have its data pushed to the requests array.

Then, after the click, I wait for 1 second and I check that:

  1. only one request was issued
  2. the request was issued to the right url
  3. the request was issued from the right code section (line 24 in nwapp/index.js)

Thats it, we finally have a full cycle test case for the send button. If you don't believe me try to change the coordinates of the click or change the url of the ajax request and see how the test fails.

Hope you enjoyed this post.

Full source is available here


UPDATE: I got some feedback about sweetening the code that does the click or about alternatives that make it easier and shorter. In my mind, the purpose of this post was to examine the low level details on how to use chrome-remote-interface with nw.js. I did add a comment regarding alternatives in the body of the post to make it clearer though

Comments powered by Disqus