Testing in a real browser with Sauce Labs with Travis CI

I recently found myself implementing a basic set of javascript integration tests on an open source project. I quickly came to the realization that there is a serious lack of good documentation on how to get a basic browser test running on a Travis CI using Sauce Labs and Sauce Connect.

After stumbling across the barren desert of outdated doc pages, incorrect stackoverflow answers, and ancient google group posting, I present to you the spoils of my quest to the underbelly of the web.

Let's approach this in the context of a real problem: Testing javascript in a real web browser.
(view the complete project here)

Setting up travis.ci

(view the full diff)
To get going we are going to setup http://travis.ci to run on our repo.

Setting up our repo to work with travis

Once travis is enabled we will want to setup a basic .travis.yml file to run a basic test on our repo.

We are telling it to use node_js and specifying the version of node. By default node projects will run npm install and npm test so everything should be good to go. And we will see our basic test run

Creating our first browser test.

To take advantage of Sauce Lab and Sauce Connect we are going to need to use selenium webdriver. In case you are not familiar with selenium, it allows you to remotely control a web browser through code (pretty awesome stuff).

The documentation for it is a bit hard to find but I will link it here so you do not have to go hunting http://seleniumhq.github.io/selenium/docs/api/javascript/module/selenium-webdriver/index_exports_WebElement.html

We will want to install selenium and setup out basic tests

Instead of going over the entire diff lets look at a few critical pieces of the code.

At a high level we are setting up selenium to launch a chrome instance, visiting the page we want to test, clicking on an element, and then asserting that the clicked element's text has changed to an expected value.

There is a bit of async complexity going on that mocha and selenium for the most part abstract for us, so no need to fret about that for now.

If you noticed we are visiting http://localhost:8000, that means that we need to boot up a server to serve our content. Travis makes this easy enough via a before_script task. In the task we will just start a python simple server and give it a few seconds to boot. The ampersand at the end of the python line tells travis to run the process in the background instead of blocking the execution thread, allowing us to run tasks at the same time.

Believe it or not that is all we need to run our tests in a real browser (or at least it seems to satisfy the requirements to run things locally).

And in Comes Sauce Labs

(view the full diff)
When we push our work from before, we are rudely awoken by the unfortunate fact that nothing is working on travis. https://travis-ci.org/samccone/travis-sauce-connect/builds/54796160

This is where we will lean on Sauce. Sauce Labs provides its services for free for Open Source projects. We can get our access keys via https://docs.saucelabs.com/ci-integrations/travis-ci/ and just add the encrypted keys to our travis.yml file, these keys will allow us to connect to Sauce's VM cluster to run our tests against.

From here we just need to enable the sauce_connect addon for travis, again only a minor change to the travis.yml file is needed.

The next step is to tweak our selenium browser build step to use sauce labs VMs instead of the local machines.

Our beforeEach step gets a tiny bit more complex. We first detect if there is a sauce environment variable and if so we setup the browser with the required parameters for it to connect to Sauce's infrastructure. After that setup is done, we can be blind to the changes that we made, since we will not have to worry about it again.

Once we push our changes we can see everything works like charm!

We even get an awesome playback video from sauce to help us debug if there were any problems Sauce Test Playback

Hopefully this helps to codify the path to adding integration tests on your javascript projects.
If you still have questions please reach out via a github issue or on twitter.
Sam Saccone @samccone