Setting up Selenium tests with Webdriver.io

Sometimes you need to check that your web application is working well. How can we test it? Unit testing is fine, it doesn't test your entire code as-is, every module is tested isolated from others. Sometimes you need some tool, that runs a real browser, simulates some actions like a real user does. Selenium can be this tool. It works with almost all browsers, even with mobile. But so often people underestimate Selenium because they treat it as a very difficult tool to use and looking for workarounds. But it doesn't if you know what to do, it seems very easy to use, and in this article, I am going to tell you how to setup tests against real browsers in your project.

Meet Selenium

Selenium is a very old tool. The project started in 2004 and went a long way and now it is the open platform, with well-documented public API and bindings for a lot of browsers. The core of Selenium is written in Java. But don't worry, you can use it with any language as a simple standalone program that runs a web server with some HTTP-methods.

You can install it from the official website, also, there is a neat wrapper to get this from npm. So for you node.js project, you can just run

npm install selenium-standalone — save-dev

Package ships an executable for you, that runs Selenium. But package itself contains the only downloader code, you need to fetch actual Selenium distro.

./node_modules/.bin/selenium-standalone install

ProTip: add this command as “prebublish” in your package.json, to download selenium automatically after npm install

Then we can run Selenium server.

./node_modules/.bin/selenium-standalone start

We can work with Selenium like with any HTTP-server with some REST-API. This API named WebDriver and contains methods to interact with browsers. If you need to run some browser on different operation system (Internet Explorer, for example) you can run Selenium on remote computer, connect it from yours and run tests as well it was on your local machine. Because it is just a web-server, it can be scaled to a lot of copies in some cloud and make Selenium Grid, where you can run a lot of tests on different hosts in parallel. There some companies already provide you Selenium-as-service, if you don’t want setup and maintain your own cluster, you can use Sass solution.

But anyway it would be good to start from tests on your local machine and browsers because it significantly reduces overhead and allow you to step over some issues and get your setup much faster.

Webdriver.io

As I said, Selenium is a just server, that handles your HTTP-requests. But you don't need to call it directly. There are already a lot of libraries to do it in a nice way. Webdriver.io is one of the popular WebDriver clients in Javascript. Recently the 4th version was released, so now it is a good time to check this out.

npm install webdriverio -- save-dev

Webdirver.io can be used as any other library:

var webdriverio = require('webdriverio');
var browser = webdriverio
  // setup your selenium server address. 
  // If you are using default settings, leave it empty
  .remote({ host: 'localhost', port: 4444 })
  // run browser that we want to test
  .init({ browserName: 'chrome', version: '45' });

Then we will get Chrome browser and can do something with it. We can open urls, do mouse clicks, look for elements on a page. Full list of available commands you can find in documentation.

But Webdriverio can do much more that that. It has special test runner that can do a lot of work for you to make your testing easier. You can find `wdio` command that is shipped with webdriverio package.

First of all, you need to run `wdio config` to went through configuration dialog and tell where you selenium server is, which browsers you want to run and other information. All possible options are listed in that section of Webdriverio docs. The result will be saved in the `wdio.conf.js` file. It is plain javascript file, that should return a configuration object. If you worked with Karma or Webpack, you will find this configuration style very similar. Here we can setup any options regarding our tests, browsers and setup some lifecycle hooks, for example, do something before each test starts.

Then you can write test code. The thing is there you will already have available `browser` object that you can use to work with a browser.

describe('webdriver.io tests', function() {
    it('is a test', function() {
        browser.click('.logo');
    });
    it('is a second test', function() {
        browser.click('.link');
    });
});

Look, now we don’t need to setup Selenium and get browser, all that things is now hidden in the configuration file, our tests very simple and straightforward.
But our setup is not finished yet. We need to run our selenium server for our tests. Webdriver.io provides an integration to run local selenium instance. You just need to install wdio-selenium-standalone-service module, then add this into services section of your wdio.conf.js file:

export.config = {
  // ...
  services: ['selenium-standalone'],
  // ...
};

Great! Now our server magically starts up and shuts down around our test run. We can write tests and not care about other stuff around. The only thing that still isn't solved is how to automatically start up your application to test, but it solely depends on your environment and I beleive that you can set this up.

Going deeper

You may write more and more tests and eventually encounter with errors in your tests. Not all of them can be easy explained. Sometimes you have to do a lot of debugging to find out the reason. In simple unit tests, we have a lot of helpers to debug. We can put a breakpoint and go through our application code and watch actual variable values there. But with Selenium, we can't afford this. Our code is run so far away and we can't attach a debugger there. We may only watch to the page like a simple user and look for some errors. What does Webdriverio have to help us? The most popular way to debug Selenium tests is screenshoting. When you can see to your test page as it looks for test code, you will probably see the reason for the error. Webdriverio has a handy method for it.

browser.saveScreenshot('my-page.png');

You can save screenshots during tests and figure out what went wrong with them. But finally you will end up with a folder full of pictures and you will not have any context about any screenshot. You can get some hints from the filename, but you can't add a lot of data there. We need to gather much more data from our test when we making a screenshot or other dumping information.

Allure-framework

Allure gives you very rich html report

Allure is a framework for better test reporting. It gathers a lot of information and dumps it on disk and after tests, you will get an HTML file where you can see a report. It looks much better than raw console reports. Allure compatible with a lot of testing frameworks, there also is plugin for Webdriverio, that you can install from npm:

npm install wdio-allure-reporter

Then you can add `allure` into `reporters` section in config to enable it:

reporters: ['dot', 'allure']

It would be better to keep default `dot` reporter to see actual progress while tests are running. After tests have been run, you will get some files in `allure-results` directory (by default, but you can change it). It is raw results, if you want to look at it, you need to do some processing. Install `allure-commandline` that will do it for you. You can install it from Homebrew, PPA, or use it as TeamCity/Bamboo/Jenkins plugin, but in this guide we are working with Node.js, so you probably want to get it from NPM. And you can:

npm install allure-commandline

To generate the report run the following:

./node_modules/.bin/allure generate allure-results

`allure-results` is a directory where you test results are. The final report will be in `allure-report` directory. It is some html files, that you can open using `allure open` command.

Why do you have to do a lot of commands? Why not just write the report after tests? Allure uses two-step model to make the second step optional. Report generation can take some time, but sometimes you don't need to see the report, for example, if all your tests were passed. Then your bill will be finished faster.

Finally, let's look at the report. All information from your tests now is structured to help you find a reason of fail much faster. There are several ways to review your results:

• Defects — here you will get a list of different exception messages that were thrown in your tests. If some test had failed with the same error, it will be grouped. Also, a little deviation is possible, some similar messages will be merged into one. Sometimes similar error has the same reason and can be fixed in one place and this view intended to help you with it.
• xUnit — it is the classical view where you can see your test in the `Suite -> Case` model. Here you can look your tests closely to how it looks in the source code and easily match failed tests with their actual code.
• Graph — some statistics about your tests. Everybody likes charts, here you can find some.

Allure have some more different views that are not perfectly integrated with Webdriverio for now, but hopefully, it will be done in future. There are:

• Behaviors – group tests by user stories. There you can group tests for different criteria rather than xUnit. If some user story is covered by test cases from different test suites, here you can see them all on one list.
• Timeline – if you can run your tests in parallel (Webdriverio launcher can't do it, at least for now), it is good to know how your tests were distributed by time. For example, if your test application had some blackout, you will see a group of failing tests here. This view is a good thing to detect this issues.

Let's try this. Then you will never want to investigate test failures in the console.

Conclusion

Selenium is the most common tool to run tests on real browsers. If you want to add some integration tests to your web application, you probably should consider it. You can use javascript if it is your main programming language. There are a lot of tools that you help with it. Webdriverio is a javascript library that works perfectly with Selenium. After a little setup, you will have some integration tests that can be easily integrated with your current testing infrastructure. Also, all code from this article is available on Github, where you can find a working project that you can clone and play around.

Good luck with testing! This thing will give more confidence that your code work as expected and you would automate that checks and will not do it manually