Web testing nirvana with Cypress

I have used a lot of tools for unit testing JavaScript, but could never find a good tool for running functional tests on my web pages. I have used PhantomJS / CasperJS / Selenium / Selenium with Protractor, yet always the tests were hard to setup, inconvenient to run, slow, flaky, and to be honest useless. A good test is only useful when it fails and allows to find the problem quickly. Functional tests often fail, yet it is hard to pin point why they failed, even if there is a legitimate software bug causing the exception.

Recently, I have watched a presentation by Brian Mann Testing, the way it should be that shows a new tool, currently in private beta for open source projects. The presentation shows how functional web testing could work if we designed a testing tool with the developer in mind. The tool is called Cypress; and after trying it on several of my own open source projects I must say: I have experienced the testing nirvana and so can you. The tool is almost magical in its simplicity and power.

• Painless installation with npm i -g cypress-cli && cypress install. Then start the tool and login using your GitHub account (the tool is in private beta).
• Picking a new project via tool's GUI creates an example test with lots of possible tests. I delete this file to avoid running it, but you can always find the Cypress demo tests in example_spec.js file from the Kitchen sink example app
• The test runner is framework-agnostic. I have used Cypress to test server-side rendering project and a vanilla JS library. The tests themselves use Mocha (my favorite!) and BDD assertions.
• The runner understands promises and automatically waits N (10 by default) seconds when checking element's presence / url / etc. This is huge, since it removes a lot of flaky boiler plate logic from the tests that deal with unpredictable timing in the real world.
• The tool integrates with the major CI services and allows one to run the functional tests on the server. Just grab the Cypress token from the command line using cypress get:key and set it as an environment variable CYPRESS_CI_KEY on your CI server. Here is an example Travis CI command to run the functional tests

before_script:
  - npm install -g cypress-cli
script:
  - npm test
  - cypress ci

I have a couple of projects using Cypress already, here are a few things learnt

todomvc-express

I have a simple server-side rendered application showing Todos todomvc-express. The functional tests require starting the server first. On Travis CI I start the server in the background and then run the functional tests.

script:
  - npm run lint
  - npm test
  - npm start &
  - cypress ci

The server running in the background is closed automatically by Travis once the tests finish. The functional tests themselves can be found in try-cy-spec.js and follow BDD format (Cypress uses Mocha). Assertions are made using Chai should syntax. Here is a typical test checking that a new TODO item has been added to the list

describe('todomvc app', function () {
  beforeEach(function () {
    cy.visit('http://localhost:3000')
  })
  it('can insert new todo', function () {
    cy.get('.new-todo')
      .type('new todo{enter}')
      .get('ul.todo-list')
        .find('li').should('not.be.empty')
  })
})

Running the tests provides a huge amount of additional information - each step is displayed in the list on the left, each step has before and after DOM snapshots that I can inspect, and each object (like the NodeList selected using .find('li')) expression) can be inspected in the browser's console by clicking on it.

Interacting with the DOM around a failure is a pleasure and makes debugging failed tests a snap. For example if we expected the TODO DOM list to be empty after inserting a new item, the failure would have all the relevant information

The tests can be opened in the multiple browsers by pointing at the same top URL (like http://localhost:2020/__/#/tests/integration/try-cy-spec.js in this case) and the tool reruns all tests in all browsers on source file changes.

tiny-toast

My second example is a small JavaScript only library tiny-toast that displayes a simple text alert. I wanted to see if we can test a library that does NOT need a local server to run. Turns out Cypress can handle nicely opening a static file, even if it includes JavaScript resources that point to the parent folder. Here is my test page cypress/integration/page.html

<head>
  <title>Tiny toast test page</title>
  <script src="../../dist/tiny-toast.js"></script>
</head>

Note that it includes tiny-toast.js from the repo's root dist folder. Yet Cypress loads everything without a trouble.

it('opens simple test page', () => {
  cy.visit('cypress/integration/page.html')
})

When testing JavaScript library that attaches itself to the window object, we need to grab the window object itself first. Since the test page runs inside an iframe isolated from the parent page, we need to use the provided Cypress API method. Here is how I check the object's API

it('has tinyToast api', () => {
  cy
    .window().should('have.property', 'tinyToast')
    .window().then((w) => w.tinyToast)
    .then((tinyToast) => {
      expect(tinyToast).to.be.a('object')
      expect(tinyToast.show).to.be.a('function')
      expect(tinyToast.hide).to.be.a('function')
    })
})

To avoid boilerplate code, we can use Underscore property access (Underscore is bundled into Cypress tools together with jQuery, momentjs, and a few other popular utilities).

it('gets tinyToast using _', () => {
  cy
    .window().then(Cypress._.property('tinyToast'))
    .then((tinyToast) => {
      ...
    })
})

An interesting test that uses default DOM waiting in Cypress shows the text message, then closes it after 2 seconds. With Cypress we don't have to setup the browser delay at all - it will wait up to 10 seconds for the element to become invisible before failing. Thus our tests is simple

it('can show and hide message with fluent api', () => {
  cy
    .window().then((w) => w.tinyToast)
    .then((tinyToast) => {
      tinyToast.show('test message').hide(2000)
    })
    .contains('h3', 'test message').should('be.visible')
    .get('h3').should('not.exist') // will automatically wait up to 10 seconds
})

In the above code the command .contains('h3', 'test messaage') finds H3 tag with the given text contents, which should become visible after we execute tinyToast.show(...). Then the element should be removed from DOM after 2 seconds controlled by .hide(2000).

The Cypress test page again provides extra useful information for each test statement.

Hovering over each assertion brings the before / after DOM snapshots with the changed elements highlighted. Additional information is automatically displayed for .get() commands, like the number of found elements (zero in this case).

Working with Cypress has been a great pleasure. It shows that there is plenty of room for improvement in functional testing for web applications, one just needs to have a strong desire to identify and solve developer's pain points.