products

Cypress tips and tricks

A few tips on getting the most out of E2E testing tool Cypress

We have been successfully using Cypress to verify our websites and web apps for a while. Here are some tips & tricks we have learned.

Even this blog is tested using Cypress :)

Read the docs

Cypress has excellent documentation at https://docs.cypress.io/, including examples, guides, videos and even a busy Gitter chat channel. There is also "Search" box that quickly finds relevant documents. The search is really good and covers the docs, the blog posts and the examples.

Recently common recipes were combined into a single place. If you still have an issue with Cypress, please search through open and closed issues.

You can also read my other blog posts about Cypress under tags/cypress.

Run Cypress on your own CI

As long as the project has a project key, you can run the tests on your own CI boxes. I found running inside Docker to be the easiest, and even built an unofficial Docker image with all dependencies.

Note that cypress ci ... will upload recorded videos and screenshots (if capturing them is enabled) to the Cypress cloud, while cypress run ... will not.

Record success and failure videos

When Cypress team has released the video capture feature in version 0.17.11 it quickly became my favorite. I have configured our GitLab CI to keep artifacts from failed test runs for 1 week, while keeping videos from successful test runs only for a 3 days.

1
2
3
4
5
6
7
8
9
10
11
12
13
artifacts:
  when: on_failure
  expire_in: '1 week'
  untracked: true
  paths:
    - cypress/videos
    - cypress/screenshots
artifacts:
  when: on_success
  expire_in: '3 days'
  untracked: true
  paths:
    - cypress/screenshots

Whenever a test fails, I watch the failure video side by side with the video from the last successful test run. The differences in the subject under test are quickly obvious.

Move common code into utility package

Testing code should be engineered as well as the production code. To avoid duplicating testing helper utilities, we advise to move all common helpers into their own NPM module. For example, we use local CSS styles with randomized class names in our web code.

1
2
3
<div class="table-1b4a2 people-5ee98">
  ...
</div>

This makes the selectors very long since we need to use prefixes.

1
cy.get('[class*=people-]') // UGLY!

By creating a tiny helper function we alleviated the selector head aches.

1
2
3
import {classPrefix} from '@team/cypress-utils/css-names'
// classPrefix('foo') returns "[class*=foo-]"
cy.get(classPrefix('people'))

The test code became a lot more readable.

Separate tests into bundles

Using a single cypress/integration/spec.js file to test a large web application quickly becomes difficult and time consuming. We have separated our code into multiple files. Additionally, we keep the source files in src folder and build the multiple bundles in cypress/integration using kensho/multi-cypress tool.

The multi-cypress tool assumes that Docker and GitLab CI are used to actually run the tests. Each test will be inside its own "test job", thus 10 spec files can be executed at once (assuming at least 10 GitLab CI runners are available).

The command to run a specific spec file is cypress run --spec "spec/filename.js" by the way.

Make JavaScript crashes useful

As I have said before, the true value of a test is not when it passes, but when it fails. Additionally, even a passing test might generate client-side errors that are not crashes and do not fail the test. For example, we often use Sentry exception monitoring service where we forward all client-side errors.

If a client-side error happens while the E2E Cypress test is running, we need this additional context: which test is executing, what steps were already finished before the error was reported, etc.

By loading a small library send-test-info before each test, we can capture the name of the test, its spec filename and even the Cypress test log as breadcrumbs; this information is then sent to Sentry with each crash. For example, the exception below shows the name of the test when the error was reported

Test name

Similarly, the Cypress test steps are logged as breadcrumbs and are sent with the exception allowing any developer to quickly get a sense when the error happens as the test runs.

Test steps

Use test names when creating data

Often as part of the E2E test we create items / users / posts. I like to put the full test title in the created data to easily see what test created which datum. Inside the test we can quickly grab the full title (which includes the full parent's title and the current test name) from the context.

I also like creating random numbers for additional difference

1
2
3
4
5
6
7
8
9
10
11
12
const uuid = () => Cypress._.random(0, 1e6)
describe('foo', () => {
  // note that we need to use "function" and not fat arrow
  // to actually get the right "this" context
  it('bar', function () {
    const id = uuid()
    const testName = this.test.fullTitle()
    const name = `test name: ${testName} - ${id}`
    // name will be "test name: foo bar - <random>"
    makeItem({name}) // use the test name to create unique item
  })
})

Explore the environment

You can pause the test execution by using debugger keyword. Make sure the DevTools are open!

1
2
3
4
it('bar', function () {
  debugger
  // explore "this" context
})

Run all spec files locally

If you separate Cypress E2E tests into multiple spec files, you soon end up with lots of files. If you need to run the scripts one by one from the command line, I wrote a small utility run-each-cypress-spec.

1
2
3
4
5
6
7
8
npm install -g run-each-cypress-spec
run-specs
...
running cypress/integration/a-spec.js
Started video recording: /Users/gleb/cypress/videos/p259n.mp4
...
running cypress/integration/b-spec.js
Started video recording: /Users/gleb/cypress/videos/62er1.mp4

If you need environment variables (like urls, passwords), you can easily inject them using as-a

1
as-a cy run-specs

Get command log on failure

While videos of the failed tests are useful, sometimes we just want to get a sense of the failure before triaging it. In the headless CI mode, you can get a JSON file for each failed test with the log of all commands. See the JSON file below as an example of a failed test and the file it generated.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
{
  "testName": "Website loads the About tab",
  "testError": "Timed out retrying: Expected to find content: 'Join Us' but never did.",
  "testCommands": [
    "visit",
    "new url https://www.company.com/#/",
    "contains a.nav-link, About",
    "click",
    "new url https://www.company.com/#/about",
    "hash",
    "assert expected **#/about** to equal **#/about**",
    "contains Join Us",
    "assert expected **body :not(script):contains(**'Join Us'**),
      [type='submit'][value~='Join Us']** to exist in the DOM"
  ]
}

This is a separate project that just needs to be included from your cypress/support/index.js file. For instructions see cypress-failed-log project.

Wait on the right thing

It is important to wait for the right element. Imagine we have a list of items in the DOM. We expect a new element to appear with text "Hello". We could select first the list, then the element containing "Hello" and check if it becomes visible

1
2
3
cy.get('.list')
  .contains('li', 'Hello')
  .should('be.visible')

But sometimes it does not work. When we cy.get('.list') Cypress saves the DOM element as the "subject" and then tries to wait for a child of that element with text "Hello" to become visible. If we use React for example, the DOM of the list might be recreated from scratch if we push a new item into the list. When that happens Cypress notices that the "subject" DOM it holds a reference to becomes "detached from the DOM" - it becomes an orphan!

A better solution to this problem is to use a composite CSS selector that will grab the list AND the item in a single operation.

1
2
cy.contains('.list li', 'Hello')
  .should('be.visible')

This forces Cypress to wait for the list AND list item element without caching a reference to the DOM element .list.

You can use advanced CSS selectors to get an element in a single command. For example instead of cy.get('.something').first() you could use cy.get('.something:first-child'). Similarly,

1
2
3
4
5
6
7
8
9
// get first element
cy.get('.something').first()
cy.get('.something:first-child')
// get last element
cy.get('.something').last()
cy.get('.something:last-child')
// get second element
cy.get('.something').eq(1)
cy.get('.something:nth-child(2)')

Write and read files

You can write files to disk directly from Cypress using cy.writeFile and read an existing file using cy.readFile. What if you want to read a file that might not exist? cy.readFile will fail the test if file does not exist, thus we need to find a work around.

In my case, the file I would like to load is a JSON of test values useful for Jest-like snapshot testing.

Here is a neat trick. Save the file using cy.writeFile, for example cy.writeFile('spanshots.json') whenever there is something to save. When reading the file, fetch it through the Cypress code proxy using fetch Notice how the test spec itself is served by Cypress from URL which looks like http://localhost:49829/__cypress/tests?p=cypress/integration/spec.js-438. Let us "hack" it to load a file that might not exist.

The code below tries to load snapshots file before each test. If the fetch call fails, the file does not exist.

1
2
3
4
5
6
7
8
9
10
11
12
let snapshots
beforeEach(function loadSnapshots() {
  return fetch('/__cypress/tests?p=./snapshots.json')
    .then(r => r.text())
    .then(function loadedText(text) {
      // ?
    })
    .catch(err => {
      console.error('snapshots file does not exist')
      snapshots = {}
    })
})

If the fetch call succeeds, the returned text will NOT be original JSON! Instead it will be webpacked module :)

For example, here is the saved file

snapshots.json
1
2
3
4
5
6
7
{
  "spec.js": {
    "works": [
      "foo"
    ]
  }
}

Here is how built-in Cypress bundler returns it in response to /__cypress/tests?p=./snapshots.json - I have shortened the webpack boilerplate preamble.

1
2
3
4
5
6
7
8
9
(function e(t,n,r){function ...)({1:[function(require,module,exports){
module.exports={
  "spec.js": {
    "works": [
      "foo"
    ]
  }
}
},{}]},{},[1]);

Notice that our JSON has been placed into module with id '1' (first line expression {1:). The entire bundle registers a stand alone require function. If we want to get into the actual "module" contents we have to do the following in our function loadedText(text) to finish the hack

1
2
3
4
5
6
7
8
function loadedText (text) {
  if (text.includes('BUNDLE_ERROR')) {
    // if the bundler tried to handle missing file
    return Promise.reject(new Error('not found'))
  }
  const req = eval(text)
  snapshots = req('1')
}

Boom, the object snapshots has been loaded from the file snapshots.json which might or might not exist.

Note while I have successfully used the above hack when running Cypress locally, it was always failing when doing cypress run in headless mode.

Conditional logic

Sometimes you might need to interact with a page element that does not always exist. For example there might a modal dialog the first time you use the website. You want to close the modal dialog.

1
2
3
4
5
cy.get('.greeting')
  .contains('Close')
  .click()
cy.get('.greeting')
  .should('not.be.visible')

But the modal is not shown the second time around and the above code will fail. In order to check if an element exists without asserting it, use the proxied jQuery function Cypress.$

1
2
3
4
5
6
7
8
9
const $el = Cypress.$('.greeting')
if ($el.length) {
  cy.log('Closing greeting')
  cy.get('.greeting')
    .contains('Close')
    .click()
}
cy.get('.greeting')
  .should('not.be.visible')

The above check is safe.

Customize Cypress test runner colors

Read Cypress Halloween Theme and check out cypress-dark plugin.

Shorten assertions

For cy.location you can pass the part you are interested in

1
2
3
cy.location().should(location => {
  expect(location.pathname).to.eq('/todos');
});

is the same as

1
cy.location('pathname').should('equal', '/todos')

You can use its to get nested properties easier

1
2
3
cy.get('@login').should(req => {
  expect(localStorage.getItem('token')).to.be.eq(req.response.body.accessToken);
});

is the same as

1
2
cy.get('@login').its('response.body.accessToken')
  .should(accessToken => expect(accessToken).to.equal(localStorage.getItem('token')))

and this could be expressed more clearly in steps

1
2
3
4
5
6
cy.then(() => {
  // by this step, there should the token stored in local storage
  const token = localStorage.getItem('token')
  // and it should come from the response
  cy.get('@login').its('response.body.accessToken').should('equal', token)
})

You can wait for network request to happen and then check something using cy.then

1
2
3
4
5
// notice that we are waiting for @login XHR alias
// before checking that the token is NOT set
cy.get('@login').should(req => {
  expect(localStorage.getItem('token')).to.be.null;
});

Instead we can use .then after waiting for the network request. It is also helpful to print the message when checking for the token, otherwise command log simply says "expect null to equal null".

1
2
3
4
cy.wait('@login')
  .then(() => {
    expect(localStorage.getItem('token'), 'token in local storage').to.be.null;
  });

You don't need to wrap elements just to dynamically use them. For example if you want to test each item from the list

1
2
3
4
5
6
7
8
9
10
11
const inputs = [
  { element: 'username', text: 'foo', error: 'Please enter your username.' },
  { element: 'password', text: 'foo', error: 'Please enter your password.' }
];
cy.wrap(inputs).each(input => {
  const { element, text, error } = input;

  cy.getByTestId(element)
    .type(text)
    .should('have.value', text);
})

Every Cypress command is automatically inserted into the queue, so you can iterate over the items and use Cypress commands, everything will be queued correctly.

1
2
3
4
5
6
7
inputs.forEach(input => {
  const { element, text, error } = input;

  cy.getByTestId(element)
    .type(text)
    .should('have.value', text);
})

I prefer cy.first() to cy.eq(0).

Disable ServiceWorker

ServiceWorkers are great - but they can really affect your end-to-end tests by introducing caching and coupling tests. If you want to disable the service worker caching you need to remove or delete navigator.serviceWorker when visiting the page with cy.visit.

First, here is the way that does not work - simply deleting the property from the navigator object.

1
2
3
4
5
6
7
8
it('disables it', function () {
  cy.visit('index.html', {
    onBeforeLoad (win) {
      delete win.navigator.serviceWorker
      // nope, win.navigator.serviceWorker is still there
    }
  })
})

The reason for this is that we are deleting the property from the wrong object. The serviceWorker is NOT defined on the navigator, it is defined on its prototype. You can confirm this by using Object.getOwnPropertyDescriptor method:

1
2
3
4
Object.getOwnPropertyDescriptor(win.navigator, 'serviceWorker')
// undefined
Object.getOwnPropertyDescriptor(win.navigator.__proto__, 'serviceWorker')
// {set: undefined, enumerable: true, configurable: true, get: ƒ}

After we delete the property from the right object, the application code will no longer think it can register the service worker.

cypress/integration/spec.js
1
2
3
4
5
6
7
it('disables it', function () {
  cy.visit('index.html', {
    onBeforeLoad (win) {
      delete win.navigator.__proto__.serviceWorker
    }
  })
})
index.html
1
2
3
4
5
6
7
8
9
<body>
  <script>
    if ('serviceWorker' in navigator) {
      console.log('registering service worker')
    } else {
      console.log('skipping sw registration')
    }
  </script>
</body>

Service worker is no longer registered

Note: once deleted, the SW stays deleted in the window, even if the application navigates to another URL.

Read also "Stub navigator API in end-to-end tests" for another example.

Control navigator.language

Imagine you have a page that shows different greeting depending on the navigator.language property.

1
2
3
4
5
6
7
8
9
10
11
12
<body>
  <div id="greeting"></div>
  <script>
    const greeting = document.getElementById('greeting')
    if (navigator.language === 'Klingon') {
      // https://www.omniglot.com/language/phrases/klingon.php
      greeting.innerText = 'nuqneH'
    } else {
      greeting.innerText = 'Hi there'
    }
  </script>
</body>

How do we check if the default English greeting is displayed? Easily

1
2
3
4
it('shows default greeting', () => {
  cy.visit('index.html')
  cy.contains('#greeting', 'Hi there').should('be.visible')
})

But how do we force the Klingon greeting to be displayed? Trying to change read-only navigator.language property throws an error.

1
2
3
4
5
6
7
8
9
it('shows Klingon greeting', () => {
  cy.visit('index.html', {
    onBeforeLoad (win) {
      // DOES NOT WORK
      win.navigator.language = 'Klingon'
    }
  })
  cy.contains('#greeting', 'nuqneH').should('be.visible')
})

Error is thrown when trying to directly set `navigator.language`

The navigator.language property is actually set on navigator.__proto__ object:

1
2
3
4
Object.getOwnPropertyDescriptor(navigator, 'language')
//> undefined
Object.getOwnPropertyDescriptor(navigator.__proto__, 'language')
//> {get: ƒ, set: undefined, enumerable: true, configurable: true}

We can create language property on the navigator object instead - thanks, prototypical inheritance! We can specify the property's value:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
it('shows Klingon greeting', () => {
  cy.visit('index.html', {
    onBeforeLoad (win) {
      // DOES NOT WORK
      // Uncaught TypeError: Cannot assign to read only property
      // 'language' of object '[object Navigator]'
      // win.navigator.language = 'Klingon'

      // instead we need to define a property like this
      Object.defineProperty(win.navigator, 'language', {
        value: 'Klingon'
      })
    }
  })
  cy.contains('#greeting', 'nuqneH').should('be.visible')
})

This test shows proper respect to each Klingon warrior browsing the web.

Klingon greeting

How do we ensure that the application actually read navigator.language property when displaying the greeting? Maybe the "nuqneH" is hard-coded! We need to track navigator.language "get" access. We can do this using cy.stub method.

1
2
3
4
5
6
7
8
9
10
11
it('checks if application gets language property', () => {
  cy.visit('index.html', {
    onBeforeLoad (win) {
      Object.defineProperty(win.navigator, 'language', {
        get: cy.stub().returns('Klingon').as('language')
      })
    }
  })
  cy.contains('#greeting', 'nuqneH').should('be.visible')
  cy.get('@language').should('have.been.calledOnce')
})

Now we know for sure how the application behaves.

Language was read by the application

Use fixtures to stub network requests

If your tests are full of stubbed network responses, move the responses into fixtures

before

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
it('does A', () => {
  cy.server()
  cy.route({
    method: 'POST',
    url: '**/api/v1/suggestion_query/query',
    response: {
      ... loooong response object ...
    }
  })
  // UI actions
})

it('does B', () => {
  cy.server()
  cy.route({
    method: 'POST',
    url: '**/api/v1/suggestion_query/query',
    response: {
      ... another loooong response object ...
    }
  })
  // UI actions
})

after 1

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
it('does A', () => {
  cy.server()
  cy.route({
    method: 'POST',
    url: '**/api/v1/suggestion_query/query',
    // loads JSON fixture from cypress/fixtures/first_query_response.json
    response: 'fixture:first_query_response'
  })
  // UI actions
})

it('does B', () => {
  cy.server()
  cy.route({
    method: 'POST',
    url: '**/api/v1/suggestion_query/query',
    // loads JSON fixture from cypress/fixtures/second_query_response.json
    response: 'fixture:second_query_response'
  })
  // UI actions
})

after 2

You can even require the JSON fixtures directly if the second response is just a little bit different from the first response

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
const responseFixture = require('../fixtures/first_query_response')
it('does A', () => {
  cy.server()
  cy.route({
    method: 'POST',
    url: '**/api/v1/suggestion_query/query',
    response: responseFixture
  })
  // UI actions
})

it('does B', () => {
  // Lodash is bundled with Cypress https://on.cypress.io/_
  // so use it to avoid changing the shared response object
  const secondResponse = Cypress._.cloneDeep(responseFixture)
  secondResponse.someProperty = 'new value'
  cy.server()
  cy.route({
    method: 'POST',
    url: '**/api/v1/suggestion_query/query',
    response: secondResponse
  })
  // UI actions
})

See cy.route and cy.fixture documentation.

Bonus: read Import Cypress fixtures

Use code coverage

You can instrument your application and use Cypress code coverage plugin to produce combined reports in multiple formats. End-to-end tests are very effective at covering a lot of code in a single test.

Use visual testing

If a test grows to be very long because it checks so many elements on the page, see if it makes sense to test the entire page using Visual Testing.

before

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
it('checks lots of things', () => {
  // perform some actions on the page
  // check results after action
  cy.get('selector1').should('be.visible')
    .find('sub-selector1').should('have.text', 'expected text')
  cy.get('selector2').should('be.visible')
    .find('sub-selector2').should('have.text', 'expected text')
  cy.get('selector3').should('be.visible')
    .find('sub-selector3').should('have.text', 'expected text')
  cy.get('selector4').should('be.visible')
    .find('sub-selector4').should('have.text', 'expected text')
  cy.get('selector5').should('be.visible')
    .find('sub-selector6').should('have.text', 'expected text')
  ...
})

after

1
2
3
4
5
6
7
8
9
10
11
12
13
it('checks lots of things', () => {
  // perform some actions on the page
  // check results after action
  // first, confirm the page has been updated after action
  cy.get('selector1').should('be.visible')
    .find('sub-selector1').should('have.text', 'expected text')

  // but instead of the checking individual elements
  // compare the entire page against expected "gold" image
  // using one of the visual testing plugins
  // https://on.cypress.io/plugins#visual-testing
  cy.takeVisualSnapshot('user action')
})

Bonus: check out bahmutov/sudoku for visual component testing using open source tools, and read Visual testing for React components using open source tools.

Interactive and headed mode

You can find out if the test is currently running using the interactive mode which is when the user calls cypress open.

1
2
3
4
5
if (Cypress.config('isInteractive')) {
  // interactive "cypress open" mode!
} else {
  // "cypress run" mode
}

When you use the interactive mode you always see the browser, thus the browser is always headed. But when using the cypress run command, the browser might be headless or headed. You can find out how the browser is displayed:

1
2
3
4
5
6
7
if (Cypress.browser.isHeaded) {
  // browser is headed
  // might be interactive mode "cypress open"
  // or "cypress run --headed"
} else {
  // browser is running headlessly
}

When using cypress run, Electron is headless by default, while Chrome and Firefox browsers are headed by default. You can control the browser though:

1
npx cypress run --browser chrome --headless

Trying to pass both --headed and --headless CLI parameters raises an error.

Produce high quality video recording

You can find most of the advice below implemented in bahmutov/cypress-movie. To generate better videos from tests

  • set video compression to false
  • set browser window size to larger value, which should work fine when using headless Chrome on CI
1
2
3
4
5
6
7
8
9
// in your plugins file
on('before:browser:launch', (browser = {}, launchOptions) => {
  if (['chrome', 'chromium'].includes(browser.name) && browser.isHeadless) {
    launchOptions.args.push(
      `--window-size=1920,1080`,
    )
  }
  return launchOptions
})
  • hide command log

You can "expand" the application under test iframe to cover the entire window

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
const clearViewport = () => {
  const runnerContainer = window.parent.document.getElementsByClassName(
    'iframes-container',
  )[0]
  runnerContainer.setAttribute(
    'style',
    'left: 0; top: 0; width: 100%; height: 100%;',
  )

  const sizeContainer = window.parent.document.getElementsByClassName(
    'size-container',
  )[0]
  sizeContainer.setAttribute('style', '')

  const sidebar = window.parent.document.getElementsByClassName(
    'reporter-wrap',
  )[0]
  sidebar.setAttribute('style', 'opacity: 0')

  const header = window.parent.document.querySelector(
    '.runner.container header',
  )
  header.setAttribute('style', 'opacity: 0')
}

Call this function before the tests.

  • use native Chrome Debugger Protocol to take full page screenshots. See "cypress-movie" project.

See also