products

Cypress tips and tricks

A few tips on getting the most out of E2E testing tool Cypress

We have been successfully using Cypress to verify our websites and web apps for a while. Here are some tips & tricks we have learned.

Even this blog is tested using Cypress :)

Read the docs

Cypress has excellent documentation at https://docs.cypress.io/, including examples, guides, videos and even a busy Gitter chat channel. There is also "Search" box that quickly finds relevant documents. Recently common recipes were combined into a single place. If you still have an issue with Cypress, please search through open and closed issues.

You can also read my other blog posts about Cypress under tags/cypress.

Run Cypress on your own CI

As long as the project has a project key, you can run the tests on your own CI boxes. I found running inside Docker to be the easiest, and even built an unofficial Docker image with all dependencies.

Note that cypress ci ... will upload recorded videos and screenshots (if capturing them is enabled) to the Cypress cloud, while cypress run ... will not.

Record success and failure videos

When Cypress team has released the video capture feature in version 0.17.11 it quickly became my favorite. I have configured our GitLab CI to keep artifacts from failed test runs for 1 week, while keeping videos from successful test runs only for a 3 days.

1
2
3
4
5
6
7
8
9
10
11
12
13
artifacts:
  when: on_failure
  expire_in: '1 week'
  untracked: true
  paths:
    - cypress/videos
    - cypress/screenshots
artifacts:
  when: on_success
  expire_in: '3 days'
  untracked: true
  paths:
    - cypress/screenshots

Whenever a test fails, I watch the failure video side by side with the video from the last successful test run. The differences in the subject under test are quickly obvious.

Move common code into utility package

Testing code should be engineered as well as the production code. To avoid duplicating testing helper utilities, we advise to move all common helpers into their own NPM module. For example, we use local CSS styles with randomized class names in our web code.

1
2
3
<div class="table-1b4a2 people-5ee98">
  ...
</div>

This makes the selectors very long since we need to use prefixes.

1
cy.get('[class*=people-]') // UGLY!

By creating a tiny helper function we alleviated the selector head aches.

1
2
3
import {classPrefix} from '@team/cypress-utils/css-names'
// classPrefix('foo') returns "[class*=foo-]"
cy.get(classPrefix('people'))

The test code became a lot more readable.

Separate tests into bundles

Using a single cypress/integration/spec.js file to test a large web application quickly becomes difficult and time consuming. We have separated our code into multiple files. Additionally, we keep the source files in src folder and build the multiple bundles in cypress/integration using kensho/multi-cypress tool.

The multi-cypress tool assumes that Docker and GitLab CI are used to actually run the tests. Each test will be inside its own "test job", thus 10 spec files can be executed at once (assuming at least 10 GitLab CI runners are available).

The command to run a specific spec file is cypress run --spec "spec/filename.js" by the way.

Make JavaScript crashes useful

As I have said before, the true value of a test is not when it passes, but when it fails. Additionally, even a passing test might generate client-side errors that are not crashes and do not fail the test. For example, we often use Sentry exception monitoring service where we forward all client-side errors.

If a client-side error happens while the E2E Cypress test is running, we need this additional context: which test is executing, what steps were already finished before the error was reported, etc.

By loading a small library send-test-info before each test, we can capture the name of the test, its spec filename and even the Cypress test log as breadcrumbs; this information is then sent to Sentry with each crash. For example, the exception below shows the name of the test when the error was reported

Test name

Similarly, the Cypress test steps are logged as breadcrumbs and are sent with the exception allowing any developer to quickly get a sense when the error happens as the test runs.

Test steps

Use test names when creating data

Often as part of the E2E test we create items / users / posts. I like to put the full test title in the created data to easily see what test created which datum. Inside the test we can quickly grab the full title (which includes the full parent's title and the current test name) from the context.

I also like creating random numbers for additional difference

1
2
3
4
5
6
7
8
9
10
11
12
const uuid = () => Cypress._.random(0, 1e6)
describe('foo', () => {
  // note that we need to use "function" and not fat arrow
  // to actually get the right "this" context
  it('bar', function () {
    const id = uuid()
    const testName = this.test.fullTitle()
    const name = `test name: ${testName} - ${id}`
    // name will be "test name: foo bar - <random>"
    makeItem({name}) // use the test name to create unique item
  })
})

Explore the environment

You can pause the test execution by using debugger keyword. Make sure the DevTools are open!

1
2
3
4
it('bar', function () {
  debugger
  // explore "this" context
})

Run all spec files locally

If you separate Cypress E2E tests into multiple spec files, you soon end up with lots of files. If you need to run the scripts one by one from the command line, I wrote a small utility run-each-cypress-spec.

1
2
3
4
5
6
7
8
npm install -g run-each-cypress-spec
run-specs
...
running cypress/integration/a-spec.js
Started video recording: /Users/gleb/cypress/videos/p259n.mp4
...
running cypress/integration/b-spec.js
Started video recording: /Users/gleb/cypress/videos/62er1.mp4

If you need environment variables (like urls, passwords), you can easily inject them using as-a

1
as-a cy run-specs

Get command log on failure

While videos of the failed tests are useful, sometimes we just want to get a sense of the failure before triaging it. In the headless CI mode, you can get a JSON file for each failed test with the log of all commands. See the JSON file below as an example of a failed test and the file it generated.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
{
  "testName": "Website loads the About tab",
  "testError": "Timed out retrying: Expected to find content: 'Join Us' but never did.",
  "testCommands": [
    "visit",
    "new url https://www.company.com/#/",
    "contains a.nav-link, About",
    "click",
    "new url https://www.company.com/#/about",
    "hash",
    "assert expected **#/about** to equal **#/about**",
    "contains Join Us",
    "assert expected **body :not(script):contains(**'Join Us'**),
      [type='submit'][value~='Join Us']** to exist in the DOM"
  ]
}

This is a separate project that just needs to be included from your cypress/support/index.js file. For instructions see cypress-failed-log project.

Wait on the right thing

It is important to wait for the right element. Imagine we have a list of items in the DOM. We expect a new element to appear with text "Hello". We could select first the list, then the element containing "Hello" and check if it becomes visible

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
cy.get('.list')
  .contains('li', 'Hello')
  .should('be.visible')

But sometimes it does not work. When we `cy.get('.list')` Cypress saves the
DOM element as the "subject" and then tries to wait for a child of that
element with text "Hello" to become visible. If we use React for example, the
DOM of the list might be recreated from scratch if we push a new item into
the list. When that happens Cypress notices that the "subject" DOM it holds
a reference to becomes "detached from the DOM" - it becomes an orphan!

A better solution to this problem is to use a composite CSS selector that
will grab the list AND the item in a single operation.

```js
cy.contains('.list li', 'Hello')
  .should('be.visible')

This forces Cypress to wait for the list AND list item element without caching a reference to the DOM element .list. 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

You can use advanced CSS selectors to get an element in a single command.
For example instead of `cy.get('.something').first()` you could
use `cy.get('.something:first-child')`. Similarly,

```js
// get first element
cy.get('.something').first()
cy.get('.something:first-child')
// get last element
cy.get('.something').last()
cy.get('.something:last-child')
// get second element
cy.get('.something').eq(1)
cy.get('.something:nth-child(2)')

Write and read files

You can write files to disk directly from Cypress using cy.writeFile and read an existing file using cy.readFile. What if you want to read a file that might not exist? cy.readFile will fail the test if file does not exist, thus we need to find a work around.

In my case, the file I would like to load is a JSON of test values useful for Jest-like snapshot testing.

Here is a neat trick. Save the file using cy.writeFile, for example cy.writeFile('spanshots.json') whenever there is something to save. When reading the file, fetch it through the Cypress code proxy using fetch Notice how the test spec itself is served by Cypress from URL which looks like http://localhost:49829/__cypress/tests?p=cypress/integration/spec.js-438. Let us "hack" it to load a file that might not exist.

The code below tries to load snapshots file before each test. If the fetch call fails, the file does not exist.

1
2
3
4
5
6
7
8
9
10
11
12
let snapshots
beforeEach(function loadSnapshots() {
  return fetch('/__cypress/tests?p=./snapshots.json')
    .then(r => r.text())
    .then(function loadedText(text) {
      // ?
    })
    .catch(err => {
      console.error('snapshots file does not exist')
      snapshots = {}
    })
})

If the fetch call succeeds, the returned text will NOT be original JSON! Instead it will be webpacked module :)

For example, here is the saved file

snapshots.json
1
2
3
4
5
6
7
{
  "spec.js": {
    "works": [
      "foo"
    ]
  }
}

Here is how built-in Cypress bundler returns it in response to /__cypress/tests?p=./snapshots.json - I have shortened the webpack boilerplate preamble.

1
2
3
4
5
6
7
8
9
(function e(t,n,r){function ...)({1:[function(require,module,exports){
module.exports={
  "spec.js": {
    "works": [
      "foo"
    ]
  }
}
},{}]},{},[1]);

Notice that our JSON has been placed into module with id '1' (first line expression {1:). The entire bundle registers a stand alone require function. If we want to get into the actual "module" contents we have to do the following in our function loadedText(text) to finish the hack

1
2
3
4
5
6
7
8
function loadedText (text) {
  if (text.includes('BUNDLE_ERROR')) {
    // if the bundler tried to handle missing file
    return Promise.reject(new Error('not found'))
  }
  const req = eval(text)
  snapshots = req('1')
}

Boom, the object snapshots has been loaded from the file snapshots.json which might or might not exist.

Note while I have successfully used the above hack when running Cypress locally, it was always failing when doing cypress run in headless mode.

Conditional logic

Sometimes you might need to interact with a page element that does not always exist. For example there might a modal dialog the first time you use the website. You want to close the modal dialog.

1
2
3
4
5
cy.get('.greeting')
  .contains('Close')
  .click()
cy.get('.greeting')
  .should('not.be.visible')

But the modal is not shown the second time around and the above code will fail. In order to check if an element exists without asserting it, use the proxied jQuery function Cypress.$

1
2
3
4
5
6
7
8
9
const $el = Cypress.$('.greeting')
if ($el.length) {
  cy.log('Closing greeting')
  cy.get('.greeting')
    .contains('Close')
    .click()
}
cy.get('.greeting')
  .should('not.be.visible')

The above check is safe.