Tips And Tricks For Writing Fast And Maintainable Front-End Tests

Recently I had privilege to deliver a presentation to the special 10th year anniversary conference AllThingsOpen in Raleigh, North Carolina. You can find the slides at https://slides.com/bahmutov/tips-tricks and embedded below. In this blog post I will summarize my main speaking points for anyone interested.

Once the video recording from my talk comes out, I will link it here.

I work at Mercari US where I set up the automation group. We take care of writing mostly end-to-end tests using Cypress. We have reached almost 700 full tests which we run several times per day.

Use the browser

I started the presentation with the following advice

Web app tests should run in the browser

To me this advice seems obvious, yet many people write front-end tests using JSDom emulation running in Node. Running Cypress tests in the browser lets you see the component and the web application the same way the real human user would see it.

Component tests

While end-to-end tests are awesome and ensure the entire system works, it is hard to control the application sometimes. Things that make writing deterministic tests difficult are:

• Random data
• External data
• Timers and clocks
• Conditional testing
• Unclear test requirements

To better isolate and test individual pieces of front-end code, we can write component tests using Cypress. Here is a typical component test for the Timer component

import React from 'react'
import { Timer } from './Timer'
import '../App.css'
import { SudokuContext } from '../context/SudokuContext'
import moment from 'moment'

describe('Timer', () => {
  it('sets the clock to the given value', () => {
    const timeGameStarted = moment().subtract(
      900,
      'seconds',
    )

    cy.mount(
      <SudokuContext.Provider value={{ timeGameStarted }}>
        <section className="status">
          <Timer />
        </section>
      </SudokuContext.Provider>,
    )
    cy.contains('15:00')
  })
})

Notice how we control the time when the game has started to ensure the Timer shows "15:00" and we check the page using the standard cy.contains('15:00') command. One of the strong points of Cypress component testing is that the component runs as a mini-web application. After mounting the component, you interact with the page and confirm the component works as you expect it to work.

The knowledge we have accumulated writing Cypress end-to-end tests directly helps us write Cypress component tests. But there is more.

Framework-independent syntax

Cypress component testing mounts the component using framework-specific code. But once cy.mount(component, options) finishes, all commands are pretty much framework-independent. Even the cy.mount itself hides all the framework-specific details.

// JSX for React
// Object syntax for Vue, Svelte
// Object or template syntax for Angular
cy.mount(component, options)
// standard Cypress commands

Thus learning Cypress component once directly benefits any developer; they can test other applications with ease.

Tip: For more examples of testing various components from React, Angular, Vue, and Svelte and how the tests look the same, check out my presentation Test Angular, React, Vue, Svelte Components Without Fear.

Picking the type of the test

I have summarized the decision table for the type of the test to write and run using Cypress in the following table. Pick the test depending on what you want to test.

Unit test

expect(formatTime({ seconds: 3 })).to.equal('00:03')

• Small chunks of code like functions and classes

Component test

import { Component } from './Component'
cy.mount(<Component props=... />)
cy.get(...).click()

• Front-end React / Angular / Vue / X components
• Easy to test edge conditions

End-to-end test

cy.visit('/')
cy.get(...).click()

• Web application
• Easy to test the entire user flow

At MercariUS we mostly use End-to-end tests and only are exploring the component tests. Thus here are out test numbers by type:

Test type	Number of front-end tests
Unit tests	346
Component tests	WIP
End-to-end tests	695

As you can see, E2E tests dominate our testing strategy

The tests should be fast

To make E2E tests, we can safely bypass the user page UI. We use API commands to create all the data for the tests, we cache data a lot using cypress-data-session. We strive to keep every E2E test under 3 minutes long. On average, our tests are 1 minute long. Even then, 700 tests would take 12 hours to run if we use one machine.

The key to running all tests quickly on CI is parallelization. We use Cypress CircleCI Orb to configure the number of machines to use during each test run. This is how we can finish all the tests under 30 minutes. Potentially we could finish all the tests in 3 minutes by running them all in parallel if we use a really large number of CI containers.

When people discuss how fast end-to-end tests are, they focus on running the tests locally. Yet, they really should consider all aspects of writing, running, debugging, and maintaining tests. I have explored this topic in my previous presentation Testing Pyramid Makes Little Sense (And What We Can Use Instead).

Maintainable tests

The key to maintainable tests is to make sure the tests are understood by someone other than their original author. Thus at MercariUS we go through the test review pull requests, just like we do for the application code itself. We focus on writing readable test code (using the custom commands, utility functions, and types when needed). Here is a typical E2E test, I think you will understand what it is doing just by reading the source code

cy.signup(seller)

cy.createListing({
  name: `Macbook one ${Cypress._.random(1e10)}`,
  description: 'Seller will delete all items',
  price: 198,
})
cy.createListing({
  name: `Macbook two ${Cypress._.random(1e10)}`,
  description: 'Seller will delete all items',
  price: 199,
})

visitBlankPage()
cy.loginUserUsingAPI(seller)
cy.visitProtectedPage('/mypage/listings/active')
cy.byTestId('Filter', 'Active').should('be.visible').and('contain', '2')
cy.byTestId('ListingRow').should('be.visible').and('have.length', 2)

Making the errors actionable

One of the big time commitments when writing tests is debugging failed tests. The key goal here is to make sure any failed test gives a clear understand of the root cause for its failure. As an example, I give a test where the page did not transition to the target URL after clicking a button. We have modified the test to spy on the network call the application makes when the user clicks the button.

cy.interceptGraphQL('editDraftItemMutation') // added
cy.byTestId('NewListerSidebarSell').click()
cy.waitGraphQL('@editDraftItemMutation') // added

With those two commands added, the test fails with a much more actionable error:

GraphQL call @editDraftItemMutation failed

Great, the backend team can investigate the source of the failure.

Tip: I cover cy.intercept and cy.wait commands and their custom implementation in my Cypress Network Testing Exercises course.

Notify the right people

When the tests run nightly, or every N hours, it is important to make sure the right team is notified when a test fails. We have implemented test tagging by feature and posting Slack messages tagging the specific team based on the test feature tag using my plugin cypress-slack-notify.

Stub stable APIs

I finish the presentation with the advice to stub any calls the component code makes from the browser, rather than trying to stub the internal source code. The internal source code is likely to change, requiring test code changes. But backend APIs are unlikely to change very often, thus a test that stubs an Ajax call is likely to work even if the internals of the component are rewritten. In the end-to-end and component tests we can use Cypress network control using the cy.intercept command.

it('shows the top times', () => {
  cy.intercept('GET', '/times/90', {
    fixture: 'times.json',
  }).as('scores')
  cy.mount(<Overlay overlay={true} time={90} />)
  cy.wait('@scores')
  cy.get('.overlay__times li').should('have.length', 4)
  cy.contains('.overlay__times li', '01:30').should(
    'have.class',
    'overlay__current',
  )
})

🎓 Seriously, my course Cypress Network Testing Exercises covers this type of testing pretty well, you are going to love writing tests like this one.

Conclusions

Here is my final slide with the main conclusions of the talk

• Write and run tests in the browser
  • End-to-end, component, and unit tests
• There are different kinds of speed
  • Writing the tests
  • Running the tests
  • Debugging the failures
• Maintenance is hard
  • Optimize for understanding

Other Mercari testing talks

I have described how we do large-scale web testing web at Mercari US. Find the presentations at https://slides.com/bahmutov/decks/mercari.

With that, Happy Testing!