process

My Vision for Component Tests in Cypress

How I see end-to-end and component and unit tests working together

I have been talking about framework-specific component testing for ages now. I have even coded a bunch of adaptors like cypress-react-unit-test, cypress-vue-unit-test and others, that allow mounting individual components into Cypress and run them as full-fledged mini-web applications. While the initial solution was technically ok, due to the way Cypress works it has a long list of issues: React Hooks do not work, styles are often lost, etc.

I am excited to say that we have an experimental support for a new way of binding test code in Cypress v4.5.0 that seems to solve all known problems. In this blog post I will show how it might work, once we release it.

⚠️ Warning: component testing is currently in Alpha release. If you find a problem, please open an issue in the adaptor repo (like [bahmutov/cypress-react-unit-test][crut], [bahmutov/cypress-vue-unit-test][cvut]). Use this awesome feature at your own risk.

Example application

I will take a nice Todo application from blog post How To Build a React To-Do App with React Hooks as the starting point. You can see this application yourself at its original https://codesandbox.io/s/oj3qm2zq06 URL.

React Todo App

I have downloaded the code from the sandbox to my repo https://github.com/bahmutov/react-todo-with-hooks; it has a CSS file and two JS files - very compact example.

1
2
3
4
5
6
7
8
/repo
  public/
    index.html
  src/
    App.css
    App.js
    index.js
  package.json

The application is bundled and served using react-scripts which is often used to serve modern React applications.

package.json
1
2
3
4
5
6
7
8
9
10
11
12
13
{
  "dependencies": {
    "react": "16",
    "react-dom": "16",
    "react-scripts": "3.1"
  },
  "scripts": {
    "start": "react-scripts start",
    "build": "react-scripts build",
    "test": "react-scripts test",
    "eject": "react-scripts eject"
  }
}

The Goal

I have noticed that the application allows us to mark Todo items as completed, but it never allows to "undo" completing an item. Once the task is done, there is no way to get it back to the initial incomplete state.

Completing an item cannot be undone

I would like to change "complete" an item into "toggle" an item. But I don't know the code of the application, so it is dangerous to simply start hacking. We need tests first; and we want to make sure we test the entire application before changing its behavior.

Adding tests

1
2
$ npm i -D cypress
+ [email protected]

Our first full end-to-end test goes through a typical user story.

cypress/integration/todo-spec.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
/// <reference types="cypress" />
describe('Todo App', () => {
  it('completes an item', () => {
    // base url is stored in "cypress.json" file
    cy.visit('/')
    // there are several existing todos
    cy.get('.todo').should('have.length', 3)
    cy.log('**adding a todo**')
    cy.get('.input').type('write tests{enter}')
    cy.get('.todo').should('have.length', 4)

    cy.log('**completing a todo**')
    cy.contains('.todo', 'write tests').contains('button', 'Complete').click()
    cy.contains('.todo', 'write tests')
      .should('have.css', 'text-decoration', 'line-through solid rgb(74, 74, 74)')

    cy.log('**removing a todo**')
    // due to quarantine, we have to delete an item
    // without completing it
    cy.contains('.todo', 'Meet friend for lunch').contains('button', 'x').click()
    cy.contains('.todo', 'Meet friend for lunch').should('not.exist')
  })
})

The test runs and passes

Full Cypress end-to-end test

Code coverage

Did we test all code paths in our application? The simplest way for us to find out is to measure how much of the application's code was executed by running this one e2e test. We can follow the Cypress Code Coverage Guide to set up coverage reporting.

  1. Instrument the application's code while running. For any application that uses react-scripts we can use module cypress-io/instrument-cra to do so on the fly.
1
2
$ npm i -D @cypress/instrument-cra
+ @cypress/[email protected]

When we start the application, we preload this module first - and we will have application's code instrumented.

package.json
1
2
3
4
5
{
  "scripts": {
    "start": "react-scripts -r @cypress/instrument-cra start"
  }
}
  1. We need to add @cypress/code-coverage to Cypress to merge coverage from tests and save report
1
2
$ npm i -D @cypress/code-coverage
+ @cypress/[email protected]

This plugin should be loaded from the support and plugins files

cypress/support/index.js
1
import '@cypress/code-coverage/support'
cypress/plugins/index.js
1
2
3
4
5
6
module.exports = (on, config) => {
  require('@cypress/code-coverage/task')(on, config)
  // IMPORTANT to return the config object
  // with the any changed environment variables
  return config
}

Start Cypress again and run the test. You see "Saving coverage from ..." and "Coverage report" messages at the end of the run.

Code coverage messages

In the folder coverage you will find the report in several formats, let's open the static HTML one

1
$ open coverage/lcov-report/index.html

Our single end-to-end test was very effective at covering almost all lines of the application.

Total code coverage

We can drill down into individual file coverage.

A single line missed

Component test

The line we missed is inside TodoForm component, and it is kind of hard to confirm its behavior. We can easily write an end-to-end test that tries to "enter" empty input.

1
2
3
cy.get('input').type('{enter}')
// there are still 3 todos
cy.get('.todo').should('have.length', 3)

BUT it does not confirm the property addTodo is not called at all. We really want to test the component, not the entire application.

Let's write a component test.

  1. Install cypress-react-unit-test v4+
1
2
$ npm i -D cypress-react-unit-test
+ @[email protected]
  1. In cypress.json enable experimental Cypress feature
1
2
3
4
5
6
{
  "baseUrl": "http://localhost:3000",
  "testFiles": "**/*spec.js",
  "experimentalComponentTesting": true,
  "componentFolder": "src"
}

While end-to-end tests reside by default in cypress/integration folder, let's place component tests right alongside the source code in src folder. We will filter the spec files using "testFiles": "**/*spec.js" parameter.

  1. Load cypress-react-unit-test from the support and plugins files
cypress/support/index.js
1
require('cypress-react-unit-test/support')
cypress/plugins/index.js
1
2
3
4
5
6
module.exports = (on, config) => {
  require('cypress-react-unit-test/plugins/cra-v3')(on, config)
  // IMPORTANT to return the config object
  // with the any changed environment variables
  return config
}

Unlike end-to-end tests, the component specs must be bundled the same way as the application code is. Thus we have added multiple plugins helpers that find and use the bundling options your application is using. In this case, Cypress will find the Webpack config from react-scripts module and will use it.

Let's write component test!

src/TodoForm.spec.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
import React from "react"
import {TodoForm} from "./App"
import {mount} from 'cypress-react-unit-test'

describe('TodoForm', () => {
  it('ignores empty input', () => {
    const addTodo = cy.stub()
    mount(<TodoForm addTodo={addTodo} />)
    cy.get('input').type('{enter}')
    .then(() => {
      expect(addTodo).not.have.been.called
    })

    cy.get('input').type('hello there{enter}')
    .then(() => {
      expect(addTodo).to.be.calledWith('hello there')
    })
  })
})

The component test directly imports TodoForm from the application code and mounts it using mount method from the cypress-react-unit-test. Once the component is mounted, it runs as a "mini" web application. We can use normal Cypress commands against it! Think of mount as cy.visit for components.

TodoForm component test

You can see the component run inside Cypress iframe (where a regular web application usually runs during end-to-end test). You can interact with the component, inspect it using DevTools, see how it behaves using time-traveling debugger - all Cypress benefits apply both to end-to-end tests and to component tests. Plus code coverage is included by default!

Styles

When mounting a component, you might want to apply additional styles to make it look the same as in real application. The mount command options let you specify inline style, CSS filename or external stylesheets. For example, here is the Todo component test.

src/Todo.spec.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import {Todo} from "./App"
it('renders new item', () => {
  const todo = {
    text: 'test item',
    isCompleted: false
  }
  // application loads Bulma library from public/index.html
  // so let's load it from the component test too
  mount(
    <Todo todo={todo} />,
    {
      stylesheets: [
        'https://cdnjs.cloudflare.com/ajax/libs/bulma/0.7.2/css/bulma.css'
      ]
    }
  )
  cy.contains('.todo button', 'Complete')
})

First Todo test

The buttons looks "normal", but the entire component still does not look the same - because our styles come from src/App.css and require certain DOM structure. Let's recreate the structure and load additional CSS file in our test.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
it('renders with styles', () => {
  const todo = {
    text: 'test item',
    isCompleted: false
  }
  const TestTodo = () => <div className="app"><Todo todo={todo} /></div>
  mount(
    <TestTodo />,
    {
      stylesheets: [
        'https://cdnjs.cloudflare.com/ajax/libs/bulma/0.7.2/css/bulma.css'
      ],
      cssFile: 'src/App.css'
    }
  )
  cy.contains('.todo button', 'Complete')
})

Second Todo test

Looks good.

Testing the interface

As I argued before - a component test is just a unit test where props are inputs and side effects like DOM, network calls, etc are outputs. Let's confirm that Todo component calls the removeTodo prop when the user clicks "Remove" button.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
it('deletes an item', () => {
  const todo = {
    text: 'test item',
    isCompleted: false,
  }
  const removeTodo = cy.stub().as('remove')

  mount(
    <Todo todo={todo} index={123} removeTodo={removeTodo} />,
    {
      stylesheets: [
        'https://cdnjs.cloudflare.com/ajax/libs/bulma/0.7.2/css/bulma.css'
      ]
    }
  )
  cy.contains('.todo', 'test item')
    .find('[data-cy="remove"]').click()
  cy.get('@remove').should('have.been.calledWith', 123)
})

Removing todo

GraphQL example

Notice how we directly passed a stub into the component as a property - because the component "lives" right inside the spec. By having direct access to the component, you can do amazing things - like combine mock and live GraphQL calls, see bahmutov/test-apollo

test-apollo/src/Library.spec.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
import React from 'react'
import {Library, GET_BOOKS} from './App'
import {mount} from 'cypress-react-unit-test'
import { MockedProvider } from '@apollo/react-testing'

// mocking GraphQL requests using
// https://www.apollographql.com/docs/react/development-testing/testing/#mockedprovider
describe('Library', () => {
  beforeEach(() => {
    cy.fixture('books').as('books')
  })

  it('shows loading while making the query', function () {
    // delays the response by 3 seconds
    const mocks = [
      {
        request: {
          query: GET_BOOKS
        },
        result: this.books,
        delay: 3000
      }
    ]
    mount(
      <MockedProvider mocks={mocks} addTypename={false}>
        <Library />
      </MockedProvider>
    )

    // 😀 compare declarative testing vs promise waits in
    // https://www.apollographql.com/docs/react/development-testing/testing/#testing-final-state
    cy.contains('Loading ...').should('be.visible')
    cy.get('[data-cy=book]').should('have.length', 2)
    cy.contains('Loading ...').should('not.exist')
  })
})

Mocking GraphQL with delay component test

Components all the way down

Component testing can work with components of any size. We have tested TodoForm and Todo components, let's test the top-level App component.

src/App.spec.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
import React from "react"
import App from "./App"
import {mount} from 'cypress-react-unit-test'

describe('App', () => {
  beforeEach(() => {
    mount(
      <App />,
      {
        stylesheets: [
          'https://cdnjs.cloudflare.com/ajax/libs/bulma/0.7.2/css/bulma.css'
        ]
      }
    )
  })

  it('works', () => {
    cy.get('.todo').should('have.length', 3)
    cy.get('input.input').type('Test with Cypress{enter}')
    cy.get('.todo').should('have.length', 4)
      .contains('Meet friend for lunch')
      .find('[data-cy=remove]').click()


    cy.get('.todo').should('have.length', 3)
  })
})

The test runs and looks ... almost like the complete application!

App component test

Component testing gives you a flexibility over the scope of the code you want to test. In addition to testing your application as a whole via end-to-end tests you can test a subset of the application "tree". If your application has the top level authentication that is hard to bypass from an end-to-end test - you can test the component that sits right under the authentication provider.

Finally, while we are testing components, let's test a few functions using unit tests. While this was always possible in Cypress, component testing mounting mode makes it much closer to testing the real code, because bundling the component spec is done using your application's settings.

src/App.spec.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
import App, {toggleOneTodo} from "./App"
describe('App', () => {
  it('toggles correctly', () => {
    const todos = [{
      isCompleted: false
    }, {
      isCompleted: false
    }, {
      isCompleted: false
    }]
    const newTodos = toggleOneTodo(todos, 2)
    expect(newTodos).to.deep.equal([{
        isCompleted: false
      }, {
        isCompleted: false
      }, {
        isCompleted: true
      }])
  })
})

From the smallest units of code to the largest components - you control the scale of what you want to test.

Benefits

In my completely biased personal opinion, Cypress component testing using the real browser has many advantages

Feature Cypress + cypress-X-unit-test
Test runs in real browser ✅
Cross-platform Chrome / Firefox / Microsoft Edge
Uses full mount ✅
Test speed as fast as the app works in the browser
Test can use additional plugins use any Cypress plugin
Test can interact with component use any Cypress command
Debugging use browser DevTools, Cypress time-traveling debugger
Re-run tests on file or test change ✅
Test output on CI terminal, screenshots, videos
Tests can be run in parallel ✅ via parallelization
Test against interface ✅ and can use @testing-library/cypress
Spying and mocking Sinon library
Code coverage ✅
Visual testing ✅ via visual plugins

Examples

We have forked a number of 3rd party projects to confirm component testing works.

Repo Description
try-cra-with-unit-test Hello world initialized with CRAv3
try-cra-app-typescript Hello world initialized with CRAv3 --typescript
react-todo-with-hooks Modern web application using hooks
test-redux-examples Example apps copies from official Redux repo and tested as components
test-react-hooks-animations Testing React springs fun blob animation
test-mdx-example Example testing MDX components using Cypress
test-apollo Component testing an application that uses Apollo GraphQL library
test-xstate-react XState component testing using Cypress
test-react-router-v5 A few tests of React Router v5
test-material-ui Testing Material UI components: date pickers, lists, autocomplete
test-d3-react-gauge Testing React D3 gauges
storybook-code-coverage Example app where we get 100% code coverage easily with a single integration spec and a few component specs, replacing several tools
react-loading-skeleton One to one Storybook tests for React skeleton components. Uses local .babelrc settings without Webpack config
test-swr Component test for Zeit SWR hooks for remote data fetching

To find more examples, see GitHub topic cypress-react-unit-test-example

Future plans

We have already released new versions of cypress-react-unit-test and cypress-vue-unit-test and plan to upgrade other framework adaptors (Angular, Svelte, etc) to be compatible with experimentalComponentTesting: true mode.

Thank you

Special thank you 👏 to Dmitriy Kovalenko @dmtrKovalenko for making Cypress Test Runner and React adaptor work through these PRs #5923 and #108. They have removed the technical limitations that prevented React Hooks, component styles and other features from working correctly inside the component tests.

Another big shout out goes to Jessica Sachs @_JessicaSachs for working on component testing support in cypress-vue-unit-test 👏.