products

Run Cypress Specs In Parallel For Free

How to use the `cypress-split` plugin to speed up your Cypress tests on CI.

A long time ago I worked at Cypress.io and we opened sourced the test runner itself. If the test runner cypress is free, how would the company make money? We needed a feature for our Cypress Dashboard that would be easy to justify business to pay for. Recording test artifacts like screenshots and videos? Ughh, yeah, but it is easy to store those static test reports, so it was not worth much.

I was playing with Mocha test runner at that time, creating its variants like Rocha (randomize the order of tests), Focha (run the previously failed tests first), and Locha (run the failed tests again with extra log verbosity). When running a lot of tests, the start-up time was annoying, and I thought for myself:

Why can't my test runner be already open, waiting for me to send it the spec to test?

This is how the Cypress parallelization feature was born. I modified the Cypress test runner to stay open after running a spec, and ask the Dashboard API to send it the next spec to run. Thus several test runners could split the entire list of specs, each test runner only running some of the specs. Then I put the paid Dashboard up and Cypress the company started selling paid plans like hot cupcakes. Who could deny that speeding up CI tests N times (where N is the number CI machines) is not worth it?!!

CI integrations

Then I created Cypress CircleCI Orb and Cypress GitHub Action to make the actual mechanics of installing, caching, and running Cypress on each CI simpler to configure. Here is how you could run your tests across 4 CircleCI machines:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
version: 2.1
orbs:
  cypress: cypress-io/cypress@2
workflows:
  build:
    jobs:
      # first get the source code and install npm dependencies
      - cypress/install:
          # run a custom app build step
          build: 'npm run build'
      - cypress/run:
          # make sure app has been installed and built
          # before running tests across multiple machines
          # this avoids installing same dependencies 10 times
          requires:
            - cypress/install
          record: true # record results on Cypress Dashboard
          parallel: true # split all specs across machines
          parallelism: 4 # use 4 CircleCI machines to finish quickly
          group: 'all tests' # name this group "all tests" on the dashboard
          start: 'npm start' # start server before running tests

The same example using GitHub Actions:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
name: Parallel Cypress Tests

on: [push]

jobs:
  test:
    name: Cypress run
    runs-on: ubuntu-22.04
    strategy:
      fail-fast: false
      matrix:
        containers: [1, 2, 3, 4]
    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - name: Cypress run
        uses: cypress-io/github-action@v5
        with:
          record: true
          parallel: true
        env:
          # pass the Cypress Cloud record key as an environment variable
          CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}

Tip: if you are running Cypress using GitHub Actions, you can simplify it using my reusable Cypress workflows; the same example above can be written using just:

1
2
3
4
5
6
7
8
9
10
name: ci
on: [push]
jobs:
  test:
    # https://github.com/bahmutov/cypress-workflows
    uses: bahmutov/cypress-workflows/.github/workflows/parallel.yml@v1
    with:
      n: 4
    secrets:
      recordKey: ${{ secrets.CYPRESS_RECORD_KEY }}

cypress-split

Now that I no longer work at Cypress.io the company and don't care about their business success, I had a chance to sit down and think how to run Cypress tests in parallel on CI without paying for 3rd party service.

You deserve your Cypress tests to run very quickly FOR FREE.

I have come up with cypress-split: a tiny Cypress plugin that lets you automatically split the entire list of Cypress specs to run in parallel on any CI(or even when running Cypress locally using Docker containers). Install the plugin following its README documentation and use one of the CI examples provided to run the tests on your server. For example, let's run Cypress tests on 4 CircleCI machines:

1
2
3
4
5
6
7
8
9
10
# to use orbs, must use version >= 2.1
version: 2.1
orbs:
  cypress: cypress-io/cypress@2
workflows:
  build:
    jobs:
      - cypress/run:
          parallelism: 4
          command: npx cypress run --env split=true

Boom 💥

No --record, no --parallel flags, no paying for an external service. All specs split across 4 machines. Other CI providers work similarly simply. I have prepared a couple of repos with GitHub Actions, Circle, and GitLab CI working.

CI repo
CircleCI cypress-split-example
GitLab CI cypress-split-gitlab-example
GitHub Actions cypress-split-example
Reusable GHA workflow cypress-split-example

🎓 I am showing how to install and configure the cypress-split plugin for different CI providers in my Cypress Plugins course.

Reusable GitHub Actions workflows

I have added a special reusable GitHub workflow to run using cypress-split to my cypress-workflows. To me, this is the simplest way to run Cypress tests in parallel on GitHub Actions:

1
2
3
4
5
6
7
8
name: ci
on: [push]
jobs:
  test:
    # https://github.com/bahmutov/cypress-workflows
    uses: bahmutov/cypress-workflows/.github/workflows/split.yml@v1
    with:
      n: 4

The cypress-split plugin even outputs its information into GitHub Actions report:

cypress-split on GitHub Actions

Timing

Cypress Dashboard uses previous spec timings to queue the specs when then run in parallel. This algorithm optimizes the total execution duration to ensure the longest specs finish first. The cypress-split plugin does not have the previous spec run information and splits the specs alphabetically. This works well, if all specs are approximately the same duration. I think I might make it work by saving historic timings, I am still playing with the plugin. If you have any ideas, let me know.

See also