process

Run Changed Cypress Specs On CI First

One simple trick to speed up your end-to-end testing.

Imagine you work on an end-to-end web test. The test works locally, you commit the code, open a pull request and ... wait for 10-30 minutes for the tests to finish. Of course, your testing CI pipeline might be optimized like mine and finish quickly. Still, running all E2E tests to give you feedback on the spec you modified seems weird, doesn't it?

Run the changed spec first.

That's it. That is my advice. Normally, Cypress runs all specs it finds alphabetically. But if you pass the list of specs using the --spec ... parameter, Cypress runs them in that order:

1
2
$ npx cypress run --spec "cypress/e2e/spec-z.cy.js,cypress/e2e/spec-a.cy.js"
# runs the spec-z.cy.js first then spec-a.cy.js

So you can control the list of specs and their execution order. You can find the last commit date for each spec using the git command with a custom pretty format:

1
$ git log -1 --pretty=format:"%ct" -- <filename>

For example, this blog post has the following recent commit Unix timestamp

1
2
$ git log -1 --pretty=format:"%ct" -- source/_posts/run-changed-specs-first.md
1742823212

You just run this command for all specs, sort the results with the latest modified timestamp being first, and pass it to Cypress. I even implemented this approach in my utility find-cypress-specs.

1
$ npx cypress run --spec $(npx find-cypress-specs --sort-by-modified)

github actions

You can split finding the specs from running Cypress. For example, if you use Trying GitHub Actions:

.github/workflows/ci.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
name: ci
on:
  - push
jobs:
  test:
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout 🛎
        uses: actions/checkout@v4
        with:
          # fetch all history to get the full commit history
          # this is needed to get the correct order of specs
          # by modified Git timestamp
          fetch-depth: 0

      - uses: actions/setup-node@v4
        with:
          node-version: 22

      - name: Install dependencies 📦
        uses: cypress-io/github-action@v6
        with:
          runTests: false

      - name: Find E2E specs 🔍
        id: find-specs
        # sets the outputs "changedSpecs" and "changedSpecsN"
        run: npx find-cypress-specs --sort-by-modified --set-gha-outputs
        env:
          DEBUG: find-cypress-specs

      - name: Run E2E tests 🧪
        uses: cypress-io/github-action@v6
        with:
          install: false
          start: npm run dev
          # run specs in the order of Git modification
          # the latest specs will be run first
          spec: ${{ steps.find-specs.outputs.changedSpecs }}

Note: this approach only looks at the spec files themselves. If you modify a utility file a spec imports, we won't use the modified dependency timestamp. You can look at just the changed specs including dependencies using the find-cypress-specs using --branch main option, then it traces the spec module dependencies and imports.

cypress-fail-fast

Improvement: for pull requests you can use early termination to stop running specs if one fails. I use the plugin cypress-fail-fast to stop on the first failed test. Since we run the specs in the modified order, an error in the spec I just changed will stop the build quickly.

Stop running Cypress tests if a test fails using cypress-fail-fast plugin

Nice!

cypress-split

Make it even faster: using the cypress-split plugin to run specs in parallel using multiple CI containers. When using the spec order, make sure to pass the specs using the environment variable in addition to the spec parameter.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
- name: Find E2E specs 🔍
  id: find-specs
  # sets the outputs "changedSpecs" and "changedSpecsN"
  run: npx find-cypress-specs --sort-by-modified --set-gha-outputs
  env:
    DEBUG: find-cypress-specs

# use cypress-split plugin to run the changed specs in order
- name: Run E2E tests in parallel 🧪
  uses: cypress-io/github-action@v6
  env:
    # pass the machine index and the total number
    SPLIT: ${{ strategy.job-total }}
    SPLIT_INDEX: ${{ strategy.job-index }}
    # IMPORTANT ⚠️
    # pass the specs to the cypress-split plugin
    SPEC: ${{ steps.find-specs.outputs.changedSpecs }}
  with:
    install: false
    start: npm run dev
    # run specs in the order of Git modification
    # the latest specs will be run first
    spec: ${{ steps.find-specs.outputs.changedSpecs }}

This should do the trick across N machines