products

Do Not Let Cypress Cache Snowball on CI

Do not use lax restore cache keys or your Cypress cache will blow up in size

Let's say you are running Cypress on Continuous Integration service, like Circle or GitHub Actions. You probably want to cache NPM modules and Cypress binary - these things are large. Let's leave the NPM cache alone - you can read the post Do Not Let NPM Cache Snowball on CI on how to avoid the cache growing out of the control. Instead let's look at the Cypress binary cache.

Note: you can find the source code for this blog post in the repository bahmutov/cypress-snowball.

We start with installing Cypress v4.12.1 and then write the circle.yml file

circle.yml
1
2
3
4
5
6
7
8
9
version: 2.1
orbs:
  node: circleci/node@4
jobs:
  build:
    executor: node/default
    steps:
      - checkout
      - run: node --version

The job passes.

Node job on CircleCI

Now let's configure caching following Circle caching docs. Before installing NPM dependencies, we need to restore the previous cache. By copying the Circle docs we get the following:

circle.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
version: 2.1
orbs:
  node: circleci/node@4
jobs:
  build:
    executor: node/default
    steps:
      - checkout
      - run: node --version
      - restore_cache:
          keys:
            # Find a cache corresponding to this specific package-lock.json checksum
            # when this file is changed, this key will fail
            - v1-deps-{{ checksum "package-lock.json" }}
            # Find the most recently generated cache used from any branch
            - v1-deps-
      - run: npm ci
      - run: npx cypress cache list
      - save_cache:
          key: v1-deps-{{ checksum "package-lock.json" }}
          paths:
            # NPM module cache
            - ~/.npm
            # Cypress binaries are stored here
            # according to https://on.cypress.io/caching
            - ~/.cache/Cypress

The build runs. It does not find a previous cache to restore, first trying the exact match "v1-deps-package-lock hash", then trying just the prefix "v1-deps-".

First cache run

Cypress v4.12.0 is installed, and is visible using cypress cache list command. The two folders are then stored under the key "prefix + package-lock hash".

Now let's upgrade Cypress dependency to the currently latest version v5.6.0

1
2
$ npm i -D [email protected]
+ [email protected]

This changes the package lock file, and when we push the commit we observe the following on Circle.

Second cache run

Hmm. We have a problem. The exact key to restore the cache was not found, because the package lock file has changed. But the previous cache did match the second restore cache key with just the prefix "v1-deps-". Thus we have restored Cypress v4.12.1 binary, installed Cypress v5.6.0 (you can see both of them in the list), and then saved both in the new cache. The cache was 171MB with just Cypress v4.12.1 and is probably much larger now with two Cypress binaries.

Let's install a different Cypress version, like v5.4.0 and trigger the CI build.

Third cache run

Wow, the cached folder is really growing. Notice the time it takes to save the updated cache with three Cypress binaries: 1 minute and 19 seconds. Do you know how long it should take to cache just a single Cypress version? 20 seconds.

Caching a single Cypress version takes just 20 seconds

It takes a lot of extra time to restore and save those extra versions of Cypress, doesn't it.

We ourselves got this wrong in GitHub Actions, see issue #219.

Advice

So what should we do to avoid snowballing Cypress binary cache? Use the exact restore key version and do not use the fallback prefix key.

circle.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
version: 2.1
orbs:
  node: circleci/node@4
jobs:
  build:
    executor: node/default
    steps:
      - checkout
      - run: node --version
      - restore_cache:
          keys:
            # only use the exact cache key to avoid
            # snowballing multiple versions
            - v1-deps-{{ checksum "package-lock.json" }}
      - run: npm ci
      - run: npx cypress cache list
      - save_cache:
          key: v1-deps-{{ checksum "package-lock.json" }}
          paths:
            # NPM module cache
            - ~/.npm
            # Cypress binaries are stored here
            # according to https://on.cypress.io/caching
            - ~/.cache/Cypress

Advice: use Cypress orb

When running Cypress tests on CircleCI, you have an even better option. You can use the official Cypress Orb to dramatically simplify installing, caching, and running Cypress tests.

For example to just install NPM dependencies with caching, including Cypress binary, one could write the following circle.yml file from branch use-cypress-orb:

circle.yml
1
2
3
4
5
6
7
8
9
10
version: 2.1
orbs:
  # https://github.com/cypress-io/circleci-orb
  cypress: cypress-io/cypress@1
workflows:
  build:
    jobs:
      - cypress/install:
          post-install:
            - run: npx cypress cache list

Note: typically you would use cypress/run job, here cypress/install is used for clarity. They work the same in terms of installation and caching.

The first run with Cypress v5.5.0 has a single binary version to cache.

First run with Cypress orb

After installing a different version of Cypress, the CI runs again. It works correctly - only a single new version of Cypress is carried.

Second run with Cypress orb after changing Cypress version

Happy caching!