products

Fast Cypress spec bundling using ESBuild

Measuring how fast esbuild bundles Cypress specs

Cypress Webpack bundler

By default, Cypress bundles the spec files using the built-in preprocessor that uses Webpack under the hood. I wanted to see how long it takes to bundle average spec files in the cypress-io/cypress-example-todomvc application. Unfortunately, the preprocessor does not expose the timings directly, but we can always hack on Cypress code right in the binary.

1
2
3
4
5
$ npx cypress info
...
Binary Caches: /Users/gleb/Library/Caches/Cypress

Cypress Version: 6.7.1

Next, I open the file in that binary cache folder that serves the spec files to the browser. In this case it is the file:

1
/Users/gleb/Library/Caches/Cypress/6.7.1/Cypress.app/Contents/Resources/app/packages/server/lib/controllers/spec.js

Tip: this is the transpiled file, you can find the original in Cypress repo

The controllers/spec.js uses the debug module to print the debug messages

server/lib/controllers/spec.js
1
const debug = require('debug')('cypress:server:controllers:spec')

Tip: most source files in Cypress Test Runner use debug to print debug messages, allowing you to peek under the hood. Find some common log sources in the docs.

I have inserted a few additional debug statements to measure how long the preprocessor takes to bundle the spec file.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
const debug = require('debug')('cypress:server:controllers:spec')
// built-in Cypress file bundler, uses Webpack
const preprocessor = require('../plugins/preprocessor')

// by filename
const starts = {}
module.exports = {
  handle (spec, req, res, config, next, onError) {
    debug('request for %o', { spec })
    starts[spec] = + new Date()
    // bundle the given spec file
    return preprocessor
    .getFile(spec, config)
      .then((filePath) => {
        const ended = + new Date()
        const elapsed = ended - starts[spec]
        debug('sending spec %o after %d ms', { filePath }, elapsed)
        const sendFile = Promise.promisify(res.sendFile.bind(res))

        return sendFile(filePath)
      })
  }
}

Let's run the tests in using cypress run command. During the execution the spec and the support files are each bundled just once. It is a cold start - Cypress does not store or load any bundling information to speed things up.

1
2
3
4
5
6
7
8
9
10
11
12
13
$ DEBUG=cypress:server:controllers:spec npm run test:ci

Running:  app_spec.js                                                                     (1 of 1)
cypress:server:controllers:spec request for { spec: 'cypress/support/index.js' } +0ms
cypress:server:controllers:spec request for { spec: 'cypress/integration/app_spec.js' } +3ms
cypress:server:controllers:spec sending spec {
  filePath: '/Users/gleb/Library/Application Support/Cypress/cy/production/projects/
  cypress-example-todomvc-411ba7b931279226890f4fef43e9d6c5/bundles/cypress/integration/app_spec.js' }
  after 1143 ms +1s
cypress:server:controllers:spec sending spec {
  filePath: '/Users/gleb/Library/Application Support/Cypress/cy/production/projects/
  cypress-example-todomvc-411ba7b931279226890f4fef43e9d6c5/bundles/cypress/support/index.js' }
  after 1702 ms +557ms

I have run the same command 3 times on my Mac to get the following timings:

Support file (ms) Spec file (ms)
1702 1143
1708 1178
1715 1171

Ok, so about 1700ms to bundle the support file and 1100ms to bundle the spec file.

We can look at the files we are bundling. The support file only imports two other modules:

cypress/support/index.js
1
2
require('./commands')
require('cypress-axe')

The commands.js file only defines 3 custom commands:

cypress/support/commands.js
1
2
3
Cypress.Commands.add('createDefaultTodos', function () { ... })
Cypress.Commands.add('createTodo', function (todo) { ... })
Cypress.Commands.add('addAxeCode', () => { ... })

The support is pretty much just bundling the cypress-axe NPM module.

The spec file has no imports, it is a single standalone JavaScript spec file.

ESBuild

The ESBuild is the new bundler that uses an optimized binary bundler. Let's see how fast it is.

1
2
$ npm i -D esbuild
+ [email protected]

We can bundle the files from the command line

1
2
3
4
5
6
7
8
9
10
11
$ npx esbuild cypress/support/index.js --bundle --outfile=out.js

  out.js  994.9kb

⚡ Done in 93ms

$ npx esbuild cypress/integration/app_spec.js --bundle --outfile=out.js

  out.js  11.7kb

⚡ Done in 4ms

Wow. Ok. Can we use the esbuild bundler from Cypress?

ESBuild file preprocessor

ESBuild has good JavaScript API which we can use to write our own file preprocessor. You can find my NPM module in bahmutov/cypress-esbuild-preprocessor repo and install via NPM

1
2
$ npm i -D @bahmutov/cypress-esbuild-preprocessor
+ @bahmutov/[email protected]

Note: esbuild is a peer dependency.

For example to build the spec file, the preprocessor does the following:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
const esbuild = require('esbuild')
const debug = require('debug')('cypress-esbuild-preprocessor')

// bundled[filename] => promise
const bundled = {}

const bundleOnce = ({ filePath, outputPath }) => {
  const started = +new Date()

  esbuild.buildSync({
    entryPoints: [filePath],
    outfile: outputPath,
    bundle: true,
  })
  const finished = +new Date()
  const elapsed = finished - started
  debug('bundling %s took %dms', filePath, elapsed)
}

const filePreprocessor = (file) => {
  const { filePath, outputPath, shouldWatch } = file

  debug({ filePath, outputPath, shouldWatch })

  if (!shouldWatch) {
    bundleOnce({ filePath, outputPath })
    return outputPath
  }
  // watch mode
}

We can point Cypress to use the above preprocessor

cypress/plugins/index.js
1
2
3
4
5
module.exports = (on, config) => {
  // `on` is used to hook into various events Cypress emits
  // `config` is the resolved Cypress config
  on('file:preprocessor', require('@bahmutov/cypress-esbuild-preprocessor'))
}

My preprocessor includes timing, thus we can directly see the bundling performance

1
2
3
4
5
6
7
$ DEBUG=cypress-esbuild-preprocessor npm run test:ci

  Running:  app_spec.js                                                                     (1 of 1)
  cypress-esbuild-preprocessor bundling /Users/gleb/git/cypress-example-todomvc/cypress/integration/app_spec.js
  took 22ms +24ms
  cypress-esbuild-preprocessor bundling /Users/gleb/git/cypress-example-todomvc/cypress/support/index.js
  took 104ms +104ms

I ran the test three times, and here are the timings

Support file (ms) Spec file (ms)
104 22
104 22
101 22

That's pretty strong statement: the ESBuild preprocessor is 17 times faster in bundling the support file and 50 (fifty!) times faster in bundling the spec file.

I am pretty impressed.