A long time ago I worked at Cypress.io and we opened sourced the test runner itself. If the test runner
cypress is free, how would the company make money? We needed a feature for our Cypress Dashboard that would be easy to justify business to pay for. Recording test artifacts like screenshots and videos? Ughh, yeah, but it is easy to store those static test reports, so it was not worth much.
I was playing with Mocha test runner at that time, creating its variants like Rocha (randomize the order of tests), Focha (run the previously failed tests first), and Locha (run the failed tests again with extra log verbosity). When running a lot of tests, the start-up time was annoying, and I thought for myself:
Why can't my test runner be already open, waiting for me to send it the spec to test?
This is how the Cypress parallelization feature was born. I modified the Cypress test runner to stay open after running a spec, and ask the Dashboard API to send it the next spec to run. Thus several test runners could split the entire list of specs, each test runner only running some of the specs. Then I put the paid Dashboard up and Cypress the company started selling paid plans like hot cupcakes. Who could deny that speeding up CI tests N times (where N is the number CI machines) is not worth it?!!
Then I created Cypress CircleCI Orb and Cypress GitHub Action to make the actual mechanics of installing, caching, and running Cypress on each CI simpler to configure. Here is how you could run your tests across 4 CircleCI machines:
The same example using GitHub Actions:
name: Parallel Cypress Tests
Tip: if you are running Cypress using GitHub Actions, you can simplify it using my reusable Cypress workflows; the same example above can be written using just:
Now that I no longer work at Cypress.io the company and don't care about their business success, I had a chance to sit down and think how to run Cypress tests in parallel on CI without paying for 3rd party service.
You deserve your Cypress tests to run very quickly FOR FREE.
I have come up with cypress-split: a tiny Cypress plugin that lets you automatically split the entire list of Cypress specs to run in parallel on any CI(or even when running Cypress locally using Docker containers). Install the plugin following its README documentation and use one of the CI examples provided to run the tests on your server. For example, let's run Cypress tests on 4 CircleCI machines:
# to use orbs, must use version >= 2.1
--parallel flags, no paying for an external service. All specs split across 4 machines. Other CI providers work similarly simply. I have prepared a couple of repos with GitHub Actions, Circle, and GitLab CI working.
|Reusable GHA workflow||cypress-split-example|
🎓 I am showing how to install and configure the
cypress-splitplugin for different CI providers in my Cypress Plugins course.
Reusable GitHub Actions workflows
I have added a special reusable GitHub workflow to run using
cypress-split to my cypress-workflows. To me, this is the simplest way to run Cypress tests in parallel on GitHub Actions:
cypress-split plugin even outputs its information into GitHub Actions report:
Cypress Dashboard uses previous spec timings to queue the specs when then run in parallel. This algorithm optimizes the total execution duration to ensure the longest specs finish first. The
cypress-split plugin does not have the previous spec run information and splits the specs alphabetically. This works well, if all specs are approximately the same duration. I think I might make it work by saving historic timings, I am still playing with the plugin. If you have any ideas, let me know.
- Read the blog post 📝 Cypress Mochawesome HTML Reports