Imagine you have 100s of Cypress specs with end-to-end tests. You have modified a React component source file "Hello.jsx". Which E2E tests should you run when you open a pull request? Ideally, you would run all of them to prevent accidentally breaking some tests. Running all specs might take a while, even if you parallelize Cypress specs for free. In this post I will show a better way. We will pick specs that use the the test IDs in the changed source file "Hello.jsx". We can run these specs first and then run all specs later.
🎁 You can find the example application shown in this blog post in the repo bahmutov/taste-the-sauce-test-ids. We will use plugin bahmutov/changed-test-ids to determine which specs to run.
The application
Our application is a typical React web app. We have pages and components, and we use different test IDs to select elements on the page during testing.
1 | ... |
The above CheckOutStepOne
component has testId
attribute "lastName". When the application runs, this becomes the HTML attribute data-test="lastName"
.
The specs
We use Cypress specs to go through the application. Right now we have only a few high level specs
1 | cypress/ |
In the specs, I used two Cypress custom commands to find elements by data-test
attribute. These two commands are similar to cy.get and cy.contains commands. These commands select HTML elements on the page using the attribute or attribute plus element's text.
1 | // find element(s) by data-test attribute |
Here is a typical checkout test using these custom commands cy.getByTestId
and cy.containsTestId
1 | // part of the checkout spec |
Right now all tests are passing.
A small source file change
Let's say we open a pull request where we modify the CheckOutStepOne.jsx
source file just a bit.
1 | - placeholder="Last name" |
You can find this change in the pull request #3. Which specs should we run? Do we need to run the login-form.cy.js
and logout.cy.js
specs?
This is where the small utility changed-test-ids comes handy. I have installed it as a dev dependency
1 | $ npm i -D changed-test-ids |
We can use this utility to parse both JSX and Cypress specs to find data test attributes used in both files. For example, let's see all test IDs in the source files:
1 | $ npx find-ids --sources 'src/**/*.jsx' |
The changed-test-ids
installs NPM bin script find-ids
. The above command parses all src/**/*.jsx
source files to find all testId="..."
props and then outputs them.
What about our Cypress specs? We use custom commands cy.getByTestId
and cy.containsTestId
, so let's parse the specs looking for those commands and report all collected ids.
1 | $ npx find-ids --specs 'cypress/e2e/**/*.cy.js' \ |
So our specs matching the glob pattern cypress/e2e/**/*.cy.js
use 8 data test attributes. There are more test ids in the source files than in the specs, so some elements are not directly selected by our E2E specs. We can warn the user about this by using both --sources
and --specs
parameters and combining the above two commands:
1 | $ npx find-ids --sources 'src/**/*.jsx' \ |
Pull request workflow
Cool, let's use this test ID information to pick Cypress specs to run based on the test IDs in the changed source files in the pull request. I am using GitHub Actions, and I can extract the test IDs from the sources changed between the current branch and the main
branch.
1 | name: pr |
The step Find Cypress specs for changed source files
is crucial.
1 | - name: Find Cypress specs for changed source files |
It looks at the Git information, finds the changed source files, finds the test IDs used in those source files, and then picks only the specs that use these test IDs.
In the changed source file CheckOutStepOne.jsx
there are multiple testId
attributes: cancel
, continue
, firstName
, lastName
, etc. We have detected only one spec that covers at least some of them: checkout.cy.js
. This is the spec we should run. By using --set-gha-outputs
the output list of specs and the number of specs is saved in the GitHub Actions outputs values. We can then pass it to the cypress-io/github-action
step to run those specs only:
1 | - name: Run any detected Cypress specs |
Compared to running all specs, we saved some time, even on this tiny project :)
Beautiful.
Update 1: tests in another repo
I have described how the same plugin changed-test-ids
can be used to pass the detected test ids to pick tests to run, even if the tests reside in another repo. Read the blog post Pick Tests Using Test Ids From Another Source Repo.
Update 2: improve checking out the repo
To compute the changes between the current code and the default repository branch, we can only check out the current code and the default branch main
1 | - name: Checkout the current merge commit 🛎️ |
To compute the difference, remove the --parent
parameter
1 | - name: Find test ids used in the changed source files |
If you want to see the verbose debug logs, add DEBUG: changed-test-ids
environment variable tothe find-ids
step
1 | - name: Find test ids used in the changed source files |