products

Run Cypress Tests For The Given URL

Use `cypress-visited-urls` plugin to find end-to-end tests that visit a specific url.

Imagine you are testing an online e-commerce store. You modified something on the product page /item/:id. What end-to-end tests should you execute to verify your change? You might find the obviously named spec item-page.cy.js. Is that the only spec file that checks something on the /item/:id page? What about the checkout flow? It starts at the homepage, find an item, and buys it. Of course, it visits the item's page! You probably want to run that test too.

Previously I dealt with this situation by running tests tagged @item, hoping that all major features of the item's page would be tested. For good measure, I would also run smoke tests using tag @sanity. I hoped that such combination of "feature X + sanity tests across all features" is enough. Hint: read how I use test tags in the blog post How To Tag And Run End-to-End Tests and use my plugin @bahmutov/cy-grep.

Today I want to show a much more direct way of picking tests when you know the page URL: by using the previously collected visited urls and my new plugin cypress-visited-urls.

Let's take an example project from my online course Testing The Swag Store. It is an online store with multiple different pages: inventory, item page, checkout screens. There are lots of specs covering different areas of the store (the orange rectangle in the image below).

The specs and the Swag Store app

I have configured the cypress-visited-urls plugin to save all visited URLs into a JSON file. You just need to include the plugin from the E2E support file, and load the plugin from the E2E `` callback.

cypress/support/e2e.js
1
2
3
4
// e2e support file
// https://glebbahmutov.com/blog/collect-tested-urls/
import { configureVisitedUrls } from 'cypress-visited-urls'
configureVisitedUrls()
cypress.config.mjs
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
// https://github.com/bahmutov/cypress-visited-urls
import visitedUrlsPlugin from 'cypress-visited-urls/src/plugin'
export default defineConfig({
  e2e: {
    env: {
      visitedUrls: {
        // collect each URL the test runner visits
        // https://glebbahmutov.com/blog/collect-tested-urls/
        collect: true,
        urlsFilename: 'cypress-visited-urls.json',
      },
    }
    setupNodeEvents(cypressOn, config) {
      visitedUrlsPlugin(on, config)
      // IMPORTANT to return the config object
      // with the any changed environment variables
      return config
    }
  },
})

Note: since the component specs do not visit meaningful URLs, we only configure this plugin in the E2E tests.

We can run a few specs to see the collected URLs

cypress-visited-urls.json
1
2
3
4
5
6
7
8
9
10
11
12
13
14
{
  "cypress/e2e/cart/add-to-cart.cy.ts": {
    "Cart / adds items to the cart": [
      "/inventory"
    ]
  },
  "cypress/e2e/cart/cart-app-action.cy.ts": {
    "Cart / can be controlled via application methods": [
      "/",
      "/inventory",
      "/cart"
    ]
  }
}

The file is organized by alphabetically sorted spec name. Each object has the full test titles (formed from the suite names plus the test title). The values are all relative URLs visited by the test.

1
2
3
4
5
6
7
8
9
{
  "spec A": {
    "test 1": ["/", "/path"],
    "test 2": ["/", "/path"]
  },
  "spec B": {
    ...
  }
}

We can collect all visited URLs by running the tests locally and committing the generated or update JSON file. Or we can run the tests once on CI and push the changed file back to the source control. For example, using GitHub Actions:

1
2
3
4
5
6
7
8
9
10
- name: Run all tests
  uses: cypress-io/github-action@v6

- name: commit updated urls file
  # https://github.com/stefanzweifel/git-auto-commit-action
  uses: stefanzweifel/git-auto-commit-action@v5
  with:
    commit_message: Updated urls file
    branch: main
    file_pattern: cypress-visited-urls.json

Note: updating the urls file from a CI run using multiple parallel machines is harder, since each file will have just a part of of all URLs file.

Preprocess URLs

Sometimes the visited url includes random parts, like the item's id in this test

The full URL includes the query parameter with the item's id

The saved visited URLs file shows the literal value

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
{
  "cypress/e2e/cart/cart.cy.ts": {
    "Cart / has id and quantity data- attributes": [
      "/inventory",
      "/cart"
    ],
    "Cart / navigates to the item from the cart": [
      "/inventory",
      "/cart",
      "/inventory-item?id=3"
    ],
    "Cart / shows the added items in order they were added": [
      "/",
      "/inventory",
      "/cart"
    ]
  }
}

For our purposes, every page like inventory-item?id=3 or inventory-item?id=101 is the same. Thus we can preprocess an URL before saving it. We can provide a callback function that can for example replace every numerical id with a predefined value.

cypress/support/e2e.js
1
2
3
4
5
6
7
8
9
// https://glebbahmutov.com/blog/collect-tested-urls/
import { configureVisitedUrls } from 'cypress-visited-urls'

configureVisitedUrls({
  filterUrl(url: string) {
    // remove the item id from the search parameters
    return url.replace(/id=\d+/, 'id=:id')
  },
})

Run the same test again to see

1
2
3
4
5
"Cart / navigates to the item from the cart": [
  "/inventory",
  "/cart",
  "/inventory-item?id=:id"
],

Nice.

Find specs by URL

Finally, let's use the collected URLs to find specs that cover them. For example, a few tests go through the checkout flow. They visit urls /checkout-step-one and /checkout-step-two. Let's see all specs that reach the second step. The plugin cypress-visited-urls includes a bin script that can list all specs with a test that matches part of the URL.

1
2
3
4
$ npx find-specs-by-url --filename cypress-visited-urls.json --url /checkout-step-two
cypress/e2e/checkout/checkout.cy.ts,cypress/e2e/checkout/glitch-user.cy.ts,
cypress/e2e/checkout/tax.cy.ts,cypress/e2e/checkout/total.cy.ts,
cypress/e2e/problem-user/checkout-step-two.cy.ts

The output is a single line with comma-separated spec filenames. We can simply pass them to the Cypress run command, assuming the application is running already

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
$ npx cypress run --spec \
  $(npx find-specs-by-url --filename cypress-visited-urls.json --url /checkout-step-two)

====================================================================================================

  (Run Finished)


       Spec                                              Tests  Passing  Failing  Pending  Skipped
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ ✔  checkout/checkout.cy.ts                  00:04        3        3        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  checkout/glitch-user.cy.ts               00:12        1        1        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  checkout/tax.cy.ts                       00:02        1        1        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  checkout/total.cy.ts                     00:03        2        2        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  problem-user/checkout-step-two.cy.t      00:02        2        2        -        -        - │
  │    s                                                                                           │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
    ✔  All specs passed!                        00:24        9        9        -        -        -

We ran just the 5 found specs.