Required Tags

I love writing end-to-end tests. In my repos, user flows are typically tested using Cypress. I also use Cypress to perform other automated tasks, like scraping my blog posts into a search index, cleaning up old test data, and downloading the CMS backup data for my courses. A typical repo is like bahmutov/required-tags-example with its specs:

cypress/e2e/clean.cy.js

describe('clean', () => {
  it('old data', () => {
    // pretend we are deleting old test data
    ...
  })
})

cypress/e2e/scrape.cy.js

describe('scrape', () => {
  it('scrapes the blog', () => {
    // pretend we are scraping the blog posts
    ...
  })
})

cypress/e2e/spec-a.cy.js

describe('spec A', () => {
  it('works', () => {})

  it('works some more', () => {})
})

cypress/e2e/spec-b.cy.js

describe('spec B', () => {
  it('works', () => {})

  it('works some more', () => {})
})

Every time we push a new commit, we run through all end-to-end specs above. I am using GitHub Actions to run the tests via my reusable Cypress workflows:

.github/workflows/ci.yml

name: ci
on: [push]
jobs:
  test:
    # use the reusable workflow to check out the code, install dependencies
    # and run the Cypress tests
    # https://github.com/bahmutov/cypress-workflows
    uses: bahmutov/cypress-workflows/.github/workflows/standard.yml@v1

The problem

Hmm, every commit scrapes the data and cleans the old test data. So in a typical run the spec-a and spec-b finish under 200ms each, while clean.cy.js and scrape.cy.js take 5 second each.

Ughh, we don't want to run clean and scrape tests at all by default. We only want to run them once a day maybe at night. Typically, we could ignore those spec files and set up a separate Cypress test command to change the config.

cypress.config.js

const { defineConfig } = require('cypress')

module.exports = defineConfig({
  e2e: {
    // baseUrl, etc
    excludeSpecPattern: ['**/clean.cy.js', '**/scrape.cy.js'],
  }
})

By default, the above config hides the clean and scrape specs. If we want to enable them, we can pass the excludeSpecPattern via CLI when running Cypress. For example, if we want to run the clean spec only:

$ npx cypress run --spec cypress/e2e/clean.cy.js --config excludeSpecPattern='whatever'

Note: you have to pass some config value excludeSpecPattern='whatever'. You cannot use undefined or an empty string to "clear" it, at least not in Cypress v12.3.0

Ughh, there must be a better way.

cy-grep and required tags

There is a better way. Recently I have added to my @bahmutov/cy-grep plugin a feature specifically to make running such "tests" simple by default. I call it required tags and here is the background. Using the @bahmutov/cy-grep plugin you can tag tests and suites to filter them out.

Let's say we tag one of the tests in spec-a.cy.js with a tag @smoke

cypress/e2e/spec-a.cy.js

describe('spec A', () => {
  it('works', () => {})

  it('works some more', { tags: '@smoke' }, () => {})
})

By default, nothing has changed. When we execute npx cypress run all specs and all tests run. But now we can grep all our tests and execute only the tests tagged @smoke

$ npx cypress run --env grepTags=@smoke

The plugin filters out the specs and then skips all the tests BUT the one I want:

This was the test tag. Now let's introduce a required test tag.

cypress/e2e/clean.cy.js

describe('clean', () => {
  it('old data', { requiredTags: '@nightly' }, () => {
    // pretend we are deleting old test data
    ...
  })
})

cypress/e2e/scrape.cy.js

describe('scrape', () => {
  it('scrapes the blog', { requiredTags: '@scrape' }, () => {
    // pretend we are scraping the blog posts
    cy.wait(5000)
  })
})

Let's say we try to run the clean.cy.js spec. We use npx cypress run --spec cypress/e2e/clean.cy.js command and get:

The tests tagged with a required tag only execute when we specifically pass the grepTags=... including that tag. Let's try again:

$ npx cypress run --spec cypress/e2e/clean.cy.js --env grepTags=@nightly

We can even omit the --spec cypress/e2e/clean.cy.js and the result will be the same; all specs without that tag present will be filtered out by the cy-grep plugin

Nice.

Workflows

So now our CI config and workflow are simple: the tests tagged with requiredTags are skipped for each "standard" push. Instead we prepare 2 separate workflows to scrape and clean old data once a day.

.github/workflows/nightly.yml

name: nightly
on:
  # run these tests either manually or on schedule
  workflow_dispatch:
  schedule:
    # run these tests every day
    - cron: '0 23 * * *'
jobs:
  test:
    # use the reusable workflow to check out the code, install dependencies
    # and run the Cypress tests
    # https://github.com/bahmutov/cypress-workflows
    uses: bahmutov/cypress-workflows/.github/workflows/standard.yml@v1
    with:
      env: grepTags=@nightly

.github/workflows/scrape.yml

name: scrape
on:
  # run these tests either manually or on schedule
  workflow_dispatch:
  schedule:
    # run these tests every day
    - cron: '0 23 * * *'
jobs:
  test:
    # use the reusable workflow to check out the code, install dependencies
    # and run the Cypress tests
    # https://github.com/bahmutov/cypress-workflows
    uses: bahmutov/cypress-workflows/.github/workflows/standard.yml@v1
    with:
      env: grepTags=@scrape

Typical CI run on pushed commit:

Triggered Nightly workflow only runs the clean.cy.js spec

Beautiful.

The interactive mode

If you are opening a spec with only the required tests, the cy-grep eliminates all of them. You will see "no tests found" error message.

You could pass the grep tags when opening Cypress:

$ npx cypress open --env grepTags=@nightly

Even simpler: you can set the grep tags from the browser's DevTools using the Cypress.grep(null, '@nightly') method call:

Powerful.

🎁 You can find the source code for this blog post in the repo bahmutov/required-tags-example.