Type Check Your Test Tags

Cypress test runner does not have built-in test title grep or test tags. Thus you need to use my @bahmutov/cy-grep plugin. I use it myself all the time and keep working on it. One question that comes up again and again is how to "check" test tags against invalid values. For example, it is super simple to enter a wrong test tag:

it('shows the user homepage', { tags: '@users' }, () => {
  ...
})

Oops, the tag "@users" it wrong. All other tests in this project use the tag "@user" (singular). Your test "shows the user homepage" is sitting by itself and not running when you trigger the test run for all tests tagged "@user"... How do you know in your code editor that the tag "@users" is not valid?

You use type checking via TypeScript. Install the @bahmutov/cy-grep v2 plugin (v1 only defines test tags to be a string), and define what the valid tags can be. You can create a file "cypress/support/index.d.ts" for this.

cypress/support/index.d.ts

/// <reference types="cypress" />

/**
 * The only allowed test tags in this project
 */
type AllowedTag = '@smoke' | '@user' | '@login'
type TestTags = AllowedTag | AllowedTag[]

declare namespace Cypress {
  interface SuiteConfigOverrides {
    tags?: TestTags
    requiredTags?: TestTags
  }

  interface TestConfigOverrides {
    tags?: TestTags
    requiredTags?: TestTags
  }
}

We need to define tags and requiredTags parameters for both individual tests and test suites functions it and describe

// SuiteConfigOverrides
suite('user', { tags: '@user' }, () => {
  // TestConfigOverrides
  it('loads', { tags: '...' }, () => {
    ...
  })
})

🎓 All code examples in this blog post are taken from lessons from my online course Cypress Plugins which has 12 lessons right now about @bahmutov/cy-grep plugin.

Your code editor reads this "index.d.ts" file and "knows" about the global ambient types like Cypress test config object. It now knows that the code it(..., { tags: ... }, () => { ... }) can only have in the tags property three possible values: @smoke, @users, or @login. When you try using test tag @users the code editor and the TS check complain:

Beautiful, we get static code analysis that helps us select only valid test tags.

Runtime check

We got the static types check working, what about runtime? If the user passes a list of test tags to run, can we check which string values are valid test tags in our project? We need the test tags to be a type and a value. Thanks to my coworker Dominik Kratz, this is easy.

Let's define an array of valid test tags. It is a string array and can be used to validate runtime arguments. It is a constant array, and thus we can define a type: only the key values to that array are allowed. To be able to import the TestTags value, we should put it into a separate source file away from the ambient types defined in the index.d.ts file.

cypress/support/test-tags.ts

// we define a constant array with the only valid test tags for this project
export const TestTags = ['@user', '@misc', '@new-todo'] as const
// and form the type from this test tags array
export type TestTag = (typeof TestTags)[number]

We can import the TestTag type into the ambient .d.ts file

cypress/support/index.d.ts

/// <reference types="cypress" />

declare namespace Cypress {
  type TestTag = import('./test-tags').TestTag

  interface SuiteConfigOverrides {
    tags?: TestTag | TestTag[]
    requiredTags?: TestTag | TestTag[]
  }

  interface TestConfigOverrides {
    tags?: TestTag | TestTag[]
    requiredTags?: TestTag | TestTag[]
  }
}

TypeScript check uses the ambient definitions from the index.d.ts file. Other files can import the TestTags array and use its values. For example, we can validate the command line tags passed by the user

cypress.config.js

const { defineConfig } = require('cypress')
// import th array "TestTags" from 'cypress/support/test-tags'
// valid tags should be an array of strings, like "@smoke", "@new-todo", etc
const { TestTags } = require('./cypress/support/test-tags')

module.exports = defineConfig({
  e2e: {
    setupNodeEvents(on, config) {
      if (config.env.grepTags) {
        console.log('checking the test tags "%s"', config.env.grepTags)
        // split the tags by comma, trim each tag
        // and filter out invalid tags using the "ValidTestTags" array
        const split = config.env.grepTags
          .split(',')
          .map((tag) => tag.trim())
          .filter((tag) => TestTags.includes(tag))
        // print the remaining validated tags
        console.log('valid tags "%s"', split.join(','))
        // and set the environment variable "config.env.grepTags"
        // to the validated tags string (comma separated)
        config.env.grepTags = split.join(',')
      }

      // https://github.com/bahmutov/cy-grep
      require('@bahmutov/cy-grep/src/plugin')(config)

      // IMPORTANT:
      // make sure to return the config object
      // as it might have been modified by the plugin
      return config
    },
  },
})

Let's see this in action. We can pass several test tags, and only the valid test tags will be used

$ CYPRESS_grepTags=@foo,@smoke npx cypress run

checking the test tags "@foo,@smoke"
valid tags "@smoke"
cy-grep: filtering using tag(s) "@smoke"
cy-grep: will omit filtered tests

Really nice! Install @bahmutov/cy-grep v2 and give it a spin.