products

Readable Cypress.io tests

How to write readable tests using custom commands and custom Chai assertions.

The tests should be simple to read and understand. The tests are already a layer on top the production code that is complex, so they should not add their own quirks or gotchas. In this blog post I will show how to make the tests express their meaning by adding custom Cypress commands, and how to make assertions really simple to understand by extending the default Chai assertions.

Note: these examples come from the repository bahmutov/todo-api-with-json-schema.

The example

In my example application, the production code and the tests are using json-schemas. These schemas are created and validated using @cypress-io/schema-tools library. See these resources to learn why we are using JSON schemas

I have created a schema for request and response to the API that validates Todo objects the web application is saving and loading from the server. Here the schemas/post-todo-request.ts file that describes what the server expects to receive.

schemas/post-todo-request.ts
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
import { ObjectSchema, versionSchemas } from '@cypress/schema-tools'
import { formats } from '../formats'

type uuid = string

/**
 * Todo item sent by the client.
 */
type PostTodoRequestExample100 = {
  text: string
  done: boolean
  uuid: uuid
}

const postTodoExample100: PostTodoRequestExample100 = {
  text: 'do something',
  done: false,
  uuid: '20514af9-2a2a-4712-9c1e-0510c288c9ec',
}

const PostTodoRequest100: ObjectSchema = {
  version: {
    major: 1,
    minor: 0,
    patch: 0,
  },
  schema: {
    title: 'PostTodoRequest',
    type: 'object',
    description: 'Todo item sent by the client',
    properties: {
      text: {
        type: 'string',
        description: 'Todo text, like "clean room"',
      },
      done: {
        type: 'boolean',
        description: 'Is this todo item completed?',
      },
      uuid: {
        type: 'string',
        format: formats.uuid.name, // "uuid"
        description: 'item random GUID',
      },
    },
    // require all properties
    required: true,
    // do not allow any extra properties
    additionalProperties: false,
  },
  example: postTodoExample100,
}

export const PostTodoRequest = versionSchemas(PostTodoRequest100)

Given an object, we can assert that it follows the above schema in a Jest test like this one:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import { assertSchema } from '@cypress/schema-tools'
import { schemas } from '../schemas'

describe('POST /todo request', () => {
  const assertTodoRequest = assertSchema(schemas)('postTodoRequest', '1.0.0')

  it('valid TODO request object', () => {
    const todo = {
      text: 'use scheams',
      done: true,
      uuid: '4899e1a9-e38f-43f9-a765-35b81a41c65d',
    }
    // all good, the object is passing schema validation
    expect(() => {
      assertTodoRequest(todo)
    }).not.toThrow()
  })
})

But any object NOT following the schema will raise an error with really good explanation message.

1
2
3
4
5
6
7
8
9
it('TODO request object missing text', () => {
  const todo = {
    done: true,
    uuid: '4899e1a9-e38f-43f9-a765-35b81a41c65d',
  }
  expect(() => {
    assertTodoRequest(todo)
  }).toThrowErrorMatchingSnapshot()
})

The snapshot in this case is

__tests__/__snapshots__/post-todo-request-test.ts.snap
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
// Jest Snapshot v1, https://goo.gl/fbAQLP

exports[`POST /todo request TODO request object missing text 1`] = `
"Schema [email protected] violated

Errors:
data.text is required

Current object:
{
  \\"done\\": true,
  \\"uuid\\": \\"4899e1a9-e38f-43f9-a765-35b81a41c65d\\"
}

Expected object like this:
{
  \\"done\\": false,
  \\"text\\": \\"do something\\",
  \\"uuid\\": \\"20514af9-2a2a-4712-9c1e-0510c288c9ec\\"
}"
`;

Nice, the errors are readable, but what about our end-to-end tests?

End-to-end tests

When we have a web application making requests to the server API, our end-to-end tests should validate two main things:

  • fixture files Cypress can use to mock complex network calls. We really want the fixture files to be valid with respect to the schemas used.
  • network requests and responses should confirm to the schema we expect

Validating fixtures

We can validate a fixture file using the methods provided by the @cypress-io/schema-tools function. For example in cypress/integration/fixture-spec.js we can import the api object created by the schema utilities and use api.assertSchema to validate a loaded object.

cypress/integration/fixture-spec.js
1
2
3
4
5
import { api } from '../../dist/schemas'

it('has todo fixture matching schema', () => {
  cy.fixture('todo').then(api.assertSchema('PostTodoRequest', '1.0.0'))
})

This is ok, but it is less than readable. Luckily we can make it better by writing a Cypress custom command. I will write a custom command in cypress/support/commands.js:

cypress/support/commands.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
/// <reference types="cypress" />
import { api } from '../../dist/schemas'

Cypress.Commands.add(
  'fixtureSchema',
  (fixtureName, schemaName, schemaVersion) => {
    // verify input arguments to prevent silly mistakes
    expect(fixtureName, 'fixture name').to.be.a('string')
    expect(schemaName, 'schema name').to.be.a('string')
    expect(schemaVersion, 'schema version').to.match(/^\d+\.\d+\.\d+$/)

    // load and verify the fixture itself
    cy.fixture(fixtureName, { log: false }).then(
      api.assertSchema(schemaName, schemaVersion)
    )
  }
)

and will include this commands.js file from the cypress/support/index.js file

cypress/support/index.js
1
import './commands'

Super, our fixture validation becomes readable:

1
2
3
4
5
6
it('loads and asserts todo schema', () => {
  // uses a custom command we have added in cypress/support/commands.js
  cy.fixtureSchema('todo', 'PostTodoRequest', '1.0.0')
    // you can chain commands to the loaded fixture
    .should('have.property', 'text', 'use fixtures')
})

The combined command cy.fixtureSchema lessens the overhead for anyone reading the spec code, leaving more mental capacity to actually think about the application and its logic.

If an object loaded from a fixture file does not match the schema, a good error is displayed in the Cypress GUI.

Object loaded from the fixture file does not match the schema

Great.

Custom assertions

In addition to the custom commands, we can also extend the Chai assertions with our own predicates. In our repository I will add file cypress/support/assertions.js and will import this file from the cypress/support/index.js file.

cypress/support/index.js
1
2
import './commands'
import './assertions'

Let's add a custom Chai assertion that validates an object against a schema.

cypress/support/assertions.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
/// <reference types="cypress" />
import { api } from '../../dist/schemas'

// how to add a custom Chai assertion to Cypress
// see "Adding Chai Assertions" recipe in
// https://github.com/cypress-io/cypress-example-recipes

const isFollowingSchema = (_chai, utils) => {
  function assertFollowingSchema (schemaName, schemaVersion) {
    // if the subject does not the schema, we will
    // get a very nice error message from "api.assertSchema"
    api.assertSchema(schemaName, schemaVersion)(this._obj)

    // but if assertion passes, we should print passing assertion
    // message which we can do using Chai
    this.assert(
      true,
      `expected subject to follow schema **${schemaName}@${schemaVersion}**`
    )
  }

  _chai.Assertion.addMethod('followSchema', assertFollowingSchema)
}
chai.use(isFollowingSchema)

Notice there are 2 assertions in the isFollowingSchema callback: the "real" one and an always-passing one this.assert(true, <message>). This is a little trick I use to have a detailed error message if an assertion fails, and a short success message when the object conforms to the schema. Here is an example test validating the response body.

1
2
3
4
5
6
7
8
9
10
11
12
it('returns new item matching schema', () => {
  cy.server()
  cy.route('POST', '/todos').as('post')
  cy.visit('/')
  cy.get('.new-todo').type('Use schemas{enter}')

  // check response passes schema
  cy.wait('@post')
    .its('response.body')
    // use custom Chai assertion
    .should('followSchema', 'PostTodoResponse', '1.0.0')
})

When the test passes, the Command Log is showing the summary thanks to that "dummy" this.assert(true, ...) assertion trick.

Returned object passes schema message

Adding IntelliSense

Cypress commands and assertions come with full JSDoc comments, which allows a modern text editor like VSCode to show help during test writing. For example when hovering over cy.should this tooltip pops up:

`cy.should` documentation

Note that the help box shows help for cy.should WITH have.property assertion! Thus we should be able to specify JSDoc for both our custom commands and for custom assertions. To do this, create a new file cypress/support/index.d.ts. It is a TypeScript file that will describe additional cy commands and assertions. The new commands are added to the Chainable<Subject> interface, and the new assertions are added to the Chainer<Subject> interface. For each command and assertion I will write a detailed JSDoc command with examples.

cypress/support/index.d.ts
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
/// <reference types="cypress" />

/**
 * Semver string, usually simple like "major.minor.patch"
 * @example
 * const version: semverString = '2.1.0'
*/
type semverString = string

declare namespace Cypress {
  interface Chainable<Subject> {
    /**
     * Load a fixture JSON and check it against a schema.
     *
     * @example
     * cy.fixtureSchema('single-todo', 'Todo', '1.1.0')
     */
    fixtureSchema(fixturePath: string,
      schemaName: string, schemaVersion: semverString): Chainable<any>
  }

  interface Chainer<Subject> {
    /**
     * Custom Chai assertion that checks if the given subject follows
     * a schema
     *
     * @example
    cy.wrap({ ... })
      .should('followSchema', 'mySchemaName', '2.1.0')
    cy.fixture('filename')
      .should('followSchema', 'PostTodoRequest', '1.0.0')
    cy.wait('@networkCallAlias')
      .its('response.body')
      .should('followSchema', 'PostTodoResponse', '1.0.0')
    */
    (chainer: 'followSchema',
      schemaName: string, schemaVersion: string): Chainable<Subject>
  }
}

Special note: my blog's syntax highlighting does not parse the original JSDoc for the above assertion correctly. Here is a screenshot of how it SHOULD be to avoid VSCode parsing to break on @networkCallAlias string and to preserve indentation:

Use tripple slashes in JSdoc to preserve example indent

I write my code examples using triple back ticks like that to preserve indentation and avoid problems caused by @ character in the network alias.

Once we have this support index.d.ts file in place, we can include it from our JavaScript spec files (and it can replace the regular triple-slash special command importing cypress types).

1
2
3
4
5
6
7
// in a JavaScript spec file, instead of this:
/// <reference types="cypress" />

// import your own support .d.ts file
// which will import Cypress in turn
// and will add your custom definitions
/// <reference path="../support/index.d.ts" />

Hover over cy.fixtureSchema and see documentation from the .d.ts file

IntelliSense for custom command

Typing a custom assertion should('followSchema', ...) brings the following help popup

IntelliSense for custom assertion

In all cases, good documentation is essential.

Conclusions

Making test code as readable as possible is a worthy goal. Each test command and assertion can express its intent, and custom commands and assertions help achieve this.

I would suggest adding custom commands and assertions to their own files in cypress/support folder and including them from cypress/support/index.js file

cypress/support/index.js
1
2
import './commands'
import './assertions'

I would also write a TypeScript definition file like cypress/support/index.d.ts that describes the new commands and assertions and is loaded by IntelliSense to provide pop-up help during editing.