products

Writing Cypress Recurse Function

A collection of videos about writing cypress-recurse NPM module

cypress-recurse is a small utility for re-running Cypress commands until a predicate function passes. This blog post collects several videos I have recorded explaining how to code cypress-recurse from zero all the way to publishing the NPM module and using it in other tests.

The problem

Recently a user asked how the test can do the following: reload the page until it shows the number 7. The page shows a random digit on every load, so we need to run cy.reload() until the page shows it. The final solution would look like this:

Reloading the page from the test until we get the number 7

The user tried to implement the test using a JavaScript while loop which only crashes the browser window

1
2
3
4
5
6
7
8
9
10
11
// ⛔️ DOES NOT WORK, crashes the browser
it('yields 7 from the task (crashes)', () => {
  let found
  while (!found) {
    cy.task('randomNumber').then((n) => {
      if (n === 7) {
        found = true
      }
    })
  }
})

In the next several videos I show the correct way of implementing the test and progressively refactor it to be a reusable function. You can read a short description of every step and watch the embedded videos from the YouTube playlist. The final solution is available on NPM as cypress-recurse module.

Recursively call cy.task from the test

First, let's figure out what goes wrong with the while loop, and how a test can avoid the runaway loop by calling an intermediate function recursively. The video below does that and then derives the correct solution.

The final solution uses the checkNumber function - it calls the Cypress command, checks the yielded value. If the value is not work we need, it calls itself recursively.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
it('yields 7 from the task', () => {
  const checkNumber = () => {
    cy.task('randomNumber').then((n) => {
      cy.log(`**${n}**`)
      if (n === 7) {
        cy.log('**NICE!**')
        return
      }
      checkNumber()
    })
  }

  checkNumber()
})

Reusable function

It makes sense to factor out the solution and make it usable from other tests.

The final result is a reusable little function that can call the given two functions: one is a chain of Cypress commands, another one is a predicate function checking the result to know when to stop.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
it('yields 7 from the task (recursive)', () => {
  const recurse = (commandsFn, checkFn) => {
    commandsFn().then((x) => {
      if (checkFn(x)) {
        cy.log('**NICE!**')
        return
      }

      recurse(commandsFn, checkFn)
    })
  }

  recurse(
    () => cy.task('randomNumber'),
    (n) => n === 7,
  )
})

Iteration limit

If our predicate never return true, we can execute the Cypress commands until the end of times. We better limit the maximum number of attempts. The next video adds the maximum iteration limit to the function.

The solution uses only local variables without global state

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
it('yields 7 from the task (limit)', () => {
  const recurse = (commandsFn, checkFn, limit = 3) => {
    if (limit < 0) {
      throw new Error('Recursion limit reached')
    }
    cy.log(`remaining attempts **${limit}**`)

    commandsFn().then((x) => {
      if (checkFn(x)) {
        cy.log('**NICE!**')
        return
      }

      recurse(commandsFn, checkFn, limit - 1)
    })
  }

  recurse(
    () => cy.task('randomNumber'),
    (n) => n === 7,
    30,
  )
})

Timeout

Most Cypress commands use a time limit, not an attempt counter limit. Thus the next improvement we are making is adding a time limit to the recurse function.

Again, we are using arguments and avoid global state

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
it('yields 7 from the task (time limit)', () => {
  const recurse = (commandsFn, checkFn, timeRemaining = 4000) => {
    const started = +new Date()
    if (timeRemaining < 0) {
      throw new Error('Max time limit reached')
    }
    cy.log(`time remaining **${timeRemaining}**`)

    commandsFn().then((x) => {
      if (checkFn(x)) {
        cy.log('**NICE!**')
        return
      }

      const finished = +new Date()
      const elapsed = finished - started
      recurse(commandsFn, checkFn, timeRemaining - elapsed)
    })
  }

  recurse(
    () => cy.task('randomNumber'),
    (n) => n === 7,
    10000,
  )
})

Options object

We want to use both the iteration and time limit and also add "log" flag to enable logging. Using positional arguments quickly becomes hard, thus we switch to using an options object.

Here is the options pattern, including applying the defaults

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
it('yields 7 from the task (options object)', () => {
  const recurse = (commandsFn, checkFn, options = {}) => {
    Cypress._.defaults(options, {
      limit: 30,
      timeRemaining: 30000,
      log: true,
    })
    const started = +new Date()

    if (options.limit < 0) {
      throw new Error('Recursion limit reached')
    }
    if (options.log) {
      cy.log(`remaining attempts **${options.limit}**`)
    }

    if (options.timeRemaining < 0) {
      throw new Error('Max time limit reached')
    }
    if (options.log) {
      cy.log(`time remaining **${options.timeRemaining}**`)
    }

    commandsFn().then((x) => {
      if (checkFn(x)) {
        if (options.log) {
          cy.log('**NICE!**')
        }
        return
      }

      const finished = +new Date()
      const elapsed = finished - started
      recurse(commandsFn, checkFn, {
        timeRemaining: options.timeRemaining - elapsed,
        limit: options.limit - 1,
        log: options.log,
      })
    })
  }

  recurse(
    () => cy.task('randomNumber', null, { log: false }),
    (n) => n === 7,
    { timeRemaining: 10000, log: true },
  )
})

The user can omit the options, or specify just some of them.

Adding types via JSDoc comments

We want to make the reusable function recurse ... easier to reuse. Having the type information for the function's signature is a big help to the users, but I do not want to write TypeScript. Thus I take a middle road by adding JSDoc comments describing the types of the arguments. Modern code editors like VSCode can read and use JSDoc type comments to show IntelliSense.

Here are these JSDoc comments

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
/**
  * @typedef {object} RecurseOptions
  * @property {number=} limit The max number of iterations
  * @property {number=} timeRemaining In milliseconds
  * @property {boolean=} log Log to Command Log
  */
/**
  * Recursively calls the given command until the predicate is true.
  * @param {() => Cypress.Chainable} commandsFn
  * @param {(any) => boolean} checkFn
  * @param {RecurseOptions} options
  */
function recurse(commandsFn, checkFn, options = {}) {
  Cypress._.defaults(options, {
    limit: 30,
    timeRemaining: 30000,
    log: true,
  })
  ...
}

Published NPM package

Our reusable function is ready - we can now move out from a Cypress test and into its own repository bahmutov/cypress-recurse and publish it on NPM.

We definitely need a CI setup and automated dependency updates to make sure our NPM package is working with the newer versions of Cypress.

cypress-recurse README file

Use case 1: find the downloaded file

Now that we have cypress-recurse, let's use it. Calling cy.task repeatedly to find a file is a good use case. Here is an example from our Cypress example recipes where we try to find a downloaded file

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
import { recurse } from 'cypress-recurse'
const isNonEmptyString = (x) => {
  return typeof x === 'string' && Boolean(x)
}

it('using recurse', { browser: '!firefox' }, () => {
  // imagine we do not know the exact filename after download
  // so let's call a task to find the file on disk before verifying it
  // image comes from the same domain as the page
  cy.visit('/')
  cy.get('[data-cy=download-png]').click()

  cy.log('**find the image**')
  const mask = `${downloadsFolder}/*.png`

  recurse(
    () => cy.task('findFiles', mask),
    isNonEmptyString
  )
  .then((foundImage) => {
    cy.log(`found image ${foundImage}`)
    cy.log('**confirm downloaded image**')
    validateImage()
  })
})

Use case 2: visual testing retries

In the blog post Canvas Visual Testing with Retries I show how to use the cypress-recurse to download and compare HTML5 canvas to a previously saved image. The code is using recurse

1
2
3
4
5
6
7
8
9
10
11
12
13
it('smiles broadly', () => {
  cy.visit('/smile')

  recurse(
    () => {
      return downloadPng('smile.png').then((filename) => {
        cy.log(`saved ${filename}`)
        return cy.task('compare', { filename })
      })
    },
    ({ match }) => match,
  )
})

You can see how I am writing the above test in this video and find the full project at bahmutov/monalego

recurse vs test retries

Cypress already includes test retries. If a test fails, you can re-run the entire test up to N times, called attempts. The test retries could be configured globally, per suite, or per individual test:

1
2
3
4
// this test can fail, then retry up to 2 times more
it('retries if fails', { retries: 2 }, () => {
  ...
})

There are differences between the test retries and cypress-recurse:

  • test retries only apply to the body of the test, they do not retry the test hooks like before and beforeEach
  • test retries re-run the entire test, while recurse only retries the given commands.

In general I would consider the recurse a "normal" part of running tests, similar to the command retry-ability, but extended to an arbitrary number of commands. Test retries are a mechanism to fight the truly unpredictable test flake, like slow responses from the server.