products

Cypress should callback

Use any assertion inside "should(cb)" function to have Cypress auto-retry its command with your assertion function.

Note: source code for this blog post is in bahmutov/cypress-should-callback, see the spec files.

Cypress has a built-in retry-ability in almost every command - a concept that still keeps blowing my mind, and makes for a great demo during my presentations. For example, here is an application that adds elements to the page one by one

app.js
1
2
3
4
5
6
7
8
9
const app = document.getElementById('app')
const addTodo = (title) => () => {
  const el = document.createElement('div')
  el.innerText = title
  app.appendChild(el)
}
setTimeout(addTodo('first child'), 1000)
setTimeout(addTodo('second child'), 2000)
setTimeout(addTodo('third child'), 3000)

My Cypress test for this is extremely simple.

cypress/integration/spec.js
1
2
3
4
it('loads 3 elements', () => {
  cy.visit('index.html')
  cy.get('#app div').should('have.length', 3)
})

get command is retried with assertion until passes

Inside each Cypress command that does not change the state of the app (like get, find) there is a retry mechanism. The command will be executed, the result passed to the assertion that follows - and if the assertion passes, then the command completes successfully. If the assertion throws an error, the command is executed again, result passes to the assertion and so on and so on - until the assertion either passes, or the default timeout of 4 seconds ends. Here is an assertion that fails on purpose, looking for 4 items, while the application only shows 3.

1
cy.get('#app div').should('have.length', 4)

assertion fails after 4 seconds

Imagine our test only checks for 2 items - it won't wait for 3 items to appear. The test passes as soon as the second item has been added.

1
cy.get('#app div').should('have.length', 2)

as soon as two items appear, the assertion passes

There is a huge variety of assertions you can use. Cypress comes with Chai, Chai-Sinon and Chai-jQuery assertions, and you can easily bring additional assertion libraries. The best part - the Cypress assertions do come with IntelliSense, which makes writing them less of memorization and more of expressing what you want to "see" in the test.

IntelliSense in assertion, just hover over ".should"

Should callback

If built-in assertions are not enough, you can pass your own callback function with assertions. For example, what if we want to confirm the text in each of the three items that appear? We could write 3 commands with 3 assertions.

1
2
3
4
5
6
7
8
9
it('3 commands', () => {
  cy.visit('index.html')
  cy.get('#app div:nth(0)')
    .should('contain', 'first child')
  cy.get('#app div:nth(1)')
    .should('contain', 'second child')
  cy.get('#app div:nth(2)')
    .should('contain', 'third child')
})

Confirming the text in 3 items

The test works, but the selectors are complex, and I would like to have a single assertion, rather than multiple ones. If I want a complex assertion that Cypress will use to rerun the previous command until it passes or times out - I need to pass a callback function to should(cb).

1
2
3
4
5
6
7
8
9
it('loads 3 elements', () => {
  cy.visit('index.html')
  cy.get('#app div')
    .should(($div) => {
      expect($div.eq(0)).to.contain('first child')
      expect($div.eq(1)).to.contain('second child')
      expect($div.eq(2)).to.contain('third child')
    })
})

I can use any BDD and TDD style assertions inside the callback function, or even throw my own errors.

Should callback passing

Notice how Cypress understands the explicit assertions we use inside the should callback and shows them as pending. The assertions appear one by one - as first assertion passes, then the first and second assertions start running. When element <div>second child</div> appears, all 3 assertions start running, and Command Log shows them as pending. Finally, when the third item appears, all assertions are shown as passing.

Should callback is an escape hatch - a way to write very complex logic to check the state of the application's user interface or internal state.

Returned value

Note that any returned value from should(cb) is ignored - the subject passed to the next function is the original subject Cypress passed to the callback function.

1
2
3
4
5
6
7
8
9
10
11
cy.get('#app div')
  .should(($div) => {
    expect($div.eq(0)).to.contain('first child')
    expect($div.eq(1)).to.contain('second child')
    expect($div.eq(2)).to.contain('third child')

    return $div.eq(2).text() // will be ignored!
  })
  .then(($div) => {
    // $div is still the original jQuery list
  })

If you want to change the subject - do it in the commands running after the assertion. At this point you know that assertion is passing and the application has the right UI and state (unless the app changes right after passing the assertion).

1
2
3
4
5
6
7
8
9
10
11
12
it('loads 3 elements', () => {
  cy.visit('index.html')
  cy.get('#app div')
    .should(($div) => {
      expect($div.eq(0)).to.contain('first child')
      expect($div.eq(1)).to.contain('second child')
      expect($div.eq(2)).to.contain('third child')
    })
    .eq(2)
    .invoke('text')
    .should('equal', 'third child')
})

Second item text

Custom commands with retry

If you want to write a custom Cypress command that would retry an assertion that follows it, it's not difficult. The code snippet below comes from cypress-xpath module we have written as good example.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
const xpath = (selector, options = {}) => {
  // actual function that returns elements using XPath
  const getValue = () => {
    ...
  }
  const resolveValue = () => {
    // retry calling "getValue" until following assertions pass
    // or this command times out
    return Cypress.Promise.try(getValue).then(value => {
      return cy.verifyUpcomingAssertions(value, options, {
        onRetry: resolveValue,
      })
    })
  }

  return resolveValue()
}
Cypress.Commands.add('xpath', xpath)

Easier custom commands with cypress-pipe

You can even remove all boilerplate of writing custom commands by using 3rd party module cypress-pipe. For example if the function that returns elements is our custom plain function, it will be retried with our should(cb) function.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
/// <reference types="cypress" />
/// <reference types="cypress-pipe" />
it('loads 3 elements', () => {
  cy.visit('index.html')
  // instead of using Cypress ".get" command
  // write our own function to return elements
  const getElements = (doc) =>
    doc.querySelectorAll('#app div')

  cy.document()
    .pipe(getElements)
    .should((divs) => {
      // note that "getElements" returns plain NodeList
      // and not jQuery
      expect(divs[0].innerText).to.contain('first child')
      expect(divs[1].innerText).to.contain('second child')
      expect(divs[2].innerText).to.contain('third child')
    })
})

Custom function retried until custom should function passes

I love using cy.pipe command because it allows me to compose "regular" functions in place in order to create a callback function. For example in the above example we get elements and then inside the should(cb) iterate over them to get innerText property. But we can use "standard" data transformation functions from a good functional library like Ramda to extract property innerText from a given list of items.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
import { compose, map, prop } from 'ramda'

it('loads 3 elements', () => {
  cy.visit('index.html')
  // instead of using Cypress ".get" command
  // write our own function to return elements
  const getElements = (doc) =>
    doc.querySelectorAll('#app div')

  const mapInnerText = map(prop('innerText'))

  const getTexts = compose(mapInnerText, getElements)

  cy.document()
    .pipe(getTexts)
    .should((texts) => {
      expect(texts[0]).to.contain('first child')
      expect(texts[1]).to.contain('second child')
      expect(texts[2]).to.contain('third child')
    })
})

In the above case, we don't even need should(cb) with custom function, and we can use deep equality to confirm the text inside the elements.

1
2
3
cy.document()
  .pipe(getTexts)
  .should('deep.equal', ['first child', 'second child', 'third child'])

We can always start with custom should(cb) callback function, then if we notice general data transformations, refactor it to make it simpler and "standard-like". Readability and simplicity is the goal.

No cy commands allowed

Because the .should(cb) callback can potentially run many times, you should never use any cy.<command> methods (you can still use the static Cypress.<command> methods). Trying to call cy.<command> from the callback will try to schedule lots of commands, changing the subject for the next iteration, and leading to the unpredictable results. Here is a typical example that incorrectly tries to check if the value sent by the server is shown on the page:

1
2
3
4
5
6
7
8
9
10
11
// INCORRECT, the test times out
function checkFruit(fruit) {
  cy.contains('#fruit', fruit)
}
it('shows the right fruit', function () {
  cy.intercept('GET', '/fruit').as('fruit')
  cy.visit('/')
  cy.wait('@fruit')
    .its('response.body.fruit')
    .should(checkFruit)
})

Instead, separate the assertions from commands. In the test above, let's confirm the fruit returned by the server is a string, and then check the page.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
// ✅ The right way to check the page
function checkFruit(fruit) {
  cy.contains('#fruit', fruit)
}
it('shows the right fruit', function () {
  cy.intercept('GET', '/fruit').as('fruit')
  cy.visit('/')
  cy.wait('@fruit')
    .its('response.body.fruit')
    .should('be.a', 'string')
    // once we get the fruit, no need to retry
    // just pass it to the next command
    .then(checkFruit)
})

For more, watch the video Do Not Use cy Commands Inside A Should Callback Function.

More information

Update 1

I wrote my own little library of functional utilities that works very well with Cypress should callbacks, read Functional Helpers For Cypress Tests.