Incredibly Powerful cy.task

There is a new super powerful command in Cypress v3 - and that is cy.task. This command allows your tests to "jump" from the browser context to Node and run any code before returning (asynchronously) the result back to the test. Let me show how to use this command for "deeper" server-side validation.

Let me take a TodoMVC application as an example. This is the same web application I have tested a lot in Testing Vue web applications with Vuex data store & REST backend blog post. You can find all code from this blog post in bahmutov/cypress-task-demo repository.

The TodoMVC application sends each new todo item entered by the user to the backend server (via XHR calls). The backend is json-server that saves the data as a plain JSON file called data.json. Good. So how can we assert that all parts of the system are working as expected?

Testing the UI

The most obvious thing that everyone writing end-to-end tests should do is to only exercise the application via its user interface. For example, we can add a new todo item, and then confirm that the text of the item appears in the list below.

ui.js

import { enterTodo, resetDatabase } from './utils'

describe('UI', () => {
  beforeEach(resetDatabase)
  beforeEach(() => {
    cy.visit('/')
  })

  it('adds todo', () => {
    // random text to avoid confusion
    const id = Cypress._.random(1, 1e6)
    const title = `todo ${id}`
    enterTodo(title)
    // confirm the new item has been added to the list
    cy.contains('.todoapp li', title)
  })
})

Here is this test in action; I am hovering over "CONTAINS" command and Cypress highlights the new item in the DOM snapshots at that moment in test.

The new item appears in the list, but was it really sent to the server? We could write another test to spy on the XHR call to observe the new item being sent to the server. But was the item really saved? Hmm, we are going deeper in the implementation details here. Why not avoid testing the implementation of the application and the server and instead test the external state - in this case the file "database" where the server saves data?

Testing the database

So every time the user enters new Todo item, it should be saved in the file "data.json" like this

data.json

{
  "todos": [
    {
      "title": "todo 423665",
      "completed": false,
      "id": "9197021112"
    }
  ]
}

We can write a test that adds an item via UI, but then checks the database to make sure the new record has been added. There is cy.readFile that can read file contents, but it is not powerful enough:

• cy.readFile fails the test if the file does not exist
• we want to write general code that can actually query any database, not just read a file.

The new command cy.task allows us to do anything. This is an "escape" hatch - a way for the end-to-end test running in the browser to run code in Node environment.

So let's write a new task - and all it has to do is to find the new text in the database. We will write this code inside cypress/plugins/index.js file - that is the place for all Node code inside Cypress tests.

cypress/plugins/index.js

const fs = require('fs')
const path = require('path')
const repoRoot = path.join(__dirname, '..', '..')

const findRecord = title => {
  const dbFilename = path.join(repoRoot, 'data.json')
  const contents = JSON.parse(fs.readFileSync(dbFilename, 'utf8'))
  const todos = contents.todos
  return todos.find(record => record.title === title)
}

module.exports = (on, config) => {
  // "cy.task" can be used from specs to "jump" into Node environment
  // and doing anything you might want. For example, checking "data.json" file!
  on('task', {
    hasSavedRecord (title) {
      console.log('looking for title "%s" in the database', title)
      return Boolean(findRecord(title))
    }
  })
}

We can "call" cy.task passing arguments (which should be serializable).

spec.js

import { enterTodo, resetDatabase } from './utils'

describe('cy.task', () => {
  beforeEach(resetDatabase)
  beforeEach(() => {
    cy.visit('/')
  })

  it('finds record in the database', () => {
    // random text to avoid confusion
    const id = Cypress._.random(1, 1e6)
    const title = `todo ${id}`
    enterTodo(title)
    // confirm the new item has been saved
    // https://on.cypress.io/task
    cy.task('hasSavedRecord', title).should('equal', true)
  })
})

Our test is passing!

The terminal where I started Cypress test runner shows the console log from the cy.task command, which happens after posting the item

POST /todos 201 8.135 ms - -
looking for title "todo 166155" in the database

Everything is great!

Or is it?

If at first you don't succeed ...

"Dust yourself off and try again", right?

We have a problem - we really assume that the item is saved before we check for it. But in the real world things are delayed, items are buffered before being sent or saved, etc. That is why Cypress is retrying all its commands - because nothing happens instantly!

To simulate the problem, let me change the TodoMVC application and add a 2 second delay when adding an item.

// addTodo event listener
addTodo (e) {
  e.target.value = ''
  // delay by 2 seconds on purpose
  setTimeout(() => {
    this.$store.dispatch('addTodo')
    this.$store.dispatch('clearNewTodo')
  }, 2000)
}

If you rerun the ui.js test file, the test still passes - the cy.contains just waits for 2 seconds; it keeps observing the DOM and passes as soon as the new item text is found in the list.

But if you run spec.js the assertion cy.task('hasSavedRecord', title).should('equal', true) fails!

Notice in the screenshot that the failed assertion is placed before the POST XHR request from the web application to the server. This gives us a clue that we checked the database too early.

The cy.task command does not retry. Because Cypress has no idea what your task is going to do - it probably is NOT idempotent action. For example cy.get is idempotent command; it does not change the state of the application, unlike cy.type or cy.click that do. Just like Cypress cannot automatically retry cy.click Cypress cannot retry cy.task command.

But we can!

Let us wrap the code that is checking the database file with a loop. We are going to keep checking the file until we find the record or hit the time limit. Instead of returning a boolean value, we are going to return a promise, and cy.task will automatically wait for the promise. Here is the plugins/index.js code that just keeps chaining promises, checking the file every 50 milliseconds.

cypress/plugins/index.js

const findRecord = title => {
  const dbFilename = path.join(repoRoot, 'data.json')
  const contents = JSON.parse(fs.readFileSync(dbFilename, 'utf8'))
  const todos = contents.todos
  return todos.find(record => record.title === title)
}

const hasRecordAsync = (title, ms) => {
  const delay = 50
  return new Promise((resolve, reject) => {
    if (ms < 0) {
      return reject(new Error(`Could not find record with title "${title}"`))
    }
    const found = findRecord(title)
    if (found) {
      return resolve(true)
    }
    setTimeout(() => {
      hasRecordAsync(title, ms - delay).then(resolve, reject)
    }, 50)
  })
}

module.exports = (on, config) => {
  // "cy.task" can be used from specs to "jump" into Node environment
  // and doing anything you might want. For example, checking "data.json" file!
  on('task', {
    hasSavedRecord (title, ms = 3000) {
      console.log(
        'looking for title "%s" in the database (time limit %dms)',
        title,
        ms
      )
      return hasRecordAsync(title, ms)
    }
  })
}

Our test now passes - beautiful!

Notice that the assertion task(...).should('equal', true) passes AFTER the web application sends XHR to the server.

Making it beautiful

User should know when the application is busy doing something, and the user should be notified when an action either succeeded or failed. Cypress command timeline shows a blue spinner while an action is being retried, and it uses icons and colors to show test commands that passed and failed. Our Node code should do the same thing for the tasks. Luckily this can be added using a nice spinner library ora.

I will change my plugins code to wrap custom promise-returning code with Bluebird promise that will control a CLI spinner.

cypress/plugins/index.js

const ora = require('ora')
const Promise = require('bluebird')
// rest of the code is the same
module.exports = (on, config) => {
  // "cy.task" can be used from specs to "jump" into Node environment
  // and doing anything you might want. For example, checking "data.json" file!
  on('task', {
    hasSavedRecord (title, ms = 3000) {
      const spinner = ora(
        `looking for title "${title}" in the database`
      ).start()
      return hasRecordAsync(title, ms)
        .tap(() => {
          spinner.succeed(`found "${title}" in the database`)
        })
        .tapCatch(err => {
          spinner.fail(err.message)
        })
    }
  })
}

Here is the result - the spinner working in the terminal, showing a message on success.

Here is the spinner when the task fails.

Final thoughts

When writing Cypress tests automatic retries are sooo convenient - you just don't have to think at all when exactly the things inside your application happen. You don't have to put wait(5000) in order to predict delays, yet the tests keep flying because they never have to wait longer than necessary. cy.task gives us a tremendous power to run any Node code, but we have to wrap it with retries ourself. Luckily it is simple to do so.

Bonus 1: cy.task fails if the promise is resolved

If the plugins code returns a rejected promise, then the cy.task command fails. For example, here is the plugins file with the task that always rejects

cypress/plugins/index.js

on('task', {
  fails () {
    return Promise.reject(new Error('Expected to fail'))
  }
})

This test always fails

cypress/integration/spec.js

it('fails when cy.task rejects', () => {
  cy.task('fails')
})