When Has The App Loaded

Let's take an application that fetches data from the server and write end-to-end tests for it. What happens if the application takes a little bit longer than usual to bootstrap, get the data, and render it? Will the test fail because it does not wait for the data to finish loading?

The example I am about to show comes from bahmutov/cypress-workshop-basics. The test visits the application URL and confirms there are no items on the page.

it('starts with zero items', () => {
  cy.visit('/')
  cy.get('li.todo').should('have.length', 0)
})

Hmm, the test passes, yet I can clearly see 2 todo items on the page? Is the assertion cy.get('li.todo').should('have.length', 0) not doing its job?

The test finishes too quickly. While the application is still loading the data, the assertion cy.get('li.todo').should('have.length', 0) checks the empty initial page. There are no todo items, and the test completes. Only then the Ajax call returns and the 2 items are rendered on the page. By then the test has finished.

Note that the .should('have.length', 0) assertion is similar to the .should('not.exist') assertion. Such negative assertions are dangerous in my opinion, they can pass for the wrong reason (like in our case). Read the blog posts Be Careful With Negative Assertions and Negative Assertions And Missing States for more examples.

Waiting for the data load

The simplest solution to this problem, is to make the test wait for the data to load. Perhaps a one second delay would be enough?

it('starts with zero items (waits)', () => {
  cy.visit('/')
  cy.wait(1000)
  cy.get('li.todo').should('have.length', 0)
})

Yes, now the test fails as expected, because the page shows the loaded data when the test checks it.

Is one second wait enough? It might be enough when running the application locally. When running the tests against a remote server, we might need to wait longer. The worst is when the data load takes approximately one second. It leads to a race condition between the application and the test. Sometimes the application takes slightly longer than one second, and the test fails to detect the data, leading to the flaky tests.

Waiting for network call to finish

A much better solution is to spy on the Ajax call made by the application and wait for the data to return before checking the page. You can use cy.intercept command to spy on calls made by the application. You can spy on Ajax calls or any resource requested by the browser.

it('starts with zero items', () => {
  // start Cypress network server
  // spy on route `GET /todos`
  // THEN visit the page
  cy.intercept('GET', '/todos').as('todos')
  cy.visit('/')
  cy.wait('@todos') // wait for `GET /todos` response
    // inspect the server's response
    .its('response.body')
    .should('have.length', 0)
  // then check the DOM
  // note that we don't have to use "cy.wait(...).then(...)"
  // because all Cypress commands are flattened into a single chain
  // automatically. Thus just write "cy.wait(); cy.get();" naturally
  cy.get('li.todo').should('have.length', 0)
})

The command cy.wait('@todos') waits for the network call, even if the call is made later.

In my application example, I can force the application to wait N seconds before making the initial load.

The application might delay the Ajax call.

app.js

// state.delay comes from URL query parameter
// ?delay=2000
setTimeout(() => {
  commit('SET_LOADING', true)

  axios
    .get('/todos')
    .then((r) => r.data)
    .then((todos) => {
      commit('SET_TODOS', todos)
      commit('SET_LOADING', false)
    })
    .catch((e) => {
      console.error('could not load todos')
      console.error(e.message)
      console.error(e.response.data)
    })
    .finally(() => {
      // an easy way for the application to signal
      // that it is done loading
      document.body.classList.add('loaded')
    })
}, state.delay)

The delay makes no difference, the test still correctly waits for the application to load its initial data.

it('starts with zero items (delay)', () => {
  cy.intercept('GET', '/todos').as('todos')
  cy.visit('/?delay=2000')
  cy.wait('@todos')
  cy.get('li.todo').should('have.length', 0)
})

Look at the recording below. The Ajax call is detected two seconds after the visit, yet cy.wait('@todos') happily waits for it before Cypress proceeds to the next command.

Render delay

What if our application has a delay between receiving the data and rendering it on the page?

app.js

axios
  .get('/todos')
  .then((r) => r.data)
  .then((todos) => {
    setTimeout(() => {
      commit('SET_TODOS', todos)
      commit('SET_LOADING', false)
    }, state.renderDelay)
  })

Hmm, our test again passes when it should have failed.

it('starts with zero items (delay plus render delay)', () => {
  cy.intercept('GET', '/todos').as('todos')
  cy.visit('/?delay=2000&renderDelay=1500')
  cy.wait('@todos')
  cy.get('li.todo').should('have.length', 0)
})

The "missing" step between finishing the network call and rendering the data on the page is exactly the problem I have described in the blog post Negative Assertions And Missing States. The test does not "know" that the application is still not done loading. We need an explicit way of signalling from the application to the test runner "I am done loading the data".

Observe the page

The best approach to tell that the application has finished loading ... is for application to set something observable to tell the test runner (and the human users) that the data load is done. For example, the application can set a CSS class or a utility data attribute

app.js

SET_LOADING(state, flag) {
  state.loading = flag
  if (flag === false) {
    // an easy way for the application to signal
    // that it is done loading
    document.body.classList.add('loaded')
  }
}

Then the test can tell when the application has finished loading by observing the <body> element.

it('starts with zero items (check body.loaded)', () => {
  // use delays to simulate the delayed load and render
  cy.visit('/?delay=2000&renderDelay=1500')
  // the application sets "loaded" class on the body
  // in the test we can check for this class
  cy.get('body').should('have.class', 'loaded')
  // then check the number of items
  cy.get('li.todo').should('have.length', 0)
})

The test behaves correctly and fails as expected.

Note: the assertion .should('have.class', 'loaded') passes after 3.5 seconds; the application requests the data after 2 seconds, and renders it after 1.5 seconds. The assertion might time out if the data load takes slightly longer, since the default command timeout is 4 seconds. I would suggest using a longer command time out in this case:

cy.get('body', { timeout: 7_000 }).should('have.class', 'loaded')

Check the window object

We can use another mechanism to signal the test runner that the application has finished loading the data. We can even pass the data we got from the server!

app.js

SET_TODOS(state, todos) {
  state.todos = todos
  // expose the todos via the global "window" object
  // but only if we are running Cypress tests
  if (window.Cypress) {
    window.todos = todos
  }
}

The test then can detect when the window.todos property is set using cy.window and cy.its commands.

it('starts with zero items (check the window)', () => {
  // use delays to simulate the delayed load and render
  cy.visit('/?delay=2000&renderDelay=1500')
  cy.window().its('todos', { timeout: 7_000 })
  // then check the number of items
  cy.get('li.todo').should('have.length', 0)
})

If we can detect when the window.todos property is set, we can read the actual todos and use them to check the rendered page.

it('starts with N items', () => {
  // use delays to simulate the delayed load and render
  cy.visit('/?delay=2000&renderDelay=1500')
  // access the loaded Todo items
  cy.window()
    // you can drill down nested properties using "."
    .its('todos.length')
    .then((n) => {
      // then check the number of items
      cy.get('li.todo').should('have.length', n)
    })
})

You can even use the window.todos to check what the page renders.

it('starts with N items and checks the page', () => {
  // use delays to simulate the delayed load and render
  cy.visit('/?delay=2000&renderDelay=1500')
  // access the loaded Todo items
  cy.window()
    .its('todos')
    .then((todos) => {
      // then check the number of items
      cy.get('li.todo').should('have.length', todos.length)
      todos.forEach((todo) => {
        if (todo.completed) {
          cy.contains('.todo', todo.title).should('have.class', 'completed')
        } else {
          cy.contains('.todo', todo.title).should('not.have.class', 'completed')
        }
      })
    })
})

For more on accessing the application data from the Cypress test via the window object, see the blog post Stub Objects By Passing Them Via Window Property.

Conclusion

The test runner should not "run away" from the application. If the application is still loading the data, the test runner should wait for the data to load and the page to be ready to continue testing. In this blog post, I have shown how to wait for the data to load using:

• a hard-coded cy.wait(N) command
• network spy using the cy.intercept command
• a body element property added after the load
• a property on the window object set by the application after the load

Happy testing!