When Has The App Loaded

How to wait for the application to load data before testing it.

Let's take an application that fetches data from the server and write end-to-end tests for it. What happens if the application takes a little bit longer than usual to bootstrap, get the data, and render it? Will the test fail because it does not wait for the data to finish loading?

The example I am about to show comes from bahmutov/cypress-workshop-basics. The test visits the application URL and confirms there are no items on the page.

1
2
3
4
it('starts with zero items', () => {
cy.visit('/')
cy.get('li.todo').should('have.length', 0)
})

Hmm, the test passes, yet I can clearly see 2 todo items on the page? Is the assertion cy.get('li.todo').should('have.length', 0) not doing its job?

The test passes when it should have failed

The test finishes too quickly. While the application is still loading the data, the assertion cy.get('li.todo').should('have.length', 0) checks the empty initial page. There are no todo items, and the test completes. Only then the Ajax call returns and the 2 items are rendered on the page. By then the test has finished.

Note that the .should('have.length', 0) assertion is similar to the .should('not.exist') assertion. Such negative assertions are dangerous in my opinion, they can pass for the wrong reason (like in our case). Read the blog posts Be Careful With Negative Assertions and Negative Assertions And Missing States for more examples.

Waiting for the data load

The simplest solution to this problem, is to make the test wait for the data to load. Perhaps a one second delay would be enough?

1
2
3
4
5
it('starts with zero items (waits)', () => {
cy.visit('/')
cy.wait(1000)
cy.get('li.todo').should('have.length', 0)
})

Yes, now the test fails as expected, because the page shows the loaded data when the test checks it.

The test waits one second before checking the page

Is one second wait enough? It might be enough when running the application locally. When running the tests against a remote server, we might need to wait longer. The worst is when the data load takes approximately one second. It leads to a race condition between the application and the test. Sometimes the application takes slightly longer than one second, and the test fails to detect the data, leading to the flaky tests.

Waiting for network call to finish

A much better solution is to spy on the Ajax call made by the application and wait for the data to return before checking the page. You can use cy.intercept command to spy on calls made by the application. You can spy on Ajax calls or any resource requested by the browser.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
it('starts with zero items', () => {
// start Cypress network server
// spy on route `GET /todos`
// THEN visit the page
cy.intercept('GET', '/todos').as('todos')
cy.visit('/')
cy.wait('@todos') // wait for `GET /todos` response
// inspect the server's response
.its('response.body')
.should('have.length', 0)
// then check the DOM
// note that we don't have to use "cy.wait(...).then(...)"
// because all Cypress commands are flattened into a single chain
// automatically. Thus just write "cy.wait(); cy.get();" naturally
cy.get('li.todo').should('have.length', 0)
})

The test waits for the network call to complete

The command cy.wait('@todos') waits for the network call, even if the call is made later.

In my application example, I can force the application to wait N seconds before making the initial load.

The application might delay the Ajax call.

app.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
// state.delay comes from URL query parameter
// ?delay=2000
setTimeout(() => {
commit('SET_LOADING', true)

axios
.get('/todos')
.then((r) => r.data)
.then((todos) => {
commit('SET_TODOS', todos)
commit('SET_LOADING', false)
})
.catch((e) => {
console.error('could not load todos')
console.error(e.message)
console.error(e.response.data)
})
.finally(() => {
// an easy way for the application to signal
// that it is done loading
document.body.classList.add('loaded')
})
}, state.delay)

The delay makes no difference, the test still correctly waits for the application to load its initial data.

1
2
3
4
5
6
it('starts with zero items (delay)', () => {
cy.intercept('GET', '/todos').as('todos')
cy.visit('/?delay=2000')
cy.wait('@todos')
cy.get('li.todo').should('have.length', 0)
})

Look at the recording below. The Ajax call is detected two seconds after the visit, yet cy.wait('@todos') happily waits for it before Cypress proceeds to the next command.

The test waits for the delayed Ajax call

Render delay

What if our application has a delay between receiving the data and rendering it on the page?

app.js
1
2
3
4
5
6
7
8
9
axios
.get('/todos')
.then((r) => r.data)
.then((todos) => {
setTimeout(() => {
commit('SET_TODOS', todos)
commit('SET_LOADING', false)
}, state.renderDelay)
})

Hmm, our test again passes when it should have failed.

1
2
3
4
5
6
it('starts with zero items (delay plus render delay)', () => {
cy.intercept('GET', '/todos').as('todos')
cy.visit('/?delay=2000&renderDelay=1500')
cy.wait('@todos')
cy.get('li.todo').should('have.length', 0)
})

The delay between the network call and showing items on the page confuses the test

The "missing" step between finishing the network call and rendering the data on the page is exactly the problem I have described in the blog post Negative Assertions And Missing States. The test does not "know" that the application is still not done loading. We need an explicit way of signalling from the application to the test runner "I am done loading the data".

Observe the page

The best approach to tell that the application has finished loading ... is for application to set something observable to tell the test runner (and the human users) that the data load is done. For example, the application can set a CSS class or a utility data attribute

app.js
1
2
3
4
5
6
7
8
SET_LOADING(state, flag) {
state.loading = flag
if (flag === false) {
// an easy way for the application to signal
// that it is done loading
document.body.classList.add('loaded')
}
}

Then the test can tell when the application has finished loading by observing the <body> element.

1
2
3
4
5
6
7
8
9
it('starts with zero items (check body.loaded)', () => {
// use delays to simulate the delayed load and render
cy.visit('/?delay=2000&renderDelay=1500')
// the application sets "loaded" class on the body
// in the test we can check for this class
cy.get('body').should('have.class', 'loaded')
// then check the number of items
cy.get('li.todo').should('have.length', 0)
})

The test behaves correctly and fails as expected.

The test waits for the body element to have class loaded

Note: the assertion .should('have.class', 'loaded') passes after 3.5 seconds; the application requests the data after 2 seconds, and renders it after 1.5 seconds. The assertion might time out if the data load takes slightly longer, since the default command timeout is 4 seconds. I would suggest using a longer command time out in this case:

1
cy.get('body', { timeout: 7_000 }).should('have.class', 'loaded')

Check the window object

We can use another mechanism to signal the test runner that the application has finished loading the data. We can even pass the data we got from the server!

app.js
1
2
3
4
5
6
7
8
SET_TODOS(state, todos) {
state.todos = todos
// expose the todos via the global "window" object
// but only if we are running Cypress tests
if (window.Cypress) {
window.todos = todos
}
}

The test then can detect when the window.todos property is set using cy.window and cy.its commands.

1
2
3
4
5
6
7
it('starts with zero items (check the window)', () => {
// use delays to simulate the delayed load and render
cy.visit('/?delay=2000&renderDelay=1500')
cy.window().its('todos', { timeout: 7_000 })
// then check the number of items
cy.get('li.todo').should('have.length', 0)
})

Continue the test when the application sets the window.todos object

If we can detect when the window.todos property is set, we can read the actual todos and use them to check the rendered page.

1
2
3
4
5
6
7
8
9
10
11
12
it('starts with N items', () => {
// use delays to simulate the delayed load and render
cy.visit('/?delay=2000&renderDelay=1500')
// access the loaded Todo items
cy.window()
// you can drill down nested properties using "."
.its('todos.length')
.then((n) => {
// then check the number of items
cy.get('li.todo').should('have.length', n)
})
})

Using the data from the app to check the rendered page

You can even use the window.todos to check what the page renders.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
it('starts with N items and checks the page', () => {
// use delays to simulate the delayed load and render
cy.visit('/?delay=2000&renderDelay=1500')
// access the loaded Todo items
cy.window()
.its('todos')
.then((todos) => {
// then check the number of items
cy.get('li.todo').should('have.length', todos.length)
todos.forEach((todo) => {
if (todo.completed) {
cy.contains('.todo', todo.title).should('have.class', 'completed')
} else {
cy.contains('.todo', todo.title).should('not.have.class', 'completed')
}
})
})
})

Check if the page renders each todo correctly

For more on accessing the application data from the Cypress test via the window object, see the blog post Stub Objects By Passing Them Via Window Property.

Conclusion

The test runner should not "run away" from the application. If the application is still loading the data, the test runner should wait for the data to load and the page to be ready to continue testing. In this blog post, I have shown how to wait for the data to load using:

  • a hard-coded cy.wait(N) command
  • network spy using the cy.intercept command
  • a body element property added after the load
  • a property on the window object set by the application after the load

Happy testing!