Set flag to start tests

Imagine your web application (especially in development mode) is slow to start. Maybe it is loading a lot of code, or asking an under-powered server for data. But it does not start working right away. If your end-to-end test runner does not know about it, it might try to start running the tests too soon. Here is a typical TodoMVC app bahmutov/todomvc-with-delay that only starts running 2.5 seconds after the page loads.

todomvc/app.js

;(function () {
  const startApp = () => {
    // do the app stuff

    // if you want to expose "app" globally only
    // during end-to-end tests you can guard it using "window.Cypress" flag
    if (window.Cypress) {
      window.app = app
      console.log('app has started')
    }
  }

  // start application after an artificial delay
  setTimeout(startApp, 2500)
}())

And here is a typical Cypress test that enters two items and tried to verify that the list has two items.

cypress/integration/spec.js

beforeEach(() => {
  cy.visit('/')
})

const addItem = text => {
  cy.get('.new-todo').type(`${text}{enter}`)
}

it('waits for app to start', () => {
  addItem('first item')
  addItem('second item')
  cy.get('li.todo').should('have.length', 2)
})

When we run this app, the test fails - the input field shows the jumbled text; the app is NOT ready to react to the user input!

Property is added

We want our test runner to wait until the window object has property app. The Chai assertion just writes itself - and Chai is bundled with Cypress, and it should work with the object returned by cy.window() command.

cypress/integration/spec.js

beforeEach(() => {
  cy.visit('/')
  cy.window().should('have.property', 'app')
})

👉 important - every assertion in Cypress automatically retries previous command if possible, see should documentation. Thus a single line cy.window().should(...) executes command cy.window() multiple times, until the assertion immediately after it passes, or it times out.

Property changes value

If there is a flag on the window from the very beginning that changes its value when the app is ready to be tested, we can write assertion in other ways. For example we can use assertion have.property <name> <expected value> against cy.window().

todomvc/app.js

;(function () {
  const startApp = () => {
    // do the app stuff

    if (window.Cypress) {
      // change appReady property
      window.appReady = true
      console.log('app has started')
    }
  }

  // set the initial flag right away
  window.appReady = false
  // start application after an artificial delay
  setTimeout(startApp, 2500)
}())

cypress/integration/spec.js

beforeEach(() => {
  cy.visit('/')
  //                    assertion      prop name  value
  cy.window().should('have.property', 'appReady', true)
})

Alternatively, we can take the cy.window() result, get specific property using its() and then assert its value.

cypress/integration/spec.js

beforeEach(() => {
  cy.visit('/')
  cy.window().its('appReady').should('equal', true)
})

Again, the last action its will be retried until the assertion passes or times out.

Stale references

The only place you can get into a weird situation is if the entire object changes, while your test keeps holding an old "orphan" reference, and retries getting its property and checking its value.

todomvc/app.js

;(function () {
  const startApp = () => {
    // do the app stuff

    if (window.Cypress) {
      // ⚠️ overwrite window.config object
      window.config = {
        appReady: true
      }
      console.log('app has started')
    }
  }

  window.config = {
    appReady: false
  }
  // start application after an artificial delay
  setTimeout(startApp, 2500)
}())

Notice how window.config is replaced with a new object when the application is ready. What happens if we already have a reference to window.config in our test?

cypress/integration/spec.js

beforeEach(() => {
  cy.visit('/')
  cy.window().its('config').its('appReady').should('equal', true)
})

Assertion .should('equal', true) only retries its immediate previous command, in this case its('appReady'), which keeps using window.config object we got right away. When the application code replaces window.config with a new object, our test still keeps checking the stale "orphan" object, and the test times out.

Luckily for us, Cypress has enough tricks up its sleeve to solve this problem in a couple of ways. For example, we can use a different Chai assertion to avoid even using its commands and retrying cy.window() command!

cypress/integration/spec.js

beforeEach(() => {
  cy.visit('/')
  cy.window().should('have.deep.property', 'config.appReady', true)
})

Alternatively, we can rely on the fact that cy.its uses Lodash.get function under the hood, which allows getting (safely) deep properties

cypress/integration/spec.js

beforeEach(() => {
  cy.visit('/')
  cy.window().its('config.appReady').should('equal', true)
})

Both ways work just fine. If you want to see more examples of asserting when properties are added / deleted / changed, check out this commit with example tests.

Conclusion

Exposing some flag from the application to let your tests work better is good practice in my opinion. I have shown ways to set a flag in the application code, which tests can detect reliably. If you do not have control over the application code, you can still detect when the application starts reacting to DOM events, but that is a little bit more complicated. Yet it can be done, because Cypress lets your test code inspect, observe and mock any application code, including browser APIs. But to make your life easier, use a flag for slowly starting apps!

Find all source code from this post in bahmutov/todomvc-with-delay and don't forget to give cypress-io/cypress a GitHub ⭐️.