Cypress examples (v13.15.0)

Negative Assertions

📺 Watch the explanation for this video in Negative Assertionsopen in new window.

The basics

Cypress has built-in retry-ability in most of its commands. If an assertion does not pass, Cypress keeps retrying. If an assertion is positive then it is usually easy to reason.

For example, the page does not show the "Loaded" text initially. We want to check if the page shows "Loaded", thus we check if the positive change happens on the page.

<div id="basics">starting...</div>
<script>
  // the text appears after a delay
  setTimeout(function () {
    const el = document.getElementById('basics')
    el.innerText = 'Loaded'
  }, 1500)
</script>

// cy.contains has a built-in "exists" assertion
cy.contains('#basics', 'Loaded')

A negative assertion

A negative assertion checks if something is not there. If the element is present, then Cypress retries until the negative assertion passes.

<div id="basics">starting...</div>
<script>
  // the text appears after a delay
  setTimeout(function () {
    const el = document.getElementById('basics')
    el.innerText = 'Loaded'
  }, 1500)
</script>

Let's use a negative assertion to confirm the text "starting..." goes away.

cy.contains('#basics', 'starting').should('not.exist')

A problem with negative assertions is timing. Often such assertions pass too early when the application has not finished loading or updating. For example, I often see the test code like this that probably would happily be passing, while the application is broken:

cy.get('button').click()
cy.location('pathname').should('equal', '/new-url')
cy.contains('#error').should('not.exist')

Guess what - the error element is not there when the page just starts loading, right. The test finishes, while the application loads its data, and then "boom!" it shows an error. But our test has completed already. In the next couple of examples, I will explain how to solve this problem.

First example (bad)

In this example, the test finishes too quickly. It does not wait for the "Loading..." text to go away, it simply checks if right now the error message does not exist. There is not error message yet, because the application is still loading.

<div id="app1">Loading...</div>
<script>
  setTimeout(function () {
    const el = document.getElementById('app1')
    el.innerText = 'Loaded'
  }, 1500)
</script>

// negative assertion
cy.get('#error').should('not.exist')

Here is the application code that might "break" the test and cause it to pass for the wrong reason:

<div>About to load...</div>

Second example (slightly better)

Let's confirm first a change in the application, like the "Loading..." text going away.

<div id="app2">Loading...</div>
<script>
  setTimeout(function () {
    const el = document.getElementById('app2')
    el.innerText = 'Loaded'
  }, 1500)
</script>

// negative assertion
cy.contains('Loading...').should('not.exist')
// negative assertion
cy.get('#error').should('not.exist')

Here is the application code that might show an error, yet the test would pass.

<div id="app2b">Loading...</div>
<script>
  setTimeout(function () {
    const el = document.getElementById('app2b')
    el.innerText = '...'
  }, 1500)
  // the error is displayed after a short delay
  setTimeout(function () {
    const el = document.getElementById('app2b')
    el.innerText = 'Error'
  }, 1550)
</script>

We still have a problem: what if the "Loading..." text does not appear immediately? Then both negative assertions would pass even before the application renders or loads anything meaningful.

Third example (better)

Let's put a positive assertion in first to confirm the Loading text appears. Then we can confirm the Loading text goes away, and only then we can check if the error does not exist.

<div id="app3">Loading...</div>
<script>
  setTimeout(function () {
    const el = document.getElementById('app3')
    el.innerText = 'Loaded'
  }, 1500)
</script>

// positive assertion
cy.contains('Loading...')
// negative assertion
cy.contains('Loading...').should('not.exist')
// negative assertion
cy.get('#error').should('not.exist')

Again, if the application is showing an error after a short delay after the loading element goes away, our test will be green, yet the user would see an error.

<div id="app2b">Loading...</div>
<script>
  setTimeout(function () {
    const el = document.getElementById('app2b')
    el.innerText = '...'
  }, 1500)
  // the error is displayed after a short delay
  setTimeout(function () {
    const el = document.getElementById('app2b')
    el.innerText = 'Error'
  }, 1550)
</script>

Positive example (the best)

Finally, let's make every change in our application detected. For example, we could check if the component showing that the data has loaded appears before checking the error does not exist.

<div id="app4">Loading...</div>
<script>
  setTimeout(function () {
    const el = document.getElementById('app4')
    el.innerHTML =
      '<div id="timing" data-duration="2 seconds" /><div>Loaded</div>'
  }, 1500)
</script>

// positive assertion
cy.contains('Loading...')
// negative assertion
cy.contains('Loading...').should('not.exist')

We know that when the load has finished, we will have #timingelement. At the same time there could be an error element. Thus if the #timing exists, then it is safe to check if the error exists or not.

// positive assertion
cy.get('#timing')
// negative assertion
cy.get('#error').should('not.exist')

See also