process

Never Use The Page As The Source Of Truth

Do not use the web page as the source of truth in your end-to-end tests.

Imagine you are testing a web page showing the purchase receipt.

The receipt page test

When the user clicks on the "View Summary" button, a dialog pops up showing just the total amount paid

cypress/e2e/receipt.cy.js
1
2
3
4
5
6
describe('Receipt', () => {
  it('shows the price on the receipt', () => {
    cy.visit('app/index.html')
    cy.contains('button', 'View Summary').click()
  })
})

The total amount paid popup

How would you confirm that the total $ shown in the summary is the same as the total amount shown in the popup dialog?

Are these numbers equal?

A typical test would simply grab one amount and check if the second element has the same text

cypress/e2e/receipt.cy.js
1
2
3
4
5
6
7
8
9
10
11
12
describe('Receipt', () => {
  it('shows the price on the receipt', () => {
    cy.visit('app/index.html')
    cy.get('#grand-total')
      .invoke('text')
      .should('be.a', 'string')
      .then((totalText) => {
        cy.contains('button', 'View Summary').click()
        cy.get('#dialog-total-value').should('have.text', totalText)
      })
  })
})

A naive approach checks if one element has the same text as another element

This is a bad idea. You are using the page as the source of truth. You are trusting the first element to be correct, without verifying it. For example, both elements might not shown any meaningful numbers!

The test is still passing

Oops, the two elements show the same meaningless text.

Comparing elements' text is also prone to trip up on formatting and display differences. For example, one element could put the $ character outside its span element.

1
2
3
4
5
6
<!-- the summary row -->
<span id="grand-total">$304.97</span>
<!-- the order popup -->
<p class="dialog-total-value">
  $ <span id="dialog-total-value">304.97</span>
</p>

To the user, both totals are correct. But the test fails.

The test fails because of different formatting

Instead of trusting the web page to be correct, let the test itself be the source of the ground truth. Stub the data, stub the network call, control the data loaded by the page - any technique is good. Even a little bit of duplication is ok in the testing code as long as the test intention is clear and easy to maintain.

cypress/e2e/receipt.cy.js
1
2
3
4
5
6
7
8
describe('Receipt', () => {
  it('shows the price on the receipt', () => {
    cy.visit('app/index.html')
    cy.get('#grand-total').should('have.text', '$304.97')
    cy.contains('button', 'View Summary').click()
    cy.get('#dialog-total-value').should('have.text', '304.97')
  })
})

The test is passing, does not trust the page to show the correct value, and is easier to read, since we avoid using a cy.then callback - something I always advocated, see the video Good Cypress Test Syntax for example.

The test uses itself as the source of truth

Solid.