process

Flaky IFrame Online Store Test

Solving a slowly loading iframe problem.

Recently, I was asked to write a solid test for an online demo e-commerce shop https://demo.prestashop.com. The site loads a temporary demo shop. We want to write a Cypress test that picks an item and adds it to the cart and goes to the checkout.

Let's start. First, visit the page.

1
2
3
it('goes to the checkout with one item', () => {
  cy.visit('/')
})

You see the loader element at the start. It is outside the iframe itself, let's wait for it to disappear.

1
2
3
4
it('goes to the checkout with one item', () => {
  cy.visit('/')
  cy.get('#loadingMessage', { timeout: 15_000 }).should('not.be.visible')
})

It might take a while to spin up a new shop instance, thus I set the maximum timeout of 15 seconds.

🎁 You can find the source code for this blog post in the repo bahmutov/test-presta-shop.

Access the iframe contents

Let's select Men's clothes. We can directly access the iframe contents, since it comes from the same domain as the top level window.

The iframe source comes from the same top-level domain

Let's get the iframe's document and find the link to the clothes.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
it('goes to the checkout with one item', () => {
  cy.visit('/')
  cy.get('#loadingMessage', { timeout: 15_000 }).should('not.be.visible')
  cy.get('iframe[title="Frame of demo shop"]')
    .its('0.contentDocument.body')
    .find('.header-top')
    .should('be.visible')
    .contains('a', 'Clothes')
    .click()
  cy.get('iframe[title="Frame of demo shop"]')
    .its('0.contentDocument.body')
    .find('.breadcrumb')
    .should('include.text', 'Clothes')
  cy.get('iframe[title="Frame of demo shop"]')
    .its('0.contentDocument.body')
    // we know the iframe body is there
    // but we need a jQuery object to be able to use cy.contains
    // otherwise we see "subject.each" is not a function error
    .then(cy.wrap)
    .contains('.category-sub-menu a', 'Men')
    .click()
  cy.get('iframe[title="Frame of demo shop"]')
    .its('0.contentDocument.body')
    .find('.breadcrumb')
    .should('include.text', 'Men')
})

Every time we click on the link or button, we check that the page has finished updating.

  • click on the "Clothes" link
    • confirm the page shows the breadcrumb "Clothes"
  • click on the "Men" link
    • confirm the page shows the breadcrumb "Men"

Note: I copy/paste the code to access the iframe's contents, we can clean it up later.

Add item to the cart

We see at least one item of clothing. Let's click on it and add it to the cart. We expect to see the following:

The item view where we want to click on the Add To Cart button

After clicking on the "Add to Cart" button we should see the cart modal fade in.

The item was added to cart

Let's write these commands in our test.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
// click on the first product item
cy.get('iframe[title="Frame of demo shop"]')
  .its('0.contentDocument.body')
  .find('.products .product')
  .should('have.length.greaterThan', 0)
  .first()
  .click()

// click on the "Add to cart" button
cy.get('iframe[title="Frame of demo shop"]')
  .its('0.contentDocument.body')
  .find('#main .product-container')
  .should('be.visible')
  .within(() => {
    cy.contains('.product-add-to-cart', 'Add to cart')
    cy.contains('button', 'Add to cart').should('be.visible').click()
  })

// the cart view appears
cy.get('iframe[title="Frame of demo shop"]')
  .its('0.contentDocument.body')
  .find('#blockcart-modal')
  .should('be.visible')

Hmm, the test clicked on the "Add to Cart" button, it the cart is empty and the test fails. What is happening? It works if we add a cy.wait before clicking on the "Add to Cart" button:

1
2
3
4
5
6
7
8
9
10
11
// click on the "Add to cart" button
cy.get('iframe[title="Frame of demo shop"]')
  .its('0.contentDocument.body')
  .find('#main .product-container')
  .should('be.visible')
  // ANTI-PATTERN: add a hard-coded wait
  .wait(200)
  .within(() => {
    cy.contains('.product-add-to-cart', 'Add to cart')
    cy.contains('button', 'Add to cart').should('be.visible').click()
  })

I would like to avoid hard-coded waits. Let's open DevTools to understand why the item does not "register" the click from the test.

Do you see the waterfall of small JavaScript scripts the page is downloading when we go to the Item view? It takes time to load all of them, one of them registers the "click" event listener on the "Add to Cart" button. The test clicks too fast, the event listener hasn't been downloaded or registered yet, so it does nothing. We need to wait for the JS resources to finish before clicking.

A good way to do this is by using my plugin cypress-network-idle. Let's wait for all JS resources before clicking.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
/// <reference types="cypress" />

// https://github.com/bahmutov/cypress-network-idle
import 'cypress-network-idle'

it('goes to the checkout with one item', () => {
  ...

  cy.get('iframe[title="Frame of demo shop"]')
    .its('0.contentDocument.body')
    .find('.breadcrumb')
    .should('include.text', 'Men')

  cy.waitForNetworkIdlePrepare({
    method: 'GET',
    pattern: '*.js',
    alias: 'js',
  })

  // click on the first product item
  cy.get('iframe[title="Frame of demo shop"]')
    .its('0.contentDocument.body')
    .find('.products .product')
    .should('have.length.greaterThan', 0)
    .first()
    .click()

  // click on the "Add to cart" button
  cy.get('iframe[title="Frame of demo shop"]')
    .its('0.contentDocument.body')
    .find('#main .product-container')
    .should('be.visible')
    .within(() => {
      cy.waitForNetworkIdle('@js', 40)
      cy.contains('.product-add-to-cart', 'Add to cart')
      cy.contains('button', 'Add to cart').should('be.visible').click()
    })
  ...
})

We start spying on *.js network calls before the item component appears. Once it appears we wait for JS resources to stop for at least 40ms.

We don't know which particular JS script reacts to the button click. If we knew, we could have targeted it better. But for now, we wait for all of them:

The cypress-network-idle plugin in action

Burn it

Good. Now we can finish the test and run it multiple times in a row to confirm that it is solid. Following Cypress Flaky Tests Exercises

cypress/e2e/burn.cy.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
// https://github.com/bahmutov/cypress-network-idle
import 'cypress-network-idle'

Cypress._.times(10, (k) => {
  it(`goes to the checkout with one item ${k + 1} / 10`, () => {
    cy.visit('/')
    cy.get('#loadingMessage', { timeout: 15_000 }).should('not.be.visible')
    ...

    cy.get('iframe[title="Frame of demo shop"]')
      .its('0.contentDocument.body')
      .find('#checkout-personal-information-step')
      .should('be.visible')
  })
})

The same test ran 10 times in a row

Seems solid.

Simplify the code

Our testing code is very verbose when accessing the iframe. We have two query chains essentially to access the elements inside the iframe:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
// BODY -> cy.find
cy.get('iframe[title="Frame of demo shop"]')
  .its('0.contentDocument.body')
  .find('.header-top')
  .should('be.visible')
  .contains('a', 'Clothes')
  .click()
// BODY -> cy.then(cy.wrap).contains
cy.get('iframe[title="Frame of demo shop"]')
  .its('0.contentDocument.body')
  // we know the iframe body is there
  // but we need a jQuery object to be able to use cy.contains
  // otherwise we see "subject.each" is not a function error
  .then(cy.wrap)
  .contains('.category-sub-menu a', 'Men')
  .click()

Note: Cypress throws an error if we directly attach cy.contains to the BODY element like this: .its('0.contentDocument.body').contains. Thus we have to use cy.then(cy.wrap) which breaks the retries. Luckily we can first confirm the element is there using cy.find and then look at the text.

Let's add utility function:

cypress/e2e/simple.cy.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
// https://github.com/bahmutov/cypress-network-idle
import 'cypress-network-idle'

const getIframeBody = (wrap = false) => {
  if (wrap) {
    return (
      cy
        .get('iframe[title="Frame of demo shop"]')
        .its('0.contentDocument.body')
        // we know the iframe body is there
        // but we need a jQuery object to be able to use cy.contains
        // otherwise we see "subject.each" is not a function error
        .then(cy.wrap)
    )
  } else {
    return cy
      .get('iframe[title="Frame of demo shop"]')
      .its('0.contentDocument.body')
  }
}

it('goes to the checkout with one item', () => {
  cy.visit('/')
  cy.get('#loadingMessage', { timeout: 15_000 }).should('not.be.visible')
  getIframeBody()
    .find('.header-top')
    .should('be.visible')
    .contains('a', 'Clothes')
    .click()
  getIframeBody().find('.breadcrumb').should('include.text', 'Clothes')
  getIframeBody(true).contains('.category-sub-menu a', 'Men').click()
  getIframeBody().find('.breadcrumb').should('include.text', 'Men')

  cy.waitForNetworkIdlePrepare({
    method: 'GET',
    pattern: '*.js',
    alias: 'js',
  })

  // click on the first product item
  getIframeBody()
    .find('.products .product')
    .should('have.length.greaterThan', 0)
    .first()
    .click()

  // click on the "Add to cart" button
  getIframeBody()
    .find('#main .product-container')
    .should('be.visible')
    .within(() => {
      cy.waitForNetworkIdle('@js', 40)
      cy.contains('.product-add-to-cart', 'Add to cart')
      cy.contains('button', 'Add to cart').should('be.visible').click()
    })

  getIframeBody()
    .find('#blockcart-modal')
    .should('be.visible')
    .contains('a', 'Proceed to checkout')
    .click()

  getIframeBody()
    .find('.cart-grid')
    .should('be.visible')
    .contains('a', 'Proceed to checkout')
    .click()

  getIframeBody()
    .find('#checkout-personal-information-step')
    .should('be.visible')
})

Nice, but we always have the same syntax getIframeBody().find(selector) and getIframeBody(true).contains(selector, text). Let's simplify it.

cypress/e2e/simple2.cy.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
// https://github.com/bahmutov/cypress-network-idle
import 'cypress-network-idle'

const iframeBody = () =>
  cy.get('iframe[title="Frame of demo shop"]').its('0.contentDocument.body')

const iframeFind = (selector) => iframeBody().find(selector)

const iframeContains = (selector, text) =>
  iframeBody()
    // we know the iframe body is there
    // but we need a jQuery object to be able to use cy.contains
    // otherwise we see "subject.each" is not a function error
    .then(cy.wrap)
    .contains(selector, text)

it('goes to the checkout with one item', () => {
  cy.visit('/')
  cy.get('#loadingMessage', { timeout: 15_000 }).should('not.be.visible')
  iframeFind('.header-top')
    .should('be.visible')
    .contains('a', 'Clothes')
    .click()
  iframeFind('.breadcrumb').should('include.text', 'Clothes')
  iframeContains('.category-sub-menu a', 'Men').click()
  iframeFind('.breadcrumb').should('include.text', 'Men')

  cy.waitForNetworkIdlePrepare({
    method: 'GET',
    pattern: '*.js',
    alias: 'js',
  })

  // click on the first product item
  iframeFind('.products .product')
    .should('have.length.greaterThan', 0)
    .first()
    .click()

  // click on the "Add to cart" button
  iframeFind('#main .product-container')
    .should('be.visible')
    .within(() => {
      cy.waitForNetworkIdle('@js', 40)
      cy.contains('.product-add-to-cart', 'Add to cart')
      cy.contains('button', 'Add to cart').should('be.visible').click()
    })

  iframeFind('#blockcart-modal')
    .should('be.visible')
    .contains('a', 'Proceed to checkout')
    .click()

  iframeFind('.cart-grid')
    .should('be.visible')
    .contains('a', 'Proceed to checkout')
    .click()

  iframeFind('#checkout-personal-information-step').should('be.visible')
})

If you want, you can add the iframeFind and iframeContains as custom commands.

See also