No one likes flaky tests, not even their mama (you who wrote them). Luckily, most flaky tests come from several specific places: application taking time to process a user action, page re-rendering, etc. I have set up a small repo with several hands-on exercises for you to practice fixing flaky Cypress tests. Can you solve each exercise?
The source code can be found in the repo bahmutov/flaky-test-cypress. Each exercise is in its separate branch: level1, level2, etc. You can get the starting code by cloning the repo and checking out the right branch:
- clone the repo to your local machine
- check out the specific branch, for example
git checkout level1 - install dependencies with
npm install - open Cypress
npx cypress open
The application is a simply form with several input fields.
Run the app.cy.js spec and see the flaky test. Can you fix it? You can only modify the spec source code, no touching the application source code allowed.
level 1
The original flaky test that sometimes passes and sometimes fails. Can you solve the problem before watching the video below?
🎓 Do you like practicing end-to-end test writing using hands-on exercises like this blog post shows? Check out my online courses at cypress.tips/courses that have hundreds of hands-on lessons!
level 2
How do you check if a test has flake? By running it multiple times. How would you run a test 10 or 50 times in a row?
level 3
The test should pass, yet somehow it fails to save the user record. Can you fix it?
level 4
The test types an email into the input box, but sometimes it loses the first couple of characters. Can you fix the test, even if you cannot fix the underlying problem?
level 5
The test clicks on the "Submit" button and occasionally the button is not there, even if you can see it. Why does that happen? How can you write this test better?
level 6
The application might be loading data on startup. If the test does not account for it, it might have weird behavior. Can you make the test wait for the loading to be finished?
level 7
The application saves the data on the server by making the POST /students network call. But sometimes the call seems to be too slow, can you make the test wait for the network call to finish before proceeding?
level 8
If the application includes an iframe that slowly loads, the test must wait with retries. The next video shows how to do it
level 9
If the application is showing a random popup, we investigate how the app saves the "hide the popup" state and set it using the localStorage ourselves at the start of the test.
See also
- the full YT playlist with these flaky test videos
- Solve Flake In Cypress Typing Into An Input Element
- Cypress Flakiness Examples
- Repeat Test to Fight Flake
- Flaky IFrame Online Store Test