- Read the docs
- Run Cypress on your own CI
- Record success and failure videos
- Move common code into utility package
- Separate tests into bundles
- Make JavaScript crashes useful
- Use test names when creating data
- Get test status
- Explore the environment
- Run all spec files locally
- Get command log on failure
- Wait on the right thing
- Write and read files
- Read JSON files with retries
- Conditional logic
- Customize Cypress test runner colors
- Shorten assertions
- Disable ServiceWorker
- Control
navigator.language
- Use fixtures to stub network requests
- Use code coverage
- Use visual testing
- Interactive and headed mode
- Produce high quality video recording
- Use the page base url
- Pass the environment variables correctly
- Parse and use URL
- Deal with
target=_blank
- Deal with
window.open
- Minimize memory use
- Start browser with specific time zone
- See also
We have been successfully using Cypress to verify our websites and web apps for a while. Here are some tips & tricks we have learned.
Even this blog is tested using Cypress :)
Read the docs
Cypress has excellent documentation at https://docs.cypress.io/, including examples, guides, videos and even a busy Gitter chat channel. There is also "Search" box that quickly finds relevant documents. The search is really good and covers the docs, the blog posts and the examples.
Recently common recipes were combined into a single place. If you still have an issue with Cypress, please search through open and closed issues.
You can also read my other blog posts about Cypress under tags/cypress.
Run Cypress on your own CI
As long as the project has a project key, you can run the tests on your own CI boxes. I found running inside Docker to be the easiest, and even built an unofficial Docker image with all dependencies.
Note that cypress ci ...
will upload recorded videos and screenshots
(if capturing them is enabled)
to the Cypress cloud, while cypress run ...
will not.
Record success and failure videos
When Cypress team has released the video capture feature in version 0.17.11 it quickly became my favorite. I have configured our GitLab CI to keep artifacts from failed test runs for 1 week, while keeping videos from successful test runs only for a 3 days.
1 | artifacts: |
Whenever a test fails, I watch the failure video side by side with the video from the last successful test run. The differences in the subject under test are quickly obvious.
Move common code into utility package
Testing code should be engineered as well as the production code. To avoid duplicating testing helper utilities, we advise to move all common helpers into their own NPM module. For example, we use local CSS styles with randomized class names in our web code.
1 | <div class="table-1b4a2 people-5ee98"> |
This makes the selectors very long since we need to use prefixes.
1 | cy.get('[class*=people-]') // UGLY! |
By creating a tiny helper function we alleviated the selector head aches.
1 | import {classPrefix} from '@team/cypress-utils/css-names' |
The test code became a lot more readable.
Separate tests into bundles
Using a single cypress/integration/spec.js
file to test a large web
application quickly becomes difficult and time consuming. We have separated
our code into multiple files. Additionally, we keep the source files
in src
folder and build the multiple bundles in cypress/integration
using kensho/multi-cypress tool.
The multi-cypress
tool assumes that Docker and GitLab CI are used to
actually run the tests. Each test will be inside its own "test job", thus
10 spec files can be executed at once (assuming at least 10 GitLab CI
runners are available).
The command to run a specific spec file is
cypress run --spec "spec/filename.js"
by the way.
Make JavaScript crashes useful
As I have said before, the true value of a test is not when it passes, but when it fails. Additionally, even a passing test might generate client-side errors that are not crashes and do not fail the test. For example, we often use Sentry exception monitoring service where we forward all client-side errors.
If a client-side error happens while the E2E Cypress test is running, we need this additional context: which test is executing, what steps were already finished before the error was reported, etc.
By loading a small library send-test-info before each test, we can capture the name of the test, its spec filename and even the Cypress test log as breadcrumbs; this information is then sent to Sentry with each crash. For example, the exception below shows the name of the test when the error was reported
Similarly, the Cypress test steps are logged as breadcrumbs and are sent with the exception allowing any developer to quickly get a sense when the error happens as the test runs.
Use test names when creating data
Often as part of the E2E test we create items / users / posts. I like to put the full test title in the created data to easily see what test created which datum. Inside the test we can quickly grab the full title (which includes the full parent's title and the current test name) from the context.
I also like creating random numbers for additional difference
1 | const uuid = () => Cypress._.random(0, 1e6) |
Get test status
If you use function () {}
as the test's body, or a body of a hook, you can find lots of interesting properties in the this
object. For example, you can see the state of the test: passed or failed. To see them all, place the debugger
keyword and run the test with DevTools open.
1 | afterEach(function () { |
There is an object for the entire test, and for the current hook
The this.currentTest
object contains the status of the entire test
Explore the environment
You can pause the test execution by using debugger
keyword. Make sure
the DevTools are open!
1 | it('bar', function () { |
Run all spec files locally
If you separate Cypress E2E tests into multiple spec files, you soon end up with lots of files. If you need to run the scripts one by one from the command line, I wrote a small utility run-each-cypress-spec.
1 | npm install -g run-each-cypress-spec |
If you need environment variables (like urls, passwords), you can easily inject them using as-a
1 | as-a cy run-specs |
Get command log on failure
While videos of the failed tests are useful, sometimes we just want to get a sense of the failure before triaging it. In the headless CI mode, you can get a JSON file for each failed test with the log of all commands. See the JSON file below as an example of a failed test and the file it generated.
1 | { |
This is a separate project that just needs to be included from your
cypress/support/index.js
file. For instructions see
cypress-failed-log project.
Wait on the right thing
It is important to wait for the right element. Imagine we have a list of items in the DOM. We expect a new element to appear with text "Hello". We could select first the list, then the element containing "Hello" and check if it becomes visible
1 | cy.get('.list') |
But sometimes it does not work. When we cy.get('.list')
Cypress saves the
DOM element as the "subject" and then tries to wait for a child of that
element with text "Hello" to become visible. If we use React for example, the
DOM of the list might be recreated from scratch if we push a new item into
the list. When that happens Cypress notices that the "subject" DOM it holds
a reference to becomes "detached from the DOM" - it becomes an orphan!
A better solution to this problem is to use a composite CSS selector that will grab the list AND the item in a single operation.
1 | cy.contains('.list li', 'Hello') |
This forces Cypress to wait for the list AND list item element without caching
a reference to the DOM element .list
.
You can use advanced CSS selectors to get an element in a single command.
For example instead of cy.get('.something').first()
you could
use cy.get('.something:first-child')
. Similarly,
1 | // get first element |
Write and read files
You can write files to disk directly from Cypress using
cy.writeFile and read an
existing file using cy.readFile.
What if you want to read a file that might not exist? cy.readFile
will fail
the test if file does not exist, thus we need to find a work around.
In my case, the file I would like to load is a JSON of test values useful for Jest-like snapshot testing.
Here is a neat trick. Save the file using cy.writeFile
, for example
cy.writeFile('spanshots.json')
whenever there is something to save.
When reading the file, fetch it through the Cypress code proxy using fetch
Notice how the test spec itself is served by Cypress from URL which looks like
http://localhost:49829/__cypress/tests?p=cypress/integration/spec.js-438
.
Let us "hack" it to load a file that might not exist.
The code below tries to load snapshots file before each test.
If the fetch
call fails, the file does not exist.
1 | let snapshots |
If the fetch
call succeeds, the returned text
will NOT be original JSON!
Instead it will be webpacked module :)
For example, here is the saved file
1 | { |
Here is how built-in Cypress bundler returns it in response to
/__cypress/tests?p=./snapshots.json
- I have shortened the webpack
boilerplate preamble.
1 | (function e(t,n,r){function ...)({1:[function(require,module,exports){ |
Notice that our JSON has been placed into module with id '1' (first line
expression {1:
). The entire bundle registers a stand alone require
function. If we want to get into the actual "module" contents we have to
do the following in our function loadedText(text)
to finish the hack
1 | function loadedText (text) { |
Boom, the object snapshots
has been loaded from the file snapshots.json
which might or might not exist.
Note while I have successfully used the above hack when running Cypress
locally, it was always failing when doing cypress run
in headless mode.
Read JSON files with retries
Cypress cy.readFile command automatically parses JSON files. It also re-reads the file if the assertions on the returned JSON fail. For example, let's validate the number of items stored by the app in its JSON "database" file:
1 | // note cy.readFile retries reading the file until the should(cb) passes |
Even if the application sends the items after a time delay, the cy.readFile(...).should(cb)
combination retries and successfully passes when all four items are found.
Conditional logic
Sometimes you might need to interact with a page element that does not always exist. For example there might a modal dialog the first time you use the website. You want to close the modal dialog.
1 | cy.get('.greeting') |
But the modal is not shown the second time around and the above code will fail. In order to check if an element exists without asserting it, use the proxied jQuery function Cypress.$
1 | const $el = Cypress.$('.greeting') |
The above check is safe.
Customize Cypress test runner colors
Read Cypress Halloween Theme and check out cypress-dark plugin.
Shorten assertions
For cy.location
you can pass the part you are interested in
1 | cy.location().should(location => { |
is the same as
1 | cy.location('pathname').should('equal', '/todos') |
You can use its
to get nested properties easier
1 | cy.get('@login').should(req => { |
is the same as
1 | cy.get('@login').its('response.body.accessToken') |
and this could be expressed more clearly in steps
1 | cy.then(() => { |
You can wait for network request to happen and then check something using cy.then
1 | // notice that we are waiting for @login XHR alias |
Instead we can use .then
after waiting for the network request. It is also helpful to print the message when checking for the token, otherwise command log simply says "expect null to equal null".
1 | cy.wait('@login') |
You don't need to wrap elements just to dynamically use them. For example if you want to test each item from the list
1 | const inputs = [ |
Every Cypress command is automatically inserted into the queue, so you can iterate over the items and use Cypress commands, everything will be queued correctly.
1 | inputs.forEach(input => { |
I prefer cy.first()
to cy.eq(0)
.
Disable ServiceWorker
ServiceWorkers are great - but they can really affect your end-to-end tests by introducing caching and coupling tests. If you want to disable the service worker caching you need to remove or delete navigator.serviceWorker
when visiting the page with cy.visit
.
First, here is the way that does not work - simply deleting the property from the navigator
object.
1 | it('disables it', function () { |
The reason for this is that we are deleting the property from the wrong object. The serviceWorker
is NOT defined on the navigator
, it is defined on its prototype. You can confirm this by using Object.getOwnPropertyDescriptor
method:
1 | Object.getOwnPropertyDescriptor(win.navigator, 'serviceWorker') |
After we delete the property from the right object, the application code will no longer think it can register the service worker.
1 | it('disables it', function () { |
1 | <body> |
Note: once deleted, the SW stays deleted in the window, even if the application navigates to another URL.
Read also "Stub navigator API in end-to-end tests" for another example.
Imagine you have a page that shows different greeting depending on the navigator.language
property.
1 | <body> |
How do we check if the default English greeting is displayed? Easily
1 | it('shows default greeting', () => { |
But how do we force the Klingon greeting to be displayed? Trying to change read-only navigator.language
property throws an error.
1 | it('shows Klingon greeting', () => { |
The navigator.language
property is actually set on navigator.__proto__
object:
1 | Object.getOwnPropertyDescriptor(navigator, 'language') |
We can create language
property on the navigator
object instead - thanks, prototypical inheritance! We can specify the property's value:
1 | it('shows Klingon greeting', () => { |
This test shows proper respect to each Klingon warrior browsing the web.
How do we ensure that the application actually read navigator.language
property when displaying the greeting? Maybe the "nuqneH" is hard-coded! We need to track navigator.language
"get" access. We can do this using cy.stub
method.
1 | it('checks if application gets language property', () => { |
Now we know for sure how the application behaves.
Use fixtures to stub network requests
If your tests are full of stubbed network responses, move the responses into fixtures
before
1 | it('does A', () => { |
after 1
1 | it('does A', () => { |
after 2
You can even require the JSON fixtures directly if the second response is just a little bit different from the first response
1 | const responseFixture = require('../fixtures/first_query_response') |
See cy.route and cy.fixture documentation.
Bonus: read Import Cypress fixtures
Use code coverage
You can instrument your application and use Cypress code coverage plugin to produce combined reports in multiple formats. End-to-end tests are very effective at covering a lot of code in a single test.
Use visual testing
If a test grows to be very long because it checks so many elements on the page, see if it makes sense to test the entire page using Visual Testing.
before
1 | it('checks lots of things', () => { |
after
1 | it('checks lots of things', () => { |
Bonus: check out bahmutov/sudoku for visual component testing using open source tools, and read Visual testing for React components using open source tools.
Interactive and headed mode
You can find out if the test is currently running using the interactive mode which is when the user calls cypress open
.
1 | if (Cypress.config('isInteractive')) { |
When you use the interactive mode you always see the browser, thus the browser is always headed. But when using the cypress run
command, the browser might be headless or headed. You can find out how the browser is displayed:
1 | if (Cypress.browser.isHeaded) { |
When using cypress run
, Electron is headless by default, while Chrome and Firefox browsers are headed by default. You can control the browser though:
1 | npx cypress run --browser chrome --headless |
Trying to pass both --headed
and --headless
CLI parameters raises an error.
Produce high quality video recording
Read the full blog post Generate High-Resolution Videos and Screenshots.
You can find most of the advice below implemented in bahmutov/cypress-movie. To generate better videos from tests
- set video compression to false
- set browser window size to larger value, which should work fine when using headless Chrome on CI
1 | // in your plugins file |
- hide command log
You can "expand" the application under test iframe to cover the entire window
1 | const clearViewport = () => { |
Call this function before the tests.
- use native Chrome Debugger Protocol to take full page screenshots. See "cypress-movie" project.
Use the page base url
Sometimes multiple tests visit a different base url. For example, you might usually go to the index page by setting baseUrl
in the config file
1 | { |
Every test can cy.visit('/')
and get to the index page. But imagine that a suite of tests is trying to visit and test the "About" page. It is located at /about.html
, so every test has to visit it.
1 | describe('About', () => { |
Now, we can refactor the tests to avoid duplication. For example, we could move the cy.visit
into beforeEach
hook.
1 | /// <reference types="cypress" /> |
Works.
Or we could create a utility function:
1 | describe('About', () => { |
It works too.
Here is one more way to (abuse) test configuration feature introduced in Cypress v5.0.0. We will change the baseUrl
in the describe
block to apply to just these tests.
1 | describe('About', { baseUrl: '/about.html' }, () => { |
Notice the { baseUrl: '/about.html' }
parameter in describe
call. We can overwrite multiple config values using this parameter. In our case, we overwrite the baseUrl
, appending the file name. In the cy.visit('')
call we pass an empty string - because we do not want to append /
at the end.
But the tests fail.
While our visit used an empty string, Cypress requires baseUrl
to be a valid base URL and appends /
at the end automatically.
Luckily, Cypress v5.4.0 has a fix for baseUrl
that has params for issue #2101. If your base url is of the form ...?foo=bar
then visiting ''
would keep the original full url. Let's change our test by appending ?
to the base url:
1 | - describe('About', { baseUrl: '/about.html' }, () => { |
Now the tests pass - and the ?/
at the end of the URL is ignored
Pass the environment variables correctly
Imagine you need to pass database username and password from your test. These values are probably available as environment variables. How would you pass them to cypress run
? You might try something like this in the package.json
file. It will NOT work:
1 | { |
First, you do not need the node_modules\\.bin
path when calling an NPM alias - they are automatically resolved by the npm run
command. Thus always use simply:
1 | { |
The above command will NOT work yet. To see why, change the cypress run
to cypress open
to see the Cypress GUI.
1 | { |
Select the "Settings" tab and inspect the resolved environment variables.
We have two problems:
- Instead of passing the environment variable's value, we got the strings "$DB_USERNAME" and "$PASSWORD". Resolving environment variables inside commands might be tricky and depend on the operating system.
- The username and the password are stored under names
db.user
anddb.password
, which might be not what you expect. From the dot notation, I would expect these values to create an objectdb
with propertiesuser
andpassword
.
But there are other trickier problems here. Let's change our NPM scripts to avoid single quotes around the values. We hope that now the environment variables are resolved correctly.
1 | { |
At first, it appears to work
Now, let's try a user name with a space.
Ummm. The space inside the username value created problems; the cypress open
command effectively received ended with --env db.user=joe smith,db.password=123
. The part after --env db.user=joe
was ignored!
Solution: Cypress can grab the environment variables in multiple ways. The most powerful and flexible way is to grab them from the process.env
object using the plugins code.
1 | module.exports = (on, config) => { |
The plugins file runs in Node environment, has access to the process.env
object. It can grab the variables, check their values, and place them into a single object db
. Now the Settings tab shows the correct values.
Additional reading: the blog post Keep passwords secret in E2E tests and the recipe Environment variables.
Tip: when working locally you can use the utility as-a to conveniently inject environment variables when running any command.
Parse and use URL
Let's take a Next.js application with dynamic routes. We can scaffold one using the provided example.
You can find the complete source code at bahmutov/dynamic-routing-app
The app was initialized using the provided example command
1 | npx create-next-app --example dynamic-routing dynamic-routing-app |
The app contains two dynamic routes:
pages/post/[id]/index.js
- e.g. matches
/post/my-example
(/post/:id
)
- e.g. matches
pages/post/[id]/[comment].js
- e.g. matches
/post/my-example/a-comment
(/post/:id/:comment
)
- e.g. matches
Let's see how we can parse the above URLs to use them during tests. First we need to install Cypress and start-server-and-test:
1 | yarn add -D cypress start-server-and-test |
I can scaffold the Cypress files with my helper bahmutov/cly
1 | npx @bahmutov/cly init |
We place the base url localhost:3000
in cypress.json
file and start the app and Cypress. Our first test can confirm the home page loads:
1 | describe('Dynamic routes', () => { |
Great, we can make sure the link "About" goes to the "About" page.
1 | it('goes to the about page', () => { |
The test passes, but with a tiny red flag. Notice little eye crossed icons next to the "contains" command?
The Next.js application fetches a static HTML with the markup present but invisible. It is hydrated later - but we want to click on the link like a real user would, after it becomes visible. Let's add an assertion.
1 | it('goes to the about page', () => { |
Much better.
We assert that we go to the right page by using the expression cy.url().should('match', /\/about$/)
. It works, but the command cy.url returns the full URL. If you click on the command you will see what it yields:
We are interested only in the relative pathname - the /about
part. Thus I suggest using cy.location command that parses the URL and can yield just the interesting part.
Now let's check the first post. We can start with the same test as before
1 | it('goes to the first post', () => { |
Now let's go to the first command. We need to click on the link, but before we do this, let's validate it. Let's grab the current post url and use it somehow in our tests.
1 | it('goes to the first comment', () => { |
After the assertion .should('include', '/post')
passes, the value is yielded to the next command where we print it.
Now we have all the JavaScript magic we need to parse and slice URL. Let's even change the test and go to the second comment of the first post.
1 | it('goes to the first post / second comment', () => { |
We can parse, validate, and use the URL in our test.
Deal with target=_blank
Imagine a link on the page uses <a href="/about.html" target="_blank">About</a>
. Cypress does not work with the second tab, so what can we do?
1 | it('loads the about page', () => { |
Figure out what you want to test. You can for example verify the link has the expected address and attribute target=_blank
and call it a day.
1 | it('loads the about page', () => { |
We can also change the target attribute before clicking the link (but after confirming it to be blank). Then we can verify the second "tab" loads correctly.
1 | it('loads the about page', () => { |
Deal with window.open
Imagine the application under test uses window.open()
to load a new URL in the second tab.
1 | <a href="/about.html" target="_blank">About</a> |
By default the /about.html
page opens in a new tab - and we do not want that. Let's stub the window.open
method instead.
1 | it('opens the about page', () => { |
If we really want to load the new URL, let's call the original window.open
method, passing _self
as the second parameter.
Tip: use <method>.wrappedMethod
to get to the original method wrapped in Sinon stub.
1 | it('opens the about page', () => { |
Minimize memory use
Sometimes Cypress can crash during CI run due to running out of memory. Typically it shows a message like this:
1 | 10:38:58 AM: We detected that the Chromium Renderer process just crashed. |
You can profile memory usage every second with environment variables
1 | export DEBUG=cypress:server:util:process_profiler |
The things you can do to minimize the amount of memory used:
- try running tests without extra reporters, plugins, and preprocessors
- split specs to have fewer tests per spec. Cypress opens and closes the browser for every spec
- turn the video recording off with
video: false
usingcypress.json
or environment variables - turn the command log off using the environment variable
CYPRESS_NO_COMMAND_LOG=1
- expose garbage collection and running it after each test. See issue 8525, but in general for Electron browser you want to use an environment variable
ELECTRON_EXTRA_LAUNCH_ARGS=--js-flags=--expose_gc
and from the test callwindow.gc()
method if available
1 | afterEach(() => { |
Tip: if Electron keeps crushing, try running tests using Chrome or Firefox browsers.
Start browser with specific time zone
You can start Cypress (and thus the browser it spawns) with specific time zone to see how the application handles it.
1 | $ <timezone> npx cypress open |
Typical time zones are America/New_York
, Europe/Berlin
, Europe/London
, Asia/Tokyo
.
Tip: an application can obtain its time zone using the following code snippet
1 | Intl.DateTimeFormat().resolvedOptions().timeZone |
For more information, read Testing Time Zones in Parallel
See also
- Writing a Custom Cypress Command and How to Publish Custom Cypress Command on NPM
- Cypress blog, I have written a large number of blog posts there.
- Notes of Best Practices for writing Cypress tests
- The Hitchhikers Guide to Cypress End-To-End Testing
- Unit testing Vuex data store using Cypress.io Test Runner
- video playlist Cypress Tips & Tricks
- Cypress blog posts by Filip Hric