Cypress vs Playwright; Browser Included

Recently I watched a pretty good video Cypress vs Playwright side-by-side coding comparison. by Artem Bondar. While not as good as my Cypress vs Playwright course, the video is pretty solid. There is one point Artem makes in the beginning that he did not stress enough, but which determines how each tool gives you the access to the browser and your web application under test.

The browser

Here is the relevant still from the video:

Playwright gives you DOM snapshots

Like Artem says, even in the playwright test --ui mode, what you see is NOT the real browser, but the browser showing the trace of the recorded test. So Artem keeps a separate browser open while testing, looking up selectors, interacting with the app, etc. Cumbersome and inefficient, especially if you need to set up the app into a particular state the test needs.

Cypress gives you the live app in the real browser

On the other hand, Cypress runs its tests in the real browser: Electron, Chrome, Firefox, Webkit. The app is included in its own iframe. You can see the markup right there!

Not just the real DOM thing: you can interact with the application to observe what it does; right here I flip the "Light" switch after the test visits the page.

Let's say I want to test the color themes. I can click on the "Light" button, inspect the theme selector markup, then quickly write the test.

Great, got it. Let's expand our test. To better show the developer experience, I will keep Cypress browser and my VSCode side by side.

Great, the color theme changes from "Light" to "Cosmic", but how can we confirm it? We can simply inspect the application DOM before and after the click.

🎁 You can find both Cy and Pw specs in the repo bahmutov/cy-vs-pw-browser.

Having a real browser showing the app is a super power. Of course, both Cypress and Playwright also have time-traveling debugger showing the DOM for each command, but seeing the real app at the end is efficient.

Can you see the tests run in the real browser in Playwright. Kind of with npx playwright test --debug --ui command. The --debug option opens a headed browser instance, which runs the tests and closes the browser.

Execute test commands one by one

Both Playwright and Cypress give you a way to execute individual test commands one by one. Playwright has page.pause and Cypress has cy.pause command.

Same if you use Playwright Inspect which opens if you use the page.pause method:

Playwright spec

test('counts seconds', async ({ page }) => {
  await page.goto('/')
  await page.pause()
  const timer = page.locator('#timer')
  await expect(timer).toHaveText('0')
  await expect(timer).toHaveText('1')
  await expect(timer).toHaveText('2')
  await expect(timer).toHaveText('3')
  await expect(timer).toHaveText('4')
  await expect(timer).toHaveText('5')
})

I am running PW using the playwright test --ui --debug command. When the test launches, a real browser instance shows, the test is paused and I can step through the commands using the Pw Inspector window:

Cypress has an equivalent cy.pause command that let's you step through the commands one-by-one

Cypress spec

it('counts seconds', { baseUrl: 'http://localhost:9090' }, () => {
  cy.visit('/')
  cy.get('#timer').should('have.text', '0').pause()
  cy.get('#timer').should('have.text', '1')
  cy.get('#timer').should('have.text', '2')
  cy.get('#timer').should('have.text', '3')
  cy.get('#timer').should('have.text', '4')
  cy.get('#timer').should('have.text', '5')
})

In both test runners, pausing the test lets the application continue running; the timer keeps ticking every second. If we really want to observe what the application is doing in response to the test command, we MUST pause both the test and the app. Can we do this?

Debugger in the same event loop

Cypress waits

One other interesting aspect of running Cypress tests and the app in the same browser window is that they share the single JavaScript event loop. Which means if you pause the application code using the debugger keyword, the test code pauses too, as this short video shows.

Note: to pause the app, the DevTools must be opened.

And vice versa: if you use the Cypress command cy.debug while the DevTools is opened, both the tests AND the application pauses. You can inspect the actual app objects, network calls, storage, everything!

Playwright does not wait

Let's see what Playwright does when there is a debugger keyword in the application code. I will run the browser in the debug mode to see if the test pauses while the application is paused:

Nope. While the application is paused, the test runner keeps executing the test and then failing it, since the timer element has not been updated. What about the other way? If the test is paused, does the application stay frozen? We can use VSCode Playwright extension to put a breakpoint on the test line and run the browser in the debug mode.

Nope again. The application "keeps ticking" so to speak while the test is frozen at a breakpoint. Trying to debug an application in this way, while the test runner and the app are two separate processes is ... complicated.

Ok, hope it makes the real difference between Cypress and Playwright a little bit clearer. If you want to learn more, check out my online course Cypress vs Playwright course or even an open-source workshop bahmutov/cypress-workshop-cy-vs-pw.