Cypress cy.prompt Vs Recording Vs Coding

Cypress-the-company is putting a lot of effort behind its cy.prompt command (which I recreated in cypress-think), so let's see if it is any good.

I took a simple login form and tried to create a test using three different approaches:

• coding in VSCode using "update the test / Cypress reruns it / update the test"
• using cy.prompt command from text
• recording user actions using Cypress Studio

TLDR; only normal coding works reliably

Coding by hand

Here is the "normal" login test: we grab the username (not a secret) and password (secret) and enter them into the login form.

cypress/e2e/spec.cy.js

describe('Login', () => {
  it('logs in', () => {
    cy.visit('public/index.html')
    cy.get('.login-container').should('be.visible')
    cy.get('#username').type(Cypress.expose('username'))
    cy.env(['password']).then(({ password }) => {
      cy.get('#password').type(password, { log: false })
    })
    cy.contains('button', 'Login').click()
    cy.get('.login-container').should('not.exist')
    cy.get('.authenticated-message').should(
      'have.text',
      'Authenticated',
    )
  })
})

Nothing fancy, just simple steps

VSCode Copilot even helps with selectors, like the ".login-container", since it has access to the local "index.html" where the element is defined

Using cy.prompt

Let's describe what we want to do and let cy.prompt command figure it out. We grab the username and the password and pass them as placeholders to the prompt.

describe('Login', () => {
  it('logs in', () => {
    cy.env(['password']).then(({ password }) => {
      cy.prompt(
        [
          'visit public/index.html',
          'assert login container is visible',
          'type {{username}} into the username field',
          'type {{password}} into the password field',
          'click the login button',
          'assert the login container is gone',
          'assert authenticated message "Authenticated" is visible',
        ],
        {
          placeholders: {
            username: Cypress.expose('username'),
            password,
          },
        },
      )
    })
  })
})

Cypress runs ... and fails on the "assert the login container is gone" line.

Hmm, isn't this the same login container as in the "assert login container is visible" line?! Ok, let's delete the problematic line and finish the rest of the test

The test runs ... and cy.prompt fails.

What is happening? Look at the updated spec file: it cleared the prompt line completely, and I guess cy.prompt breaks if any of the lines are empty

cy.prompt(
  [
    'visit public/index.html',
    'assert login container is visible',
    'type {{username}} into the username field',
    'type {{password}} into the password field',
    'click the login button',
    "",
    'assert authenticated message "Authenticated" is visible',
  ],
  {
    placeholders: {
      username: Cypress.expose('username'),
      password,
    },
  },
)

Is anyone testing this stuff?!!

Ok, delete the empty string and run the cy.prompt again.

Click on the "Code" button and check out the generated code. Hmm.

Oh my.

// Prompt step 2: assert login container is visible
cy.get("body").should("be.visible")

Does this look acceptable? How about the inability to use the placeholders, the literals were put into the code!

// Prompt step 3: type {{username}} into the username field
cy.get("[name=\"username\"]").type("cypress")

// Prompt step 4: type {{password}} into the password field
cy.get("[name=\"password\"]").type("rocks")

Tough. The generated prompt requires cleaning up by hand.

Recording test using Cypress Studio

Ok, let's try the last approach: simply interacting with the page and recording the test using the Studio mode. We start with the page

cypress/e2e/record.cy.js

describe('Login', () => {
  it('logs in', () => {
    cy.visit('public/index.html')
  })
})

Pro-tip: you do NOT open Studio for an existing test, instead you click "Edit in Studio" button that only becomes available when you hover next to the test title. User-friendly /s

I enter "cypress" and "rocks" into the input fields and click the "Login" button. Studio shows the generated commands plus recommended assertions. I only accept the last assertion and save the file.

Ok, so what do we see in our code editor?

cypress/e2e/record.cy.js

/// <reference types="cypress" />

describe('Login', () => {
  it('logs in', () => {
    cy.visit('public/index.html')
    // Page URL changed.
    cy.url().should('eq', 'http://localhost:53732/public/index.html')
    // Page title changed. The page title is 'Login'.
    cy.title().should('eq', 'Login')
    // The heading 'Sign In' is displayed.
    cy.get('h2').should('contain.text', 'Sign In')
    // The username input field is visible.
    cy.get('[name="username"]').should('have.attr', 'required')
    // The password input field is visible.
    cy.get('[name="password"]').should('have.attr', 'required')
    // The login button is visible.
    cy.get('button').should('contain.text', 'Login')

    cy.get('[name="username"]').click()
    cy.get('[name="username"]').type('cypress')
    // The username input field now contains the text 'cypress'.
    cy.get('[name="username"]').should('have.value', 'cypress')

    cy.get('[name="password"]').click()
    cy.get('[name="password"]').type('rocks')
    cy.get('button').click()
    // A message 'Authenticated' is displayed.
    cy.get('body').should('contain.text', 'Authenticated')
  })
})

• it inserted all AI-generated assertions, ignoring me not clicking the "Accept" button
• it completely skipped using recommended / unique selectors for the submit button or "Authenticated" element
• it could not generate any negative assertions to confirm that the login form is gone or there are no error messages
• it does not understand secrets and placeholders

Ughh, the recorded test needs a lot of cleaning up

And the winner is... my code editor. It is much faster to simply write the test by hand, watching the test re-run on "Save". If you can use Copilot, it suggests good selectors and code snippets, speeding test coding even more. If you want to see Copilot and Cypress working together, check out my course Write Cypress Tests Using GitHub Copilot. Everything else requires so much cleanup, it is not worth wasting time on.