Install the wick-a11y plugin

First, let's install wick-a11y plugin.

1
2

$ npm i -D wick-a11y
+wick-a11y@3.0.1

Register the plugin's tasks in the Cypress config file

1
2
3
4
5
6
7
8
9
10

import { defineConfig } from 'cypress'
// https://github.com/sclavijosuero/wick-a11y
import addAccessibilityTasks from 'wick-a11y/accessibility-tasks'

export default defineConfig({
  ...
  setupNodeEvents(on, config) {
    addAccessibilityTasks(on, config)
  }
})

Include the plugin's custom commands by importing it from the support file loaded by the browser

1
2

// https://github.com/sclavijosuero/wick-a11y
import 'wick-a11y'

That's it. Now I can confirm the page is accessible by calling cy.checkAccessibility() command.

Check a11y

Let's confirm the Teams page works for everyone. I simply add the accessibility command at the end of the test

1
2
3
4
5
6
7
8
9

it('enables Add button when typing the name', () => {
  cy.visit('/team/')
  cy.get('[data-cy="teams-container"]').should('be.visible')
  cy.contains('button', 'Add team').should('not.exist')
  cy.get('[name=teamName]').type('Name')
  cy.contains('button', 'Add team').should('be.visible')

  cy.checkAccessibility()
})

Right away it shows 1 critical problem: the text "Zero players" has a color that is hard to read on white background.

You can get the error details by clicking on the "Fixme" log line in the Command Log

The fix is simple: use a darker gray color class

1
2
3
4
5
6
7

  <div
    data-cy="zero-teams"
-   class="flex h-24 items-center justify-center text-gray-400"
+   class="flex h-24 items-center justify-center text-gray-500"
  >
    No teams yet
  </div>

Note: this plugin by default checks. If you want to see lower severity errors, enable them when running the command; could be verbose at first.

1
2
3
4
5
6

cy.checkAccessibility(undefined, {
  // fail on critical errors
  includedImpacts: ['critical', 'serious'],
  // only warn for non-critical errors
  onlyWarnImpacts: ['moderate', 'minor'],
})

Our page is missing <main> landmark (and other landmarks, like <footer>), so we get several warnings, highlighted in yellow

Detailed reports

This plugin generates detailed static HTML reports (which you can control). Open the report file from cypress/accessibility folder. Each error has a separate section, showing the HTML snapshot with offending element.

I suggest saving cypress/accessibility folder as a test artifact on your CI. If the cy.checkAccessibility command fails, these reports will make fixing it a breeze.

Here is where TypeScript string template literal types come in very handy. Let's define an ItemId type.

1
2
3
4
5

type ItemId = `m${number}`

const id1: ItemId = 'm123' // valid
const id2: ItemId = 'x123' // invalid, does not start with 'm'
const id3: ItemId = 'mabc' // invalid, does not end with a number

TypeScript immediately complains about "x123" and "mabc" strings - these are NOT item ids. My VSCode editor highlights the errors

We don't have to run a test to know it does not work; static types check tells us about our mistake.

If we have a runtime ID value (which we could load from a network call for example), we can confirm and typecast it using is a <type> syntax. For example, here is utility function to check if a string is an item id:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

type ItemId = `m${number}`

const id1: ItemId = 'm123' // valid

function isItemId(value: string): value is ItemId {
  return /^m\d+$/.test(value)
}

if (isItemId(id1)) {
  console.log(`${id1} is a valid ItemId`)
} else {
  // never happens
  console.log('invalid id1', id1)
}

If we look at the "else" branch, we can see the type inference; TS can safely say that "id1" in the "else" branch has "never" type, meaning this code should be unreachable.

We can go beyond a predicate, we can write a utility function that throws an exception if it is given an invalid string value; and this function tells TypeScript compiler that the input has certain type afterwards.

1
2
3
4
5
6
7
8
9
10
11
12
13

type ItemId = `m${number}`

const id1: ItemId = 'm123' // valid

function assertItemId(id: string): asserts id is ItemId {
  if (!/^m\d+$/.test(id)) {
    throw new Error(`Invalid ItemId: ${id}`)
  }
}

const id2: string = 'm123'
assertItemId(id2) // Type assertion
// Now id2 is treated as ItemId

What about other strings, like phone numbers? Let's say that every US phone number we have to store must be fully formatted. We can define a type:

1
2
3
4
5

type PhoneNumber = `+1-${number}-${number}-${number}`

const phone1: PhoneNumber = '+1-123-456-7890' // valid
const phone2: PhoneNumber = '123-456-7890' // invalid, missing country code
const phone3: PhoneNumber = '+1-123-4567-890' // invalid, incorrect format, NOT DETECTED

Looks good, but notice my comment on the phone3 - TypeScript does not flag its incorrect format.

Hmm, we do detect the "abridged" phone number strings, yet TS "missed" the obvious problem. Even worse, TS will miss completely wrong numbers!

1
2
3
4

type PhoneNumber = `+1-${number}-${number}-${number}`

const phone1: PhoneNumber = '+1-123-456-7890' // valid
const phone2: PhoneNumber = '+1-1-2-3' // no TS errors!

Look at the PhoneNumber type definition: +1-${number}-${number}-${number} says nothing about how many digits each number part should have. Thus the string "+1-1-2-3" satisfies the type, yet it is a bad phone number string.

Let's step back for a second and look at the bank card codes or credit card CVV codes. We can define a type for a string that is just 4 digits in a row:

1
2
3
4
5
6
7

type digit = '0' | '1' | '2' | '3' | '4' | '5' | '6' | '7' | '8' | '9'

type PinCode = `${digit}${digit}${digit}${digit}`

const pin1: PinCode = '1234' // valid
const pin2: PinCode = '123' // invalid, too short
const pin3: PinCode = '12345' // invalid, too long

Look at the TS error given for pin2 and pin3

TypeScript does NOT tell us "123 is too short" or "123 does not match digit+digit+digit+digit", instead it shows us how such string literal type works. It simply "expands" every possible combination. String type PinCode is a union of ALL strings like 0000 | 0001 | 0002 | ... | 9999. Wow, brute force works!

Ok, let's use the digit type to form a better phone number type.

1
2
3
4
5
6
7
8
9

type digit = '0' | '1' | '2' | '3' | '4' | '5' | '6' | '7' | '8' | '9'

// more accurate phone number type DOES NOT WORK
type PhoneNumber =
  `+1-${digit}${digit}${digit}-${digit}${digit}${digit}-${digit}${digit}${digit}${digit}`

const phone1: PhoneNumber = '+1-123-456-7890' // valid?
const phone2: PhoneNumber = '123-456-7890' // invalid?
const phone3: PhoneNumber = '+1-123-4567-890' // invalid?

Does the above code work? I don't see any TS errors, hmm. What is the PhoneNumber type, let's hover over it

There are too many combinations for the PhoneNumber string literal type, so TS does NOT expand it into +1-000-000-0000 | +1-000-000-0001 | ... - there would be way too many strings in that union to keep track. From my checks, anything longer than 4 digits is not handled, so US 5-digit zip codes are out.

Ok, so we saw string template literal types, how they work under the hood, and even their limitations. One more note, if you need to check how your web application handles invalid inputs, you do NOT need to special types. You can simply use strings. Let's confirm that entering an invalid phone number leads to an error:

1
2
3
4
5
6
7
8
9
10
11
12
13

const invalidNumbers: string[] = [
  '123-456-7890',
  '+1-123-4567-890',
  '+1-1-2-3'
]
// confirm that entering invalid phone number
// leads the app to show an error message
invalidNumbers.forEach(number => {
  cy.get('#error').should('not.exist')
  cy.get('#phone').clear().type(number)
  cy.get('#error').should('be.visible')
  cy.get('#phone').clear()
})

The above Cypress test simply verifies that entering a bad phone number string is handled by the app. Aside from this test, your testing code probably should only accept PhoneNumber. If you must make an exception (for checking the error handling for example), you can be explicit and use @ts-expect-error when calling it:

1
2
3
4
5
6
7
8

const ItemPage = {
  visit(id: ItemId, errorPage: boolean = false) {
    ...
  }
}

// @ts-expect-error - confirm the error is shown for non-existent item ids
ItemPage.visit('invalid-item-id', true)

Without @ts-expect-error our TS compiler would not let us pass non-item id string into the page object method ItemPage.visit, but we do want to pass it.

Learn more

We can go beyond string literal types into branded types to represent things like currencies and time durations. I also suggest reading 6 TypeScript Tips to Write Safer, Cleaner Code article. Finally, if you are new to TypeScript, check my blog post Trying TypeScript.

Looking at the changes, I noticed a missing area: it is not easy to pass public variables (non-secrets) to Cypress tests. I used a lot of "configuration" values that were meant to be used inside the spec files, and now these values are private by default. For example, let's say we want to run a test only on CI. We could do something like this:

1
2
3
4
5
6
7
8
9

// only run these tests on CI
if (Cypress.env('ci')) {
  it('has CI variable', () => {
    expect(Cypress.env('ci'), 'CI variable').to.equal(true)
    expect(Cypress.env('ciName'), 'CI name variable').to.equal(
      'GitHub Actions',
    )
  })
}

In Cypress before v15 we could have set / used "ci" and "ciName" via environment variables, for example, if using Cypress github action:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

name: ci
on: push

jobs:
  test:
    runs-on: ubuntu-24.04
    steps:
      - name: Checkout 🛎
        # https://github.com/actions/checkout
        uses: actions/checkout@v6

      - name: Run tests 🧪
        # https://github.com/cypress-io/github-action
        uses: cypress-io/github-action@v7
        env:
          # default CI variable
          CYPRESS_ci: true
          CYPRESS_ciName: 'GitHub Actions'

The test runs, Cypress.env() returns { ci: true, ciName: 'GitHub Actions' }, everyone is happy. But in Cypress v16 the process environment variables that start with CYPRESS_ are automatically added to the private cy.env space, and are not accessible outside test blocks / hooks. We could still pass them, but now we need to use a complete custom cypress run --expose ... command, instead of letting Cypress GitHub Action do its thing!

This is why I wrote cypress-expose plugin. It grabs every CYPRESS_EXPOSE_... process environment variable and sets it into Cypress.expose object, letting the specs access it. This is meant for public data only! Simply add plugin to your setupNodeEvents callback function and pass the config object.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

const { defineConfig } = require('cypress')
const cypressExpose = require('cypress-expose')

module.exports = defineConfig({
  allowCypressEnv: false,
  e2e: {
    // baseUrl, etc
    supportFile: false,
    fixturesFolder: false,

    setupNodeEvents(on, config) {
      cypressExpose(config)

      // IMPORTANT: return the config object
      // to let Cypress know we modified it
      return config
    },
  },
})

Boom, you are all set. Now any CYPRESS_EXPOSE_... variable is available under camel-cased name. Let's say we run Cypress using the following command:

1
2
3

$ CYPRESS_API_KEY=123secret! \
  CYPRESS_EXPOSE_ANSWER=42 \
  cypress open --env PASS_WORD=my-password --expose USER_NAME=Joe

The api key and the password are going to be reachable via cy.env command, while the answer and the username are publicly accessible using Cypress.expose:

The EXPOSE_ANSWER value in the cy.env is null, the plugin overwrites it, but cannot delete it (since Cypress merged any modifications to the env object with the original values). I feel like having null there is enough.

Take the plugin for a spin, let me know if it works for you.

First, let me say: Cypress end-to-end browser tests run in the browser. This is different from Playwright and different from how other test runners execute tests. When you run cypress open or cypress run, the Cypress Electron-based binary starts the Node process, launches the browser, loads the e2e spec into the browser, and lets it run. So let's imagine we have a few environment variables that we are going to use during the test:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

import { defineConfig } from 'cypress'

export default defineConfig({
  e2e: {
    env: {
      username: 'joe@acme.co',
      password: 'secret!',
      apiToken: 'abc123$v101'
    },
    setupNodeEvents(on, config) {
      on('task', {
        checkUserSession({ username, token }) {
          // check DB using { username, token }
        }
      })
    }
  },
})

The above env block passes variables to be used by the test, which could be something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

it('logs in', () => {
  cy.visit('/')
  cy.get('form')
    .within(() => {
      cy.get('#username').type(Cypress.env('username'))
      cy.get('#password').type(Cypress.env('password'))
      cy.get('#submit').click()
    })
  // there should be user session on the backend
  cy.task('checkUserSession', {
    token: Cypress.env('apiToken'),
    username: Cypress.env('username')
  })
})

Ok, great, so where are these values "username", "password", and "apiToken"? Are they only in the Node process, the browser process, or both? In Cypress v15, these values are in both! Here is how you can visualize where the variables are:

Hmm, ok, so the spec has all these values. What about the application under test? What about every 3rd-party script loaded by the web app under test? Turns out, every app under Cypress test can access everything inside the global Cypress object:

1
2
3
4
5
6
7
8

if (window.Cypress) {
  console.log(window.Cypress.env())
  // {
  //   "username": "joe...",
  //   "password": "123...",
  //   "apiToken": "abc..."
  // }
}

The above piece of code inside the website will print the username, the password, and the api token used by the test runner. Hmm, is this a good idea? I do not know why Cypress sets window.Cypress global object, it could simply set window.Cypress = true to let the app know that it is being tested, but there was no need to set the entire object.

Ok, so there are 3 different types of environment variables I see in our config file.

• username is something the test must enter into the input field. It does not seem to be very secret, it could be a public value.
• password is something we probably want to hide from the application code, but something the spec running in the browser needs to get at some point.
• apiToken is NOT needed inside the browser, it could reside inside the Node.js Cypress task code. It should be kept secret from the web application, and sending it into the browser seems dangerous. Keep it inside the config Node.js process.

Refactor for safety

Even in Cypress v15 we can use the following best practice for keeping secrets really secret:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

import { defineConfig } from 'cypress'

export default defineConfig({
  e2e: {
    env: {
      username: 'joe@acme.co',
    },
    setupNodeEvents(on, config) {
      on('task', {
        getSecret(name) {
          // we assume that we pass all real secrets
          // via process.env properties
          return process.env[name]
        },
        checkUserSession({ username }) {
          // check DB using { username, token: process.env.API_TOKEN }
        }
      })
    }
  },
})

Then the test can ask for the password value when needed, and never even ask for the "apiToken" value! The test can also "ask" for secrets using cy.task('getSecret') command, when needed. Here is the rewritten test.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

it('logs in', () => {
  cy.visit('/')
  cy.get('form')
    .within(() => {
      cy.get('#username').type(Cypress.env('username'))
      cy.task('getSecret', 'USER_PASSWORD')
        .then(password => {
          cy.get('#password').type(password)
        })
      cy.get('#submit').click()
    })
  // there should be user session on the backend
  cy.task('checkUserSession', {
    username: Cypress.env('username')
  })
})

Just don't name any secrets using CYPRESS_ prefix to avoid accidentally putting the value into Cypress.env object! We start Cypress with process environment variables set like this:

1

USER_PASSWORD=secret! API_TOKEN=abc123$v101 cypress run

💡 You can use my CLI utility as-a to inject multiple blocks of environment variables at once.

If an application tries to print Cypress.env() object, it sees just the "username", since everything else is "kept" inside the Node.js process env object and not bundled into the browser spec.

1
2
3
4
5
6

if (window.Cypress) {
  console.log(window.Cypress.env())
  // {
  //   "username": "joe...",
  // }
}

Cypress v16 cy.env command

So here is the big change coming to Cypress v16: instead of writing custom task "getSecret", there is a new command cy.env for retrieving the secret on demand. Your Cypress config file splits all variables into "expose" (public) and "env" (secrets) blocks:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

import { defineConfig } from 'cypress'

// Cypress v16
export default defineConfig({
  e2e: {
    expose: {
      username: 'joe@acme.co',
    },
    env: {
      password: 'secret!'
    },
    setupNodeEvents(on, config) {
      on('task', {
        checkUserSession({ username }) {
          // check DB using { username, token: process.env.API_TOKEN }
        }
      })
    }
  },
})

Do you need the username? You can get it immediately using the new Cypress.expose static method. You want to hear a secret? Call cy.env command.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

const username = Cypress.env('username')

it('logs in', () => {
  cy.visit('/')
  cy.get('form')
    .within(() => {
      cy.get('#username').type(username)
      cy.env(['password'])
        .then(({ password }) => {
          cy.get('#password').type(password)
        })
      cy.get('#submit').click()
    })
  // there should be user session on the backend
  cy.task('checkUserSession', {
    username
  })
})

Any process env values that start with CYPRESS_ prefix are automatically set to the env block, so they are considered secrets. So you can overwrite the password specified in the Cypress config file:

1
2
3
4

# overwrite using CYPRESS_ env variable
CYPRESS_password=newSecret! npx cypress run
# overwrite using --env CLI
npx cypress run --env password=newSecret!

You can also overwrite the non-secret values using the new CLI option --expose (there is no process env setting to overwrite an exposed value)

1

npx cypress run --expose username=gb@acme.com --env password=gbPass

Migrating from Cypress.env for everything might take some time. I have a few plugins that I am migrating right now, but overall it is not too bad. Just think what you really need for your test, and if the values are secrets to be used in the browser vs secrets that should never go into the browser.

• For non-secret values, use the expose object and the Cypress.expose static method.
• For secrets needed by the browser test, use the cy.env command to get the value when needed
• For "background" secrets that are not meant to be used inside the browser at all, use the process.env object and access from the Cypress config and the setupNodeEvents callback

Stay safe out there!

We could use publish to NPM using local npm CLI commands, entering the 2FA token, etc. But I really hate this idea. I have more than 400 NPM packages, so the release process MUST be automated and be performed by CI. So let's look at the trusted publishing where NPM "knows" that a particular workflow from GitHub Actions is allowed to publish (and no one else). I already use trusted publishing to publish NPM packages cypress-split and cypress-map, so it should be simple to apply the same steps to cypress-timestamps.

📺 You can watch the steps from this blog post explained in this video I recorded after writing the blog post.

First, go to the package settings under your NPM registry account.

The top settings section is for configuring the trusted publishing. Seems both GitHub Actions and GitLab CI providers can be configured as of this writing (February 2026). I am using GHA, so I will click "GitHub Actions" button.

⚠️ Setting up trusted publishing requires an existing NPM package, thus the very first version should be published from your terminal command line using the npm publish command. If this is the very first package version, it might not be even scraped yet, so NPM search returns nothing. In that case, simply go to the last packages page on your profile to find the newly created package.

Enter the GitHub username (organization name) and the repository name, and the name of the workflow file (inside the .github/workflows folder). In my case, I am pointing at the ci.yml file.

Enter the 2FA token if needed

You should see the "success" banner.

Great, let's now update the ci.yml file. We can remove the old NPM token and bump the semantic release. For clarity, I added comments to the workflow code to explain each step

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

name: ci
on: push

jobs:
  test:
    runs-on: ubuntu-24.04
    permissions:
      # allow the release step to comment on
      # issues, pull requests, and write to the repo
      contents: write
      issues: write
      pull-requests: write
      # Required for OIDC to let NPM registry know
      # that this workflow is trusted
      id-token: write

    steps:
      # important: the default Node version is v18
      # which gives us an auth error trying to release
      # so install v24 explicitly
      - uses: actions/setup-node@v6
        with:
          node-version: 24

      - name: Checkout 🛎
        # https://github.com/actions/checkout
        uses: actions/checkout@v6

      - name: Run tests 🧪
        # https://github.com/cypress-io/github-action
        uses: cypress-io/github-action@v6

      - name: Check type declarations 🔎
        run: npm run typecheck

      - name: Semantic Release 🚀
        # https://github.com/cycjimmy/semantic-release-action
        uses: cycjimmy/semantic-release-action@v6
        with:
          branch: main
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

I try to limit the GITHUB_TOKEN permissions to each job, if possible. The important GHA to NPM registry bit is the id-token: write that lets NPM "know" that this CI workflow is legit and is allowed to publish new NPM versions. We can see the successful GHA workflow

We can see the new package version on NPM

Nice.

But sometimes a weird thing happens: one or more checkboxes remain unchecked!

📺 Watch the examples in this blog post explained in my video.

You start looking at the test code. Looks ok, right?

1
2
3
4
5
6
7
8
9
10
11
12

it('are completed by checking the boxes', function () {
  const todos = '.todo-list li'
  cy.visit('/')
  cy.get(todos).should('have.length', this.n)
  // complete all todos by clicking the checkboxes
  cy.get('.todo-list li .toggle').click({ multiple: true })
  // confirm all todos are marked as completed
  // after reloading the page
  cy.reload()
  cy.get('.loaded')
  cy.get('[data-cy="remaining-count"]').should('have.text', '0')
})

This test starts by creating a random number of Todo items, something I covered in the "Cypress Vs Playwright" online course available at cypress.tips/courses.

Ok, there seems to be some flake, and we know how to solve end-to-end testing flake pretty well. We start adding assertions, trying to confirm that our commands have finished successfully before reloading the page.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

it('are completed by checking the boxes', function () {
  const todos = '.todo-list li'
  cy.visit('/')
  cy.get(todos).should('have.length', this.n)
  // complete all todos by clicking the checkboxes
  cy.get('.todo-list li .toggle').click({ multiple: true })

  // confirm all todos are done
  cy.get('[data-cy="remaining-count"]').should('have.text', '0')

  // confirm all todos are marked as completed
  // after reloading the page
  cy.reload()
  cy.get('.loaded')
  cy.get('[data-cy="remaining-count"]').should('have.text', '0')
})

We inserted the "0 todos" check before the cy.reload command, and ... it did not solve the problem!

What is happening, how can the same assertion pass once, but then immediately fail? The clue is in the number of network calls shown int the Command Log. In this instance, we have 4 todos to complete. Yet, there are only 3 "PATCH" network calls!

This is typical web app behavior: update the local state immediately, and send the update to the backend. But what happens if the app does not have time to send the network call before the page reloads? The network call does not happen, and the backend does not "see" one or more "PATCH" network calls. Checking just the page UI using cy.get('[data-cy="remaining-count"]').should('have.text', '0') does not solve the problem: the problem is that the backend still has not been updated.

There is no way for a testing framework to "know" that the application has scheduled setTimeout to make a network call and "wait" for it. Well, you could have some flag or global queue of timers in your app, and Cypress could look it up. But 99.99% of web apps would not do this overhead just for testing, so we have to work with what we got. Luckily, the problem is pretty manageable.

We can solve this issue in several ways.

Check the network call count

Using the incredibly powerful cy.intercept command, we can "continue" the test after N network calls happen.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

it('are completed by checking the boxes (N network calls)', function () {
  const todos = '.todo-list li'
  cy.visit('/')
  cy.get(todos).should('have.length', this.n)

  cy.intercept('PATCH', '/todos/*').as('updateTodo')

  // complete all todos by clicking the checkboxes
  cy.get('.todo-list li .toggle').click({ multiple: true })

  // confirm all network calls have finished
  cy.get('@updateTodo.all').should('have.length', this.n)

  // confirm all todos are marked as completed
  // after reloading the page
  cy.reload()
  cy.get('.loaded')
  cy.get('[data-cy="remaining-count"]').should('have.text', '0')
})

We spy on the PATCH /todos/* calls using the cy.intercept command before we start clicking. Then we check the count of intercepts calls using the cy.get('@updateTodo.all').should('have.length', this.n) command and assertion combo, which retries. Even if the app takes a few seconds to fire the network call, the test will be flake-free.

Observe each network call

Instead of using .click({ multiple: true }), we could click each box individually and confirm its network calls finished. I like this approach even more than the "click multiple elements" option, since it works for any action, not just cy.click, For example, let's use cy.check command, which does not have multiple: true option.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

it('are completed by checking the boxes (one at a time)', function () {
  const todos = '.todo-list li'
  cy.visit('/')
  cy.get(todos).should('have.length', this.n)

  cy.intercept('PATCH', '/todos/*').as('updateTodo')

  cy.get('.todo-list li .toggle').each(($el) => {
    cy.wrap($el, { log: false }).check()
    // confirm a single network call has finished
    cy.wait('@updateTodo')
  })

  // confirm all todos are marked as completed
  // after reloading the page
  cy.reload()
  cy.get('.loaded')
  cy.get('[data-cy="remaining-count"]').should('have.text', '0')
})

We are using cy.each command. It gives us each element as a jQuery object. To properly click it using Cypress, simply cy.wrap silently to avoid Command Log noise and run the cy.check, followed by cy.wait command with a network alias - it waits for 1 network call. If you want to confirm the call was successful, add an assertion.

1
2
3
4
5
6
7

cy.get('.todo-list li .toggle').each(($el) => {
  cy.wrap($el, { log: false }).check()
  // confirm a single network call has finished successfully
  cy.wait('@updateTodo')
    .its('response')
    .should('have.property', 'statusCode', 200)
})

Ask the server

If we are not sure when the application has finished updating the backend, why not ask the backend directly? We can ping the backend ourselves, just like the app does using the cy.request command. To ping the server multiple times until all todos are completed, we can use my plugin cypress-recurse:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

import { recurse } from 'cypress-recurse'

it('are completed by checking the boxes (check the backend)', function () {
  const todos = '.todo-list li'
  cy.visit('/')
  cy.get(todos).should('have.length', this.n)

  cy.get('.todo-list li .toggle').each(($el) => {
    cy.wrap($el, { log: false }).check()
  })

  // confirm the backend has only completed todos
  recurse(
    () => cy.request('/todos').its('body'),
    (todos) => todos.every((todo) => todo.completed),
    {
      log: 'All todos are completed on the server',
      timeout: 30_000,
      delay: 1000,
    },
  )

  // confirm all todos are marked as completed
  // after reloading the page
  cy.reload()
  cy.get('.loaded')
  cy.get('[data-cy="remaining-count"]').should('have.text', '0')
})

The important part is the recurse call with two functions: fetching the list of todos and the predicate to know when to stop pinging:

1
2
3
4
5
6
7
8
9
10

recurse(
  () => cy.request('/todos').its('body'), // produce the values
  (todos) => todos.every((todo) => todo.completed), // check the value
  // options: how many times to check, how long to wait between the checks, etc
  {
    log: 'All todos are completed on the server',
    timeout: 30_000,
    delay: 1000,
  },
)

The recurse calls the first function repeatedly, passes the yielded value into the predicate, and stops the iteration when the predicate returns a truthy value. We use 1 second delays for clarity, and I slowed down the web app to space out network calls. You can see several REQUEST /todos commands - this is our check working

Map chain

Finally, what if you want to check the list of updated todos? We need their ids, so our iteration must produce a list of numbers. But cy.each yields the original list of elements, not a custom value.

Plugin cypress-map to the rescue! It has cy.mapChain command where you can do something with each element, but then produce new values for the next command in the chain. Once we get all completed ids, we can compare it with the list from the beforeEach where we saved the created todo ids

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

import 'cypress-map'

beforeEach(function createRandomTodos() {
  const n = Cypress._.random(1, 3)
  ...
  // save created ids for later
  cy.wrap(todos.map((t) => t.id)).as('ids')
})

it('are completed by checking the boxes (collect their ids)', function () {
  const todos = '.todo-list li'
  cy.visit('/')
  cy.get(todos).should('have.length', this.n)

  cy.intercept('PATCH', '/todos/*').as('updateTodo')
  cy.get('.todo-list li .toggle')
    .mapChain(($el) => {
      cy.wrap($el, { log: false }).check()
      // from each network call, grab the id of the updated todo
      cy.wait('@updateTodo').its('response.body.id')
    })
    .should('deep.equal', this.ids)

  // confirm all todos are marked as completed
  // after reloading the page
  cy.reload()
  cy.get('.loaded')
  cy.get('[data-cy="remaining-count"]').should('have.text', '0')
})

The command .mapChain(fn) runs the commands inside the function and collects all yielded values into an array that it will yield to the next assertion. Thus our code works, here is the relevant part:

1
2
3
4
5
6

.mapChain(($el) => {
  cy.wrap($el, { log: false }).check()
  // from each network call, grab the id of the updated todo
  cy.wait('@updateTodo').its('response.body.id')
})
.should('deep.equal', this.ids)

You can see the original ids in the "BEFORE EACH" hook, and you can see the array assertion inside the test; these are the same ids, and the order is correct.

Nice. I must say these tests and how they interact with this example application are better shown than explained in a blog post. I will record a video showing these tests in action and will post on my youtube.com/@gleb channel, stay tuned.

Of course, each CI allows you to cache dependencies between the jobs - but why do you need to even think about it? You want to cache all dependencies the very first time npm ci runs for the given package.json or package-lock.json file, not the first time each the workflow runs! Most CIs let you control the container image used for running tests; I assume you use one of Cypress Docker images or one of Cypress Docker images with browsers installed. So what if we could create our own Docker image based on Cypress (or any appropriate Docker base image you might want) and run npm ci in that image, and then use that Docker image to simply run our current tests?

I had the idea of caching separately PROD and DEV images inside Docker containers a loooong time ago, see the bahmutov/double-docker repo. In this blog post, I will show a simple approach that works for repositories with tests only, the situation described in the blog post Separate Application And Tests Repos GitHub Actions Setup. We will use GitHub Actions and the public Docker registry.

In the nutshell:

• whenever the user changes package.json or Dockerfile, we build a new Docker image with the node_modules and Cypress binary folders (but no other source code)
• we push the built Docker image to the registry
• every CI run simply pulls that Docker image and checks out the tests into the container
• the tests run immediately after the checkout step, since there is nothing to install

Let's see the code

• Dockerfile
• Workflow
  • Build and push
  • Test
• Run on GitHub Actions
• Using The Cypress Action
• GitHub Container Registry
• Google Artifact Registry

🎁 I have the working example in the public repo bahmutov/cypress-tests-image.

Dockerfile

First, let's look a the Dockerfile

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

# pick the image to build from
# either the base or the browsers image

# https://hub.docker.com/r/cypress/base/tags
FROM cypress/base:24.12.0

# https://hub.docker.com/r/cypress/browsers/tags
# FROM cypress/browsers:node-24.12.0-chrome-143.0.7499.169-1-ff-146.0.1-edge-143.0.3650.96-1

# diagnostics
RUN echo "node -v"
RUN echo "npm -v"

# copy ONLY the package.json and package-lock.json files
WORKDIR /e2e
COPY package.json package-lock.json ./

# install npm dependencies
# and put the Cypress binary in the local subfolder
# https://on.cypress.io/installation
ENV CYPRESS_CACHE_FOLDER=/e2e/cypress_cache
RUN npm ci

# verify Cypress installation
RUN npx cypress verify

# the Docker image should have all Cypress OS dependencies installed
# plus inside the "/e2e" folder
# we will have
#   - node_modules with Cypress installed
#   - cypress_cache with the Cypress binary

Tip: you can use yarn.lock or any other lock file with this approach.

I am naming my image bahmutov/cy:<tag>, you can find them at the Docker hub. Note that it is important to build the image on the same OS architecture as the CI machines, in my case it will be ubuntu-latest.

Workflow

The first step in the workflow computes the combined checksum of package.json and Dockerfile files. It also checks if the Docker image tagged with this checksum exists already using action tyriis/docker-image-tag-exists.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

name: CI
on: push

jobs:
  # computes the hash of package.json and and stores it in the output
  # also checks if the Docker image with this tag already exists
  # outputs:
  #   hash: the package.json hash
  #   tag: whether the Docker image with this tag already exists, "found" or "not found"
  package-hash:
    runs-on: ubuntu-latest
    outputs:
      hash: ${{ steps.hash.outputs.checksum }}
      tag: ${{ steps.tag-exists.outputs.tag }}
    steps:
      - name: Checkout 🛎️
        # https://github.com/actions/checkout
        # only needed to get package.json and Dockerfile to compute the hash
        uses: actions/checkout@v6
        with:
          sparse-checkout: |
            package.json
            Dockerfile

      - name: Package.json + Dockerfile checksum
        id: hash
        run: echo "checksum=${{ hashFiles('package.json', 'Dockerfile') }}" >> $GITHUB_OUTPUT

      # https://github.com/tyriis/docker-image-tag-exists
      - name: Check if Docker image tag exists
        id: tag-exists
        uses: tyriis/docker-image-tag-exists@v2.1.0
        with:
          registry: docker.io
          repository: bahmutov/cy
          # The container image tag
          tag: ${{ steps.hash.outputs.checksum }}

      - name: Report the check results
        # print the tag result into Github Actions summary
        run: |
          echo "## Docker image check" >> $GITHUB_STEP_SUMMARY
          echo "Package.json + Dockerfile hash: ${{ steps.hash.outputs.checksum }}" >> $GITHUB_STEP_SUMMARY
          echo "Docker image bahmutov/cy:${{ steps.hash.outputs.checksum }} **${{ steps.tag-exists.outputs.tag }}**" >> $GITHUB_STEP_SUMMARY

Build and push

Great, let's build the Docker image if one is missing. We could have a job with every internal step using a condition:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

build-docker-image:
  # builds the Docker image and pushes it to the Docker hub
  # but only if it does not exist yet
  runs-on: ubuntu-latest
  needs: package-hash
  steps:
    - name: Checkout 🛎️
      if: ${{ needs.package-hash.outputs.tag == 'not found' }}
      # https://github.com/actions/checkout
      uses: actions/checkout@v6

    - name: Log in to Docker Hub
      if: ${{ needs.package-hash.outputs.tag == 'not found' }}
      # https://github.com/docker/login-action
      uses: docker/login-action@f4ef78c080cd8ba55a85445d5b36e214a81df20a
      ...

But I think there is a slightly nicer way. We can control the entire job by the needs.package-hash.outputs.tag value and skip it, if the image exists:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

name: CI
on: push

jobs:
  # computes the hash of package.json and and stores it in the output
  # also checks if the Docker image with this tag already exists
  # outputs:
  #   hash: the package.json hash
  #   tag: whether the Docker image with this tag already exists, "found" or "not found"
  package-hash:
    runs-on: ubuntu-latest
    outputs:
      hash: ${{ steps.hash.outputs.checksum }}
      tag: ${{ steps.tag-exists.outputs.tag }}
    steps:
      ...

  build-docker-image:
    # builds the Docker image and pushes it to the Docker hub
    # but only if it does not exist yet
    runs-on: ubuntu-latest
    needs: package-hash
    if: ${{ needs.package-hash.outputs.tag == 'not found' }}
    steps:
      - name: Checkout 🛎️
        # https://github.com/actions/checkout
        uses: actions/checkout@v6

      - name: Log in to Docker Hub
        # https://github.com/docker/login-action
        uses: docker/login-action@f4ef78c080cd8ba55a85445d5b36e214a81df20a
        with:
          username: ${{ secrets.DOCKER_USERNAME }}
          password: ${{ secrets.DOCKER_PASSWORD }}

      # we could also use the action
      # https://github.com/docker/build-push-action

      - name: Build docker image
        run: docker build -t bahmutov/cy:${{ needs.package-hash.outputs.hash }} .

      - name: Push Docker images
        run: docker push bahmutov/cy:${{ needs.package-hash.outputs.hash }}

Test

Now that we have Docker image build (if needed), let's use it. We can define another job that simply uses the bahmutov/cy:<tag> container, but it cannot depend on the build-docker-image job - if the job is skipped on GitHub Actions, any job that depends on it will be skipped too. Luckily, there is an easy fix: simply have yet another job that will simply "ping" build-docker-image job status. Once the build-docker-image job finishes or is skipped, our "ping" job resolves. The "ping" is done using lewagon/wait-on-check-action 3rd-party action. Here is how the last part of the workflow looks:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57

name: CI
on: push

jobs:
  # computes the hash of package.json and and stores it in the output
  # also checks if the Docker image with this tag already exists
  # outputs:
  #   hash: the package.json hash
  #   tag: whether the Docker image with this tag already exists, "found" or "not found"
  package-hash:
    ...
  build-docker-image:
    # builds the Docker image and pushes it to the Docker hub
    # but only if it does not exist yet
    runs-on: ubuntu-latest
    needs: package-hash
    if: ${{ needs.package-hash.outputs.tag == 'not found' }}
    ...

  wait-for-build:
    # a trick to allow other jobs to run, even if the "build" job is skipped
    # runs in parallel with the "build" job and keeps checking if it is finished
    # or is skipped
    runs-on: ubuntu-latest
    needs: package-hash
    steps:
      - name: Wait for the Docker image build / skip
        # https://github.com/lewagon/wait-on-check-action
        uses: lewagon/wait-on-check-action@v1.4.1
        with:
          ref: ${{ github.ref }}
          check-name: build-docker-image
          repo-token: ${{ secrets.GITHUB_TOKEN }}
          # seconds between checks
          wait-interval: 10

  test:
    # this job finishes after the Docker image is built (or exists already)
    runs-on: ubuntu-latest
    needs: [package-hash, wait-for-build]
    container: bahmutov/cy:${{ needs.package-hash.outputs.hash }}
    steps:
      - name: Checkout 🛎️
        # https://github.com/actions/checkout
        uses: actions/checkout@v6

      # THE IMPORTANT STEP: symlink the node modules
      # from the Docker image into the working folder
      # so we can skip the installation step
      - name: Symlink node modules
        run: ln -s /e2e/node_modules ./node_modules

      - name: Print Cypress version
        run: npx cypress --version

      - name: Run Cypress tests
        run: npx cypress run

The sym linking step is very important. After checkout finishes, the container has the following code

1
2
3
4
5
6
7
8

/e2e/
  cypress_cache/
  node_modules/
/<folder with checked out source code>/
  cypress/
  cypress.conf.js
  package.json
  package-lock.json

The current directory is set to /<folder with checked out source code>, so we need to make sure Node and Cypress can find the DEV dependencies. We do it by creating the symlink:

1
2
3
4
5
6
7
8
9

/e2e/
  cypress_cache/
  node_modules/
/<folder with checked out source code>/
  node_modules -> /e2e/node_modules/
  cypress/
  cypress.conf.js
  package.json
  package-lock.json

We tell Cypress NPM module to find its binary in the /e2e/cypress_cache/ folder by using the environment variable baked into the Dockerfile: ENV CYPRESS_CACHE_FOLDER=/e2e/cypress_cache.

Run on GitHub Actions

Let's confirm that it works. We can push the code for the very first time, or change package.json or Dockerfile

The build and push step took 1m14s, and the "ping" job that checked the job status every ten seconds finished in 1m22s. The test job simply pulled the Docker container and ran the specs, without any additional installation or resting a cache.

Pulling the container from the Docker image is by far the longest step in the job.

Now let's push another commit - maybe we changed the specs, but haven't touched the package.json or Dockerfile. The Docker image should have been found.

We start testing pretty quickly, since we don't have to install or build anything.

Using The Cypress Action

Many years ago I wrote cypress-io/github-action, and I am happy to report that it works well with the Docker image with baked DEV dependencies. Here is another workflow job that simply uses the action after sym linking:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

test-action:
  # this job finishes after the Docker image is built (or exists already)
  # and verifies the Cypress GitHub action works
  runs-on: ubuntu-latest
  needs: [package-hash, wait-for-build]
  container: bahmutov/cy:${{ needs.package-hash.outputs.hash }}
  steps:
    - name: Checkout 🛎️
      # https://github.com/actions/checkout
      uses: actions/checkout@v6

    # the important step: symlink the node modules
    # from the Docker image into the working folder
    # so we can skip the installation step
    - name: Symlink node modules
      run: ln -s /e2e/node_modules ./node_modules

    # confirm the Cypress action works
    - name: Cypress action
      # https://github.com/cypress-io/github-action
      uses: cypress-io/github-action@v6
      with:
        install: false

Finally, the GitHub Actions summary tells us if the Docker image exists or not based on the checksum.

GitHub Container Registry

If you are using GH Container registry, you can use the same Docker image approach, here is the example workflow from bahmutov/cypress-tests-image repo.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123

# this workflow uses Docker images via GitHub Container Registry (GHCR)
name: CI Using GHCR
on: push

env:
  REGISTRY: ghcr.io
  IMAGE_NAME: ${{ github.repository }}

jobs:
  # computes the hash of package.json and and stores it in the output
  # also checks if the Docker image with this tag already exists
  # outputs:
  #   hash: the package.json hash
  #   tag: whether the Docker image with this tag already exists, "found" or "not found"
  package-hash:
    runs-on: ubuntu-latest
    outputs:
      hash: ${{ steps.hash.outputs.checksum }}
      tag: ${{ steps.tag-exists.outputs.tag }}
    steps:
      - name: Checkout 🛎️
        # https://github.com/actions/checkout
        # only needed to get package.json and Dockerfile to compute the hash
        uses: actions/checkout@v6
        with:
          sparse-checkout: |
            package.json
            Dockerfile

      - name: Package.json + Dockerfile checksum
        id: hash
        run: echo "checksum=${{ hashFiles('package.json', 'Dockerfile') }}" >> $GITHUB_OUTPUT

      # https://github.com/tyriis/docker-image-tag-exists
      - name: Check if Docker image tag exists
        id: tag-exists
        uses: tyriis/docker-image-tag-exists@v2.1.0
        with:
          registry: ${{ env.REGISTRY }}
          repository: ${{ env.IMAGE_NAME }}
          # The container image tag
          tag: ${{ steps.hash.outputs.checksum }}

      - name: Report the check results
        # print the tag result into Github Actions summary
        run: |
          echo "## Docker image check" >> $GITHUB_STEP_SUMMARY
          echo "Registry: ${{ env.REGISTRY }}" >> $GITHUB_STEP_SUMMARY
          echo "Image: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}" >> $GITHUB_STEP_SUMMARY
          echo "Package.json + Dockerfile hash: ${{ steps.hash.outputs.checksum }}" >> $GITHUB_STEP_SUMMARY
          echo "Docker image ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ steps.hash.outputs.checksum }} **${{ steps.tag-exists.outputs.tag }}**" >> $GITHUB_STEP_SUMMARY

  build-docker-image:
    # builds the Docker image and pushes it to the registry
    # but only if it does not exist yet
    runs-on: ubuntu-latest
    needs: package-hash
    permissions:
      # don't forget to allow workflows to write to GHCR
      # https://github.com/<org>/<repo>/settings/actions
      packages: write
    if: ${{ needs.package-hash.outputs.tag == 'not found' }}
    steps:
      - name: Checkout 🛎️
        # https://github.com/actions/checkout
        uses: actions/checkout@v6

      - name: Log in to GitHub Container Registry
        # https://github.com/docker/login-action
        uses: docker/login-action@f4ef78c080cd8ba55a85445d5b36e214a81df20a
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      # we could also use the action
      # https://github.com/docker/build-push-action

      - name: Build docker image
        run: docker build -t ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ needs.package-hash.outputs.hash }} .

      - name: Push Docker image to the correct registry
        run: docker push ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ needs.package-hash.outputs.hash }}

  wait-for-build:
    # a trick to allow other jobs to run, even if the "build" job is skipped
    # runs in parallel with the "build" job and keeps checking if it is finished
    # or is skipped
    runs-on: ubuntu-latest
    needs: package-hash
    steps:
      - name: Wait for the Docker image build / skip
        # https://github.com/lewagon/wait-on-check-action
        uses: lewagon/wait-on-check-action@v1.4.1
        with:
          ref: ${{ github.ref }}
          check-name: build-docker-image
          repo-token: ${{ secrets.GITHUB_TOKEN }}
          # seconds between checks
          wait-interval: 10

  test:
    # this job finishes after the Docker image is built (or exists already)
    runs-on: ubuntu-latest
    needs: [package-hash, wait-for-build]
    # seems we cannot use the env variables here
    container: ghcr.io/${{ github.repository }}:${{ needs.package-hash.outputs.hash }}
    steps:
      - name: Checkout 🛎️
        # https://github.com/actions/checkout
        uses: actions/checkout@v6

      # THE IMPORTANT STEP: symlink the node modules
      # from the Docker image into the working folder
      # so we can skip the installation step
      - name: Symlink node modules
        run: ln -s /e2e/node_modules ./node_modules

      - name: Print Cypress version
        run: npx cypress --version

      - name: Run Cypress tests
        run: npx cypress run

I tried to make the workflow slightly more generic by moving the registry and the image into the env variables:

1
2
3

env:
  REGISTRY: ghcr.io
  IMAGE_NAME: ${{ github.repository }}

Nice.

Google Artifact Registry

Let's say you want to use private Docker images stored on Google Artifact Registry. You need to create it on GCP, then create a Service Account to access it from CI (this is simpler for me than using Workload identity), see the docs.

So here is my Artifact Registry called gleb-google-artifact-registry-test running inside us-east4-docker.pkg.dev configured to run under the project helloworld-330918

We will write and read Docker images using name cypress-tests-image. I will use a single Service Account for this with these permissions

For this account, grab the JSON key file and set it as GitHub Actions environment secret GCR_KEY. Now let's build and use images with dependencies. Here is the workflow file from bahmutov/cypress-tests-image.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140

# this workflow uses Docker images via Google Artifacts Registry (GCR)
# the registry is private, so we will use a Service Account to log in
name: CI Using Google Artifacts Registry
on: push

env:
  # gcr.io / Artifacts Registry
  REGISTRY: us-east4-docker.pkg.dev
  GCP_PROJECT: helloworld-330918
  # Artifacts Registry repository name
  REPOSITORY: gleb-google-artifact-registry-test
  IMAGE_NAME: cypress-tests-image

jobs:
  # computes the hash of package.json and and stores it in the output
  # also checks if the Docker image with this tag already exists
  # outputs:
  #   hash: the package.json hash
  #   tag: whether the Docker image with this tag already exists, "found" or "not found"
  package-hash:
    runs-on: ubuntu-latest
    outputs:
      hash: ${{ steps.hash.outputs.checksum }}
      tag: ${{ steps.tag-exists.outputs.tag }}
    steps:
      - name: Checkout 🛎️
        # https://github.com/actions/checkout
        # only needed to get package.json and Dockerfile to compute the hash
        uses: actions/checkout@v6
        with:
          sparse-checkout: |
            package.json
            Dockerfile

      - name: Package.json + Dockerfile checksum
        id: hash
        run: echo "checksum=${{ hashFiles('package.json', 'Dockerfile') }}" >> $GITHUB_OUTPUT

      - name: Log in to Google Artifacts Registry
        # https://github.com/docker/login-action
        uses: docker/login-action@f4ef78c080cd8ba55a85445d5b36e214a81df20a
        with:
          registry: ${{ env.REGISTRY }}
          username: _json_key
          password: ${{ secrets.GCR_KEY }}

      # https://github.com/tyriis/docker-image-tag-exists
      - name: Check if Docker image tag exists
        id: tag-exists
        uses: tyriis/docker-image-tag-exists@v2.1.0
        with:
          registry: ${{ env.REGISTRY }}
          repository: ${{ env.GCP_PROJECT }}/${{ env.REPOSITORY }}/${{ env.IMAGE_NAME }}
          # The container image tag
          tag: ${{ steps.hash.outputs.checksum }}

      - name: Report the check results
        # print the tag result into Github Actions summary
        run: |
          echo "## Docker image check" >> $GITHUB_STEP_SUMMARY
          echo "Registry: ${{ env.REGISTRY }}" >> $GITHUB_STEP_SUMMARY
          echo "Image: ${{ env.REGISTRY }}/${{ env.GCP_PROJECT }}/${{ env.REPOSITORY }}/${{ env.IMAGE_NAME }}" >> $GITHUB_STEP_SUMMARY
          echo "Package.json + Dockerfile hash: ${{ steps.hash.outputs.checksum }}" >> $GITHUB_STEP_SUMMARY
          echo "Docker image ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ steps.hash.outputs.checksum }} **${{ steps.tag-exists.outputs.tag }}**" >> $GITHUB_STEP_SUMMARY

  build-docker-image:
    # builds the Docker image and pushes it to the registry
    # but only if it does not exist yet
    runs-on: ubuntu-latest
    needs: package-hash
    permissions:
      # don't forget to allow workflows to write to GHCR
      # https://github.com/<org>/<repo>/settings/actions
      packages: write
    if: ${{ needs.package-hash.outputs.tag == 'not found' }}
    steps:
      - name: Checkout 🛎️
        # https://github.com/actions/checkout
        uses: actions/checkout@v6

      - name: Log in to Google Artifacts Registry
        # https://github.com/docker/login-action
        uses: docker/login-action@f4ef78c080cd8ba55a85445d5b36e214a81df20a
        with:
          registry: ${{ env.REGISTRY }}
          username: _json_key
          password: ${{ secrets.GCR_KEY }}

      # we could also use the action
      # https://github.com/docker/build-push-action

      - name: Build docker image
        run: docker build -t ${{ env.REGISTRY }}/${{ env.GCP_PROJECT }}/${{ env.REPOSITORY }}/${{ env.IMAGE_NAME }}:${{ needs.package-hash.outputs.hash }} .

      - name: Push Docker image to the correct registry
        run: docker push ${{ env.REGISTRY }}/${{ env.GCP_PROJECT }}/${{ env.REPOSITORY }}/${{ env.IMAGE_NAME }}:${{ needs.package-hash.outputs.hash }}

  wait-for-build:
    # a trick to allow other jobs to run, even if the "build" job is skipped
    # runs in parallel with the "build" job and keeps checking if it is finished
    # or is skipped
    runs-on: ubuntu-latest
    needs: package-hash
    steps:
      - name: Wait for the Docker image build / skip
        # https://github.com/lewagon/wait-on-check-action
        uses: lewagon/wait-on-check-action@v1.4.1
        with:
          ref: ${{ github.ref }}
          check-name: build-docker-image
          repo-token: ${{ secrets.GITHUB_TOKEN }}
          # seconds between checks
          wait-interval: 10

  test:
    # this job finishes after the Docker image is built (or exists already)
    runs-on: ubuntu-latest
    needs: [package-hash, wait-for-build]
    # seems we cannot use the env variables here
    container:
      image: us-east4-docker.pkg.dev/helloworld-330918/gleb-google-artifact-registry-test/cypress-tests-image:${{ needs.package-hash.outputs.hash }}
      credentials:
        username: _json_key
        password: ${{ secrets.GCR_KEY }}
    steps:
      - name: Checkout 🛎️
        # https://github.com/actions/checkout
        uses: actions/checkout@v6

      # THE IMPORTANT STEP: symlink the node modules
      # from the Docker image into the working folder
      # so we can skip the installation step
      - name: Symlink node modules
        run: ln -s /e2e/node_modules ./node_modules

      - name: Print Cypress version
        run: npx cypress --version

      - name: Run Cypress tests
        run: npx cypress run

Weird GitHub Actions limitation: one cannot use env keys inside the image: ... string, thus we have to duplicate the full Docker string

1
2
3
4
5
6
7
8
9
10
11
12
13
14

env:
  # gcr.io / Artifacts Registry
  REGISTRY: us-east4-docker.pkg.dev
  GCP_PROJECT: helloworld-330918
  # Artifacts Registry repository name
  REPOSITORY: gleb-google-artifact-registry-test
  IMAGE_NAME: cypress-tests-image

# later
container:
  image: us-east4-docker.pkg.dev/helloworld-330918/gleb-google-artifact-registry-test/cypress-tests-image:${{ needs.package-hash.outputs.hash }}
  credentials:
    username: _json_key
    password: ${{ secrets.GCR_KEY }}

The finished workflow works great

Just keep the GCR_KEY secret!

So I got myself a "plain" music player so I can read / think / walk in peace and leave my phone at home. ECHO Mini plays everything, is tiny, has 2 audio jacks, has bluetooth, and even shows a tiny cassette tape on its front display!

So now I leave my phone at home and bring a music player (which can fit lots of MP3/OGG/M4A/FLAC files), some headphones, and a regular watch to know the time. Now I can think, read, and work in peace.

For example, let's say I need to test if item prices on the inventory page are sorted.

We can use cypress-map queries to create a retry-able chain of commands.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

import { LoginPage } from '@support/pages/login.page'

describe('Inventory page', () => {
  // visit the login page before each test
  beforeEach(() => {
    const user = Cypress.env('users').standard
    LoginPage.login(user.username, user.password)
  })

  it('sorts items by price, low to high', () => {
    cy.visit('/inventory')
    cy.get('[data-test="product_sort_container"]').select('Price (low to high)')
    cy.get('.inventory_item_price')
      .map('innerText')
      // slice the leading $
      .mapInvoke('slice', 1)
      .map(parseFloat)
      .print('prices %o')
      // confirm the numbers are sorted in ascending order
  })
})

🎓 This example comes from my online courses Testing The Swag Store and Write Cypress Tests Using GitHub Copilot.

How do we write the last assertion following the // confirm the numbers are sorted in ascending order comment? We can ask Copilot or wait for it to suggest the code completion. Here is one possible suggestion:

1
2
3
4

.then((prices) => {
  const sortedPrices = [...prices].sort((a, b) => a - b)
  expect(prices).to.deep.equal(sortedPrices)
})

The test runs and shows an anonymous deeply equal assertion inside the .then(callback) function:

Nothing wrong with this suggestion (except it is not retry-able), but I prefer using chai-sorted plugin instead. The assertions are easier to read and much shorter. Let's install the plugin:

1

$ npm i -D chai-sorted

Then we update the cypress/support/e2e.js file to include the plugin:

1
2

// https://www.chaijs.com/plugins/chai-sorted/
chai.use(require('chai-sorted'))

Now we need to tell Copilot to use the newly installed plugin. We can update the Copilot instruction file

1
2
3
4
5
6
7
8

This is a project that uses Cypress for end-to-end testing, and follows best practices.

To check if the numbers are sorted, always use the chai-sorted plugin for Cypress, for example:

    // sort the numbers in ascending order
    cy.get('.price').map('innerText').map(parseFloat).should('be.ascending')
    // sort the numbers in descending order
    cy.get('.price').map('innerText').map(parseFloat).should('be.descending')

Now let's return the test and see what Copilot suggests for the last assertion:

Much shorter and easier to read, and it even includes helpful text, unlike .then(callback) we had before! This is exactly what we wanted. Here is the passing test

Perfect.

The skeleton itself is simple, just a DIV with some gradient CSS.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

if (isLoading) {
  return (
    <div>
      <div className="login_logo" />
      <div className="login_wrapper">
        <div className="login_wrapper-inner">
          <div id="login_button_container" className="form_column">
            <div className="login-box">
              <div className="skeleton skeleton-heading"></div>
              <div className="skeleton skeleton-text"></div>
              <div className="skeleton skeleton-text"></div>
              <div className="skeleton skeleton-heading"></div>
              <div className="skeleton skeleton-text"></div>
            </div>
          </div>
          <div className="bot_column" />
        </div>
        <div className="login_credentials_wrap">
          <div className="login_credentials_wrap-inner">
            <div className="skeleton skeleton-heading"></div>
            <div className="skeleton skeleton-text"></div>
            <div className="skeleton skeleton-text"></div>
          </div>
        </div>
      </div>
    </div>
  )
}
// else return the real component

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

.skeleton {
  background: linear-gradient(90deg, #f0f0f0 25%, #e0e0e0 50%, #f0f0f0 75%);
  background-size: 200% 100%;
  animation: loading 1.5s ease-in-out infinite;
  border-radius: 4px;
  margin-bottom: 15px;
}
.skeleton-heading {
  height: 24px;
  width: 60%;
}
.skeleton-text {
  height: 16px;
  width: 100%;
}
@keyframes loading {
  0% {
    background-position: 200% 0;
  }
  100% {
    background-position: -200% 0;
  }
}

Test the loading skeleton

Let's confirm the loading skeleton is shown initially and then replaced by the real component. If the loading skeleton is always displayed, we can write a test similar to this one:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

import { LoginPage } from '@support/pages/login.page'

describe('Login form skeleton', () => {
  // visit the login page before each test
  beforeEach(() => {
    cy.visit('/')
  })

  it('shows the loading skeleton first', () => {
    cy.get('#login_button_container').should('be.visible')
    cy.get('.skeleton').should('be.visible').and('have.length.greaterThan', 2)
    // skeleton should go away
    // and the login form is immediately visible (within 100ms)
    cy.get('.skeleton').should('not.exist')
    cy.get(LoginPage.selectors.username, { timeout: 100 }).should('be.visible')
  })
})

Tip: if the skeleton does NOT always appear (for example, if the data is cached and the skeleton is skipped), we can clear the data and/or slow down the network request to make the skeleton always appear.

Test the skeleton positioning

One of the disturbing aspects of the loading skeleton is mismatch between its positions and the loaded text. For example, take a look at this loop - can you see the "jump" between the skeleton heading DIV and the "Accepted usernames are:" H4?

We want to catch this skeleton position mismatch. Let's compare the "top" property of the skeleton DIV and the rendered H4 elements - they should be close to each other in order to avoid disorienting the user. We could write a test like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

describe('Login form skeleton', () => {
  const skeletonHeading = '.login_credentials_wrap .skeleton-heading'
  const loginCredentials = '.login_credentials_wrap .login_credentials h4'
  const tolerance = 20 // pixels

  // visit the login page before each test
  beforeEach(() => {
    cy.visit('/')
  })

  it('does not move from the top', () => {
    // confirm the skeleton heading does NOT change
    // it "top" position on the page too much (within tolerance)
    cy.get(skeletonHeading)
      .should('be.visible')
      .then(($el) => {
        const rect = $el[0].getBoundingClientRect()
        return rect.top
      })
      // round to pixels for nicer comparison
      .then(Math.round)
      .as('initialTop', { type: 'static' })
    cy.get(skeletonHeading).should('not.exist')

    cy.get('@initialTop').then((initialTop) => {
      cy.get(loginCredentials)
        .should('be.visible')
        .then(($el) => {
          const rect = $el[0].getBoundingClientRect()
          return rect.top
        })
        .then(Math.round)
        .should('be.closeTo', initialTop, tolerance)
    })
  })
})

The test catches the heading skeleton "moving" 100 pixels when the real heading H4 element is shown.

Because I love writing concise and elegant code, I will use my cypress-map plugin to rewrite the above test a little bit to make it shorter and easier to read.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21