products

Run Two Cypress Test Runners At The Same Time

How to test a real-time Socket.io chat app by running two Cypress test runners concurrently.

Please start by reading the blog post Test a Socket.io Chat App using Cypress. In this blog post we will run two Cypress test runners concurrently without any synchronization between them to test a real-time chat application.

Two Cypress test runners chatting with each other

Separate specs

First, we want to give each Cypress test runner its own configuration and the spec to run. While the first Cypress test runner executes the spec file cypress/pair/first-user.js, the second Cypress test runner should execute the spec file cypress/pair/second-user.js.

cypress/pair/first-user.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
/// <reference types="cypress" />

// this test behaves as the first user to join the chat
it('chats with the second user', () => {
  const name = 'First'
  const secondName = 'Second'

  cy.visit('/', {
    onBeforeLoad(win) {
      cy.stub(win, 'prompt').returns(name)
    },
  })

  // make sure the greeting message is shown
  cy.contains('#messages li i', `${name} join the chat..`)
    .should('be.visible')

  // at some point, the second user enters the chat and posts a message
  cy.contains('#messages li', 'Good to see you')
    .contains('strong', secondName)

  // reply to the second user
  cy.get('#txt').type('Glad to be here{enter}')
  cy.contains('#messages li', 'Glad to be here')
    .contains('strong', name)
})

The second user's spec sends the opposite message:

cypress/pair/second-user.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
/// <reference types="cypress" />

// this test behaves as the second user to join the chat
it('chats with the first user', () => {
  const name = 'Second'
  // we are chatting with the first user
  const firstName = 'First'
  cy.visit('/', {
    onBeforeLoad(win) {
      cy.stub(win, 'prompt').returns(name)
    },
  })

  // make sure the greeting message is shown
  cy.contains('#messages li i', `${name} join the chat..`)
    .should('be.visible')

  cy.get('#txt').type('Good to see you{enter}')

  // a message from the first user arrives
  cy.contains('#messages li', 'Glad to be here')
    .contains('strong', firstName)
})

The configuration files

To precisely control the test runners, each instance will have its own configuration file. The first test runner will use cy-first-user.json file.

cy-first-user.json
1
2
3
4
5
6
7
8
9
10
11
12
13
14
{
  "fixturesFolder": false,
  "supportFile": false,
  "pluginsFile": false,
  "baseUrl": "http://localhost:8080",
  "integrationFolder": "cypress/pair",
  "testFiles": "**/first-user.js",
  "viewportWidth": 400,
  "viewportHeight": 400,
  "defaultCommandTimeout": 15000,
  "videosFolder": "cypress/videos-pair/first",
  "screenshotsFolder": "cypress/screenshots-pair/first",
  "$schema": "https://on.cypress.io/cypress.schema.json"
}

The property $schema allows the modern code editors to apply custom JSON schema to the configuration file and show intelligent code popups for the fields.

The second configuration file is almost identical.

cy-second-user.json
1
2
3
4
5
6
7
8
9
10
11
12
13
14
{
  "fixturesFolder": false,
  "supportFile": false,
  "pluginsFile": false,
  "baseUrl": "http://localhost:8080",
  "integrationFolder": "cypress/pair",
  "testFiles": "**/second-user.js",
  "viewportWidth": 400,
  "viewportHeight": 400,
  "defaultCommandTimeout": 15000,
  "videosFolder": "cypress/videos-pair/second",
  "screenshotsFolder": "cypress/screenshots-pair/second",
  "$schema": "https://on.cypress.io/cypress.schema.json"
}
  1. While running two Cypress instances, we are not using any plugins or custom commands, thus we disable the fixtures, the support files, and the plugins file.
  2. We use a custom integration folder to separate the two concurrent specs from any other any other individual specs. We also set the explicit single test file for each runner.
  3. We increase the default command timeout because it might take a while to start the second test runner.
  4. We use separate folders and videos to avoid each test runner clobbering the common folder on start.

Tip: we could have used cypress-extends plugin to reuse the common configuration data, but I prefer to be explicit in these examples for clarity.

Now we can manually start the application, open the first Cypress instance with npx cypress open --config-file cy-first-user.json, then open the second Cypress instance with npx cypress open --config-file cy-second-user.json command. The tests pass:

The first and the second tests

Running concurrently

To execute the two test runners together, we can install NPM module concurrently

1
2
$ npm i -D concurrently
+ [email protected]

Let's define common scripts in the package.json file

1
2
3
4
5
6
7
8
{
  "scripts": {
    "start": "node .",
    "cy:first": "cypress run --config-file cy-first-user.json",
    "cy:second": "cypress run --config-file cy-second-user.json",
    "chat": "concurrently npm:cy:first npm:cy:second"
  }
}

Whenever we need to run the tests, we start the application and call npm run chat. The concurrently module shows the output from each instance prefixed by the command's name:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
$ npm start &
$ npm run chat

> [email protected] chat /Users/gleb/git/cypress-socketio-chat
> concurrently npm:cy:first npm:cy:second

[cy:first]
[cy:first] > [email protected] cy:first /Users/gleb/git/cypress-socketio-chat
[cy:first] > cypress run --config-file cy-first-user.json
[cy:first]
[cy:second]
[cy:second] > [email protected] cy:second /Users/gleb/git/cypress-socketio-chat
[cy:second] > cypress run --config-file cy-second-user.json
[cy:second]
[cy:second]
...

[cy:first]
[cy:first]
[cy:second]        Spec                                              Tests  Passing  Failing  Pending  Skipped
[cy:second]   ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
[cy:second]   │ ✔  second-user.js                           00:01        1        1        -        -        - │
[cy:second]   └────────────────────────────────────────────────────────────────────────────────────────────────┘
[cy:second]     ✔  All specs passed!                        00:01        1        1        -        -        -
[cy:second]
[cy:first]        Spec                                              Tests  Passing  Failing  Pending  Skipped
[cy:first]   ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
[cy:first]   │ ✔  first-user.js                            00:01        1        1        -        -        - │
[cy:first]   └────────────────────────────────────────────────────────────────────────────────────────────────┘
[cy:first]     ✔  All specs passed!                        00:01        1        1        -        -        -
[cy:first]
[cy:first] npm run cy:first exited with code 0
[cy:second] npm run cy:second exited with code 0

We can watch the video saved by each test runner. We can even run the E2E tests on CI. I am using GitHub Actions, first running all E2E tests, then running the two Cypress instances:

.github/workflows/ci.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
- name: Check out code 🛎
  uses: actions/checkout@v2

# install dependencies, start the app,
# and run E2E tests using Cypress GitHub action
# https://github.com/cypress-io/github-action
- name: Run tests 🧪
  uses: cypress-io/github-action@v2
  with:
    start: npm start
    wait-on: 'http://localhost:8080'

# run two Cypress instances at the same time
# so they truly chat with each other
- name: Run 2 Cypresses 🧪
  uses: cypress-io/github-action@v2
  with:
    # we already have installed everything
    install: false
    # the application is running already
    # from the previous "start" command
    command: npm run chat

Tip: sometimes starting two Cypress instances creates a race condition while starting the X11 server. A more robust approach would start a separate XServer, then start two Cypress runners.

See also