products

Test a Socket.io Chat App using Cypress

How to write simple test for a real-time chat web application.

This is the first blog post showing how to test a Socket.io real-time chat using the Cypress.io test runner. Subscribe to the Atom RSS feed and follow @bahmutov to learn when the new parts are published.

The app

As the example app, I used the chat app from dkhd/node-group-chat repo, as described in Build A Group-Chat App in 30 Lines Using Node.js blog post. The app allows you to open multiple chat browser windows and exchange group messages.

1
2
3
4
5
6
7
8
9
10
11
12
npm start

> [email protected] start /Users/gleb/git/cypress-socketio-chat
> node .

listening on *:8080
new connection
set username Joe
new connection
set username Ann
> Joe: Hi there, Ann
> Ann: Good to see you, Joe

Two users chatting

The server is relaying the messages from the users, the front end code is using event subscriptions to display new messages.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
<head>
  <script src="../../socket.io/socket.io.js"></script>
</head>
<body>
  <ul id="messages"></ul>
  <form action="/" method="POST" id="chatForm">
    <input
      id="txt"
      autocomplete="off"
      autofocus="on"
      placeholder="type your message here..."
    /><button>Send</button>
  </form>
  <script>
    var socket = io.connect('http://localhost:8080')
    // submit text message without reload/refresh the page
    $('form').submit(function (e) {
      e.preventDefault() // prevents page reloading
      socket.emit('chat_message', $('#txt').val())
      $('#txt').val('')
      return false
    })

    socket.on('chat_message', (msg) =>
      $('#messages').append($('<li>').html(msg)),
    )

    socket.on('is_online', (username) =>
      $('#messages').append($('<li>').html(username)),
    )

    // ask username
    var username = prompt('Please tell me your name')
    socket.emit('username', username)
  </script>
</body>

Note: you can find the source code for the app, and the tests in the repo bahmutov/cypress-socketio-chat.

The first test

Even if there is a single user, the chat application is showing every message. Thus our first test can confirm the submitted messages appear.

cypress/integration/first-spec.js
1
2
3
4
5
6
7
8
9
10
11
12
13
/// <reference types="cypress" />

it('shows the message', () => {
  cy.visit('/', {
    onBeforeLoad(win) {
      // when the application asks for the name
      // return "Cy" using https://on.cypress.io/stub
      cy.stub(win, 'prompt').returns('Cy')
    },
  })
  cy.get('#txt').type('Hello there{enter}')
  cy.contains('#messages li', 'Hello there').contains('strong', 'Cy')
})

Tip: to start the server and open Cypress when the localhost:8080 responds I use start-server-and-test utility.

First test passing

As soon as I have the first Cypress test passing, I configure the continuous integration service to test every commit. In this repo, I will be using Cypress GitHub Action to run the tests, see .github/workflows/ci.yml file:

.github/workflows/ci.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
name: ci
on: push
jobs:
  test:
    runs-on: ubuntu-20.04
    steps:
      - name: Check out code 🛎
        uses: actions/checkout@v2

      # install dependencies, start the app,
      # and run E2E tests using Cypress GitHub action
      # https://github.com/cypress-io/github-action
      - name: Run tests 🧪
        uses: cypress-io/github-action@v2
        with:
          start: npm start
          wait: 'http://localhost:8080'

You can find the CI runs at bahmutov/cypress-socketio-chat/actions.

Test runs on CI using GitHub Actions

Use a random user name

We are working towards testing the full chat, thus we will have multiple users. To avoid every test typing the same Cy name, let's create the user names randomly using the bundled Lodash library.

cypress/integration/random-name-spec.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
/// <reference types="cypress" />

it('chats', () => {
  const name = `Cy_${Cypress._.random(1000)}`
  // we can make text bold using Markdown "**"
  cy.log(`User **${name}**`)
  cy.visit('/', {
    onBeforeLoad(win) {
      cy.stub(win, 'prompt').returns(name)
    },
  })
  cy.get('#txt').type('Hello there{enter}')
  cy.contains('#messages li', 'Hello there').contains('strong', name)
})

Using a randomly generates user name

🌍 Notice the name Cy_350 in the command log. By coincidence 350.org is a climate organization fighting to bring us back to 350 parts per million (ppm) of carbon dioxide gas (CO2) in the atmosphere - which is the safe limit. Currently we are at 420 ppm - a level of this greenhouse planet-warming gas we had 4 million years ago, when the average temperatures were 3C / 7F higher, and the sea level were higher by 26m / 70 feet. Plan your future with this in mind.

Sending events from the test

We need to see what happens when some other user joins the chat and sends a message. Because this communication happens over WebSockets, we cannot simply use cy.intercept, at least not in v7. But let's see what happens immediately after a new message arrives via a socket channel:

1
2
3
4
5
6
7
socket.on('chat_message', (msg) =>
  $('#messages').append($('<li>').html(msg)),
)

socket.on('is_online', (username) =>
  $('#messages').append($('<li>').html(username)),
)

We want to trigger those events from the test. The simplest way is to create an object abstraction between the socket and the rest of the code.

1
2
3
4
5
6
7
8
9
10
11
12
13
var socket = io.connect('http://localhost:8080')

const clientActions = {
  onChatMessage(msg) {
    $('#messages').append($('<li>').html(msg))
  },
  isOnline(username) {
    $('#messages').append($('<li>').html(username))
  },
}

socket.on('chat_message', clientActions.onChatMessage)
socket.on('is_online', clientActions.isOnline)

The application works, the same, but now we can expose the clientActions object during end-to-end tests.

1
2
3
if (window.Cypress) {
  window.clientActions = clientActions
}

Here is how we can use it from the test to test what happens when another user joins the chat and posts a message. We are not delivering the messages via the socket connection, instead we trigger them right after.

cypress/integration/client-api-spec.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
it('shows status for 2nd user', () => {
  const name = `Cy_${Cypress._.random(1000)}`
  cy.log(`User **${name}**`)
  cy.visit('/', {
    onBeforeLoad(win) {
      cy.stub(win, 'prompt').returns(name)
    },
  })

  cy.get('#txt').type('Hello there{enter}')
  cy.contains('#messages li', 'Hello there')
    .contains('strong', name)
    .then(() => {
      cy.log('**second user**')
    })

  // pretend to send a message from another user
  cy.window()
    .its('clientActions')
    .invoke('isOnline', '👻 <i>Ghost is testing</i>')
  cy.window()
    .its('clientActions')
    .invoke('onChatMessage', '<strong>Ghost</strong>: Boo')
  cy.contains('#messages li', 'Boo').contains('strong', 'Ghost')
})

Testing messages from another user

We can refactor the spec code and save the clientActions reference as an alias

1
2
3
4
5
// pretend to send a message from another user
cy.window().its('clientActions').as('client')
cy.get('@client').invoke('isOnline', '👻 <i>Ghost is testing</i>')
cy.get('@client').invoke('onChatMessage', '<strong>Ghost</strong>: Boo')
cy.contains('#messages li', 'Boo').contains('strong', 'Ghost')

Tip: the above approach is what I call app actions.

Use Socket.io from Cypress

The above approach has an advantage - it does not require the backend to run. We can simply use clientActions without a server to exercise our web page as needed. But we really won't have any confidence in the server running correctly. We won't have confidence in our WebSocket communication channel working either. Let's truly "drive" the conversation by opening a second connection to the server and "chatting" with our web user.

First, let's install the socket.io-client NPM module

1
2
$ npm i -D socket.io-client
[email protected]

Now in Cypress plugin file cypress/plugins/index.js we will add an optional socket.io client logic to be controlled using the cy.task command.

cypress/plugins/index.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
/// <reference types="cypress" />

// Socket.io client to allow Cypress itself
// to connect from the plugin file to the chat app
// to play the role of another user
// https://socket.io/docs/v4/client-initialization/
const io = require('socket.io-client')

/**
 * @type {Cypress.PluginConfig}
 */
// eslint-disable-next-line no-unused-vars
module.exports = (on, config) => {
  // `on` is used to hook into various events Cypress emits
  // `config` is the resolved Cypress config

  // connection to the chat server
  let socket

  on('task', {
    connect(name) {
      console.log('Cypress is connecting to socket server under name %s', name)
      socket = io('http://localhost:8080')

      socket.emit('username', name)

      return null
    },
  })
}

The test can call the task and confirm the web UI receives the new user message

cypress/integration/socket-spec.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
/// <reference types="cypress" />

it('sees the 2nd user join', () => {
  // the browser is the 1st user
  const name = `Cy_${Cypress._.random(1000)}`
  cy.log(`User **${name}**`)
  cy.visit('/', {
    onBeforeLoad(win) {
      cy.stub(win, 'prompt').returns(name)
    },
  })

  // make sure the greeting message is shown
  cy.contains('#messages li i', `${name} join the chat..`).should('be.visible')

  // connect to the server using 2nd user
  const secondName = 'Ghost'
  cy.task('connect', secondName)
  cy.contains('#messages li i', `${secondName} join the chat..`).should(
    'be.visible',
  )
})

The test passes, our web page really sees the users joining the running Socket.io server.

Seeing the second user join

Let's send a message from the second user and confirm the web page receives it. We can add another task to the plugin file:

1
2
3
4
5
// task in cypress/plugins/index.js file
say(message) {
  socket.emit('chat_message', message)
  return null
},

The test can send the message using the 2nd user (via cy.task) and verify it shows up in the web page

cypress/integration/socket-spec.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
it('sees messages from the 2nd user', () => {
  // the browser is the 1st user
  const name = `Cy_${Cypress._.random(1000)}`
  cy.log(`User **${name}**`)
  cy.visit('/', {
    onBeforeLoad(win) {
      cy.stub(win, 'prompt').returns(name)
    },
  })

  const secondName = 'Ghost'
  cy.task('connect', secondName)
  const message = 'hello from 2nd user'
  cy.task('say', message)
  cy.contains('#messages li', message)
})

The test passes - but it shows the typical distributed system behavior; it is hard to guarantee that messages are delivered in the expected order.

Unexpected order of messages

We really want to ensure the first user joins before sending messages from the 2nd user, let's add assertions.

cypress/plugins/index.js file
1
2
3
4
5
6
7
8
9
10
11
// make sure the greeting message is shown
cy.contains('#messages li i', `${name} join the chat..`).should('be.visible')

const secondName = 'Ghost'
cy.task('connect', secondName)
cy.contains('#messages li i', `${secondName} join the chat..`)
  .should('be.visible')

const message = 'hello from 2nd user'
cy.task('say', message)
cy.contains('#messages li', message)

Now the test behaves as expected

Additional commands ensure the expected order of events

We have verified that web page shows the messages sent to the Socket.io server from other users.

Verify message received by the second user

Let's verify the messages sent by the web page reach the other users connected to the Socket.io server. We will store the last received message in the plugin file and will add fetching it via cy.task. This is similar to checking the last received email I have described in the blog post Full Testing of HTML Emails using SendGrid and Ethereal Accounts.

cypress/plugins/index.js file
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
// connection to the chat server
let socket
let lastMessage

on('task', {
  connect(name) {
    console.log('Cypress is connecting to socket server under name %s', name)
    socket = io('http://localhost:8080')

    socket.emit('username', name)
    socket.on('chat_message', (msg) => (lastMessage = msg))

    return null
  },

  say(message) {
    socket.emit('chat_message', message)
    return null
  },

  getLastMessage() {
    // cy.task cannot return undefined value
    return lastMessage || null
  },
})

The test sends the message using the web page UI and then fetches it for the second user.

cypress/integration/socket-spec.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
it('verifies messages received', () => {
  // the browser is the 1st user
  const name = `Cy_${Cypress._.random(1000)}`
  cy.log(`User **${name}**`)
  cy.visit('/', {
    onBeforeLoad(win) {
      cy.stub(win, 'prompt').returns(name)
    },
  })

  // make sure the greeting message is shown
  cy.contains('#messages li i', `${name} join the chat..`).should('be.visible')

  const secondName = 'Ghost'
  cy.task('connect', secondName)
  cy.contains('#messages li i', `${secondName} join the chat..`)
    .should('be.visible')

  // send the message from the first user
  // and use a random code to guarantee we truly receive our message
  const message = `Hello there ${Cypress._.random(10000)}`
  cy.get('#txt').type(message)
  cy.get('form').submit()

  // verify the web page shows the message
  // this ensures we can ask the 2nd user for its last message
  // and it should already be there
  cy.contains('#messages li', message).contains('strong', name)

  cy.task('getLastMessage')
    // note that the message includes the sending user
    // and the message itself
    .should('include', name)
    .and('include', message)
})

Verifying the message is received

Tip: you can poll the plugin process for the last message using cypress-recurse plugin.

Test user leaving the chat

Finally, let's confirm the web page shows the users leaving the chat. In the plugin file, let's add another task:

1
2
3
4
5
// task in cypress/plugins/index.js file
disconnect() {
  socket.disconnect()
  return null
}

The test can verify the messages shown by the web page.

1
2
3
4
5
6
7
8
const secondName = 'Ghost'
cy.task('connect', secondName)
cy.contains('#messages li i', `${secondName} join the chat..`)
  .should('be.visible')
// the 2nd user is leaving
cy.task('disconnect')
cy.contains('#messages li i', `${secondName} left the chat..`)
  .should('be.visible')

Beautiful.

The user has left the chat test

🧭 You can find the source code for the app, and the tests in the repo bahmutov/cypress-socketio-chat.

Alternative: connect to the Socket.io server from the spec file

You can connect to the Socket.io server from the plugin file. You could also connect to the server straight from the spec file. Here is a complete test that simulates the second user via separate chat connection. The first user works through the page, while the second user works through the chat messages.

cypress/integration/socket-from-browser-spec.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
/// <reference types="cypress" />

const io = require('socket.io-client')

describe('Open 2nd socket connection', () => {
  it('sees the 2nd user join', () => {
    // the browser is the 1st user
    const name = `Cy_${Cypress._.random(1000)}`

    cy.log(`User **${name}**`)
    cy.visit('/', {
      onBeforeLoad(win) {
        cy.stub(win, 'prompt').returns(name)
      },
    })

    // make sure the greeting message is shown
    cy.contains('#messages li i', `${name} join the chat..`)
      .should('be.visible')
      .then(() => {
        // and now connect to the server using 2nd user
        // by opening a new Socket connection from the same browser window
        const secondName = 'Ghost'

        const socket = io.connect('http://localhost:8080')
        socket.emit('username', secondName)

        // keep track of the last message sent by the server
        let lastMessage
        socket.on('chat_message', (msg) => (lastMessage = msg))

        // the page shows that the second user has joined the chat
        cy.contains('#messages li i', `${secondName} join the chat..`).should(
          'be.visible',
        )

        // the second user can send a message and the page shows it
        const message = 'hello from 2nd user'
        socket.emit('chat_message', message)
        cy.contains('#messages li', message)

        // when the first user sends the message from the page
        // the second user receives it via socket
        const greeting = `Hello there ${Cypress._.random(10000)}`
        cy.get('#txt').type(greeting)
        cy.get('form').submit()

        // verify the web page shows the message
        // this ensures we can ask the 2nd user for its last message
        // and it should already be there
        cy.contains('#messages li', greeting).contains('strong', name)

        // place the assertions in a should callback
        // to retry them, maybe there is a delay in delivery
        cy.should(() => {
          // using "include" assertion since the server adds HTML markup
          expect(lastMessage, 'last message for 2nd user').to.include(greeting)
          expect(lastMessage, 'has the sender').to.include(name)
        })

        cy.log('**second user leaves**').then(() => {
          socket.disconnect()
        })
        cy.contains('#messages li i', `${secondName} left the chat..`).should(
          'be.visible',
        )
      })
  })
})

The test behaves as expected.

The 2nd user is a separate socket connection from the same browser

See also