Testing Sentry Call with Cypress

How to verify Sentry call happens on an unhandled exception

I love using Sentry for tracking errors in my web applications. Recently I have added crash reporting to our open source 350-mass-cambridge-somerville/350-actions-client project in pull request #24 - see the app at https://www.maclimateactions.com/.

The entire web application is tested using Cypress - let's validate that the app is sending a crash report to Sentry service using an end-to-end test too. Let's start a placeholder test that opens the app and does nothing.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
describe('current action', () => {
beforeEach(() => {
cy.server()
cy.route('/actioncards/latest/', 'fixture:latest')
cy.visit('/')
// check the page - should have info from the stubbed response
// loaded from cypress/fixtures/latest.json
cy.contains('Action Card 23').should('be.visible')
cy.get('[data-cy=action-check-display]').should('have.length', 6)
})

it.only('sends an error to Sentry', () => {
// TODO write test commands
})
})

Once the test runs and stops, open the DevTools inside Cypress. Switch to the application's context using a drop down. A nice trick to throw an unhandled exception in the application, and not in the DevTools it to make it asynchronous - for example by using setTimeout function. You can do setTimeout(() => { throw new Error('test error') }, 1000) and see Sentry handler executing an XHR call.

Triggering an error manually

Note: by default, Sentry uses fetch protocol to send the error object to its service API. Since Cypress does not support stubbing fetch requests yet, we force every client-side library to drop down to XHR using the example from "Stubbing window.fetch" recipe. We simply delete the window.fetch property when every window is created.

cypress/support/index.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
let polyfill

// grab fetch polyfill from remote URL, could be also from a local package
before(() => {
const polyfillUrl = 'https://unpkg.com/unfetch/dist/unfetch.umd.js'

cy.request(polyfillUrl).then(response => {
polyfill = response.body
})
})

Cypress.on('window:before:load', win => {
delete win.fetch
// since the application code does not ship with a polyfill
// load a polyfilled "fetch" from the test
win.eval(polyfill)
win.fetch = win.unfetch
})

Let's confirm this call happens. We need to prepare to intercept the call from the test. Look at the call's details, especially at the method and the destination URL.

Sentry call details

Also look at the response object the Sentry service sends back to the caller.

Sentry response object

Let's stub this route using cy.route command and response with similar object.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
it('sends an error to Sentry', () => {
cy.route('POST', 'https://sentry.io/api/*/store/*', {
id: 'abc123',
}).as('sentry')
// create an error, as if application has thrown it
cy.window().invoke(
'setTimeout',
() => {
throw new Error('test error')
},
1000,
)
// confirm the call has happened
cy.wait('@sentry')
})

The test runs - and we can see the stubbed XHR call to Sentry correctly detected (and stopped)

The call to Sentry API does happen on error

Great, but - a thrown error is still a thrown error, and Cypress treats is as a test failure. Thus we need to handle it as described in the blog post Testing Edge Data Cases with Network Stubbing and App Actions. We will expect uncaught:exception event and will confirm it has our test error message and not some other actual application error.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
it('sends an error to Sentry', () => {
cy.route('POST', 'https://sentry.io/api/*/store/*', {
id: 'abc123',
}).as('sentry')

cy.on('uncaught:exception', e => {
// only ignore OUR test error message
return e.message === 'test error'
})

// create an error, as if application has thrown it
cy.window().invoke(
'setTimeout',
() => {
throw new Error('test error')
},
1000,
)
// confirm the call has happened
cy.wait('@sentry')
})

The test passes. Let's confirm some of the fields the Sentry browser client sends to the service - it should have at least the error message.

1
2
3
4
5
6
7
8
9
10
11
// confirm the call has happened
cy.wait('@sentry')
.its('requestBody')
.should(body => {
expect(body.level).to.equal('error')
expect(body.exception.values).to.have.length(1)
expect(body.exception.values[0]).to.deep.contain({
type: 'Error',
value: 'test error',
})
})

Passing test asserting Sentry is working correctly

Tip: writing XHR assertions this way is cumbersome, I suggest using helper library cy-spok in the blog post Asserting Network Calls from Cypress Tests.

See also