products

Magic Backed Inspection Mode

Using cypress-magic-backend mode to detect structural changes in API calls.

In my previous blog post Magic Backed For E2E Testing I have shown my new plugin cypress-magic-backend. The plugin can record the network API calls your web application makes during end-to-end tests, then replay them. Mocking network calls makes your front-end tests much faster. In this blog post I will show the other major feature of the plugin: the "Inspect" button labeled 🪄 🧐.

This blog post shows the Inspect button from the cypress-magic-backend plugin

🎁 You can find the source code used for this blog post in the repo cypress-magic-backend-example.

Many things can go wrong

Imagine a web application. It has JavaScript front-end code and it makes calls to the server APIs. If you are just testing the HTML page shown to the user, and things go wrong, where does the problem come from? Imagine the Cypress test was passing before, now it started failing. If we draw a simple diagram, there could be 3 potential error areas:

Test action to page updates code execution

If a previously passing test is failing now, the problem could be in the code blocks "A", "B", or "C"

  • A: The event handler code is handling the user event differently
  • B: The server code is processing the same API request differently
  • C: The page update logic that shows the API response has changed

We want to find the first place the change has happened; obviously if the code in step "A" produced a different API request, then the server is likely to respond with a difference response.

The problem with just testing the page UI

If we simply test the page DOM, then we cannot tell apart errors due to "A", "B", or "C" changes.

cypress/e2e/add-todo.cy.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
beforeEach(() => {
  const mode = Cypress.env('magic_backend_mode')
  if (mode !== 'playback') {
    mode && cy.log(`during the test the mode is "${mode}"`)
    cy.request('POST', '/reset', { todos: [] })
  }
})

it('adds a todo', () => {
  cy.visit('/')
  cy.log('**confirm the items are loaded**')
  cy.get('.loaded')
  cy.get('.new-todo').type('item 1{enter}')
  cy.get('li.todo').should('have.length', 1)
  cy.get('.new-todo').type('item 2{enter}')
  cy.get('li.todo').should('have.length', 2)
  cy.log('**confirm the items are saved**')
  cy.reload()
  cy.get('li.todo').should('have.length', 2)
  cy.contains('li.todo', 'item 1')
  cy.contains('li.todo', 'item 2')
})

The test passes, each API request takes 100ms.

Imagine I have recorded the test execution using the cypress-magic-backend plugin's "🪄 🎥" button.

Recorded the API calls using the cypress-magic-backend plugin

Great, we now have a JSON file with all API calls to the /todos endpoint

cypress/magic-backend/cypress/e2e/add-todo.cy.js_adds_a_todo_api_calls.json
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
{
  "name": "cypress-magic-backend",
  "version": "1.0.3",
  "apiCallsInThisTest": [
    {
      "method": "GET",
      "url": "http://localhost:3000/todos",
      "request": "",
      "response": [],
      "duration": 109
    },
    {
      "method": "POST",
      "url": "http://localhost:3000/todos",
      "request": {
        "title": "item 1",
        "completed": false,
        "id": "7331690400"
      },
      "response": {
        "title": "item 1",
        "completed": false,
        "id": "7331690400"
      },
      "duration": 108
    },
    {
      "method": "POST",
      "url": "http://localhost:3000/todos",
      "request": {
        "title": "item 2",
        "completed": false,
        "id": "7377098672"
      },
      "response": {
        "title": "item 2",
        "completed": false,
        "id": "7377098672"
      },
      "duration": 108
    },
    {
      "method": "GET",
      "url": "http://localhost:3000/todos",
      "request": "",
      "response": [
        {
          "title": "item 1",
          "completed": false,
          "id": "7331690400"
        },
        {
          "title": "item 2",
          "completed": false,
          "id": "7377098672"
        }
      ],
      "duration": 110
    }
  ]
}

Let's say a developer comes in and changes the following single line of the application's app.js file

app.js
1
2
3
4
5
6
const todo = {
  // title: state.newTodo
  text: state.newTodo,
  completed: false,
  id: randomId(),
}

The developer changed the field name title to be text. What does the test show now?

The test is failing because the items no longer have text shown

Hmm, if you are debugging this failing test, where would you start looking? In the front-end code? Backend logic?

Inspect mode

This is where the "Inspect" mode button comes in handy. You MUST have pre-recorded API calls for this button to work. Let's see what it does in our test.

Notice that just happened. We see red triangles 🔺 and 🔻 in the Command Log. The up triangle means the cypress-magic-backend plugin has detected a difference in the structure of the outgoing request. If we click on the message, we will see the difference details:

The application made POST /todos request that was different than before

The diff used by the cypress-magic-backend is different from the standard "object difference" algorithms. It does not compare the values of objects and arrays, just "shapes". In our case, the POST /todos call used to send an object with title, completed, and id properties. Now it is sending text, completed, and id properties. The plugin warns us: object added key "text" and lost key "title".

So it means we know where the root cause of the failed test lies. It is in the frontend JavaScript code block marked "A", since the request body sent to the server API has changed.

The problem is likely to come from the code block reacting to the user input

The Inspect mode shows the differences in the request and response bodies. Here is the 🔻 message output showing similar differences in the response to the POST /todos call:

The server response to POST /todos call has changed

We can even use the names of the changed fields to narrow our search, especially once we inspect the Git changes in the front-end bundle

Commit changes highlight the problem matching the API request differences

If we decide to rename the field, we can proceed with the matching HTML page changes:

1
2
<!-- <label>{{ todo.title }}</label> -->
<label>{{ todo.text }}</label>

The test is passing now, but the plugin is warning us about detected API changes.

The E2E test is passing but we still see the API call differences as warnings

It is important to make a decision about these warnings. Have we handled the title to text changes everywhere in our application? There might be field validation, for example, that no longer applies, since it does not see the field title, so it silently passes. We need to re-record the API calls with the new requests, if we decide to go with the change.

Tip: Inspect mode compares the request and response bodies against the recorded JSON fixtures. If the data is an array, the plugin looks at the array length, and then compares each item in the array using keys and types. Only the first detected change is reported

The test warns about changes in the GET /todos response objects

Sweet.