products

E2E Testing json-graphql-server using Cypress

How to test TodoMVC application that uses GraphQL.

This blog post explains how Cypress.io test runner can test an application that uses GraphQL to load / save data on the server. The backend will be done using json-graphql-server and the E2E tests will load that server into the browser. You can find the source code at https://github.com/bahmutov/todo-graphql-example. Let's start.

GraphQL backend

My demo application is a regular TodoMVC application with React frontend and GraphQL backend. To simplify the backend programming, I am using json-graphql-server. The entire "database" file db.js has only 2 todo items initially

db.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
module.exports = {
  todos: [
    {
      id: 1,
      title: 'use GraphQL',
      completed: false
    },
    {
      id: 2,
      title: 'write React frontend',
      completed: false
    }
  ]
}

Running json-graphql-server db.js gives me instantly a GraphQL API. I can open localhost:3000 and play with GraphQL queries. For example to load all todo items:

1
2
3
4
5
6
7
query {
  allTodos {
    id,
    title,
    completed
  }
}

which returns JSON response

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
{
  "data": {
    "allTodos": [
      {
        "id": "1",
        "title": "use GraphQL",
        "completed": false
      },
      {
        "id": "2",
        "title": "write React frontend",
        "completed": false
      }
    ]
  }
}

If I accidentally try to reference an invalid field, the response will contain "errors" list

1
2
3
4
5
6
7
query {
  allTodos {
    nope,
    title,
    completed
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
{
  "errors": [
    {
      "message": "Cannot query field \"nope\" on type \"Todo\".",
      "locations": [
        {
          "line": 3,
          "column": 5
        }
      ]
    }
  ]
}

Application frontend

To write frontend to my application I picked React with apollo-boost and react-apollo. To serve the application I will use Parcel bundler.

package.json
1
2
3
4
5
6
{
  "scripts": {
    "start": "json-graphql-server db.js",
    "app": "parcel serve src/index.html"
  }
}

The src/index.html just includes placeholder element with id=root and index.js bundle.

src/index.html
1
2
3
4
5
6
7
8
9
10
11
12
<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />
    <title>GraphQL TodoMVC Example</title>
  </head>
  <body>
    <div class="todoapp" id="root"></div>
    <script src="index.js"></script>
  </body>
</html>
src/index.js
1
2
3
4
5
6
import React from 'react'
import { render } from 'react-dom'
import 'todomvc-app-css/index.css'
import App from './App'

render(<App />, document.getElementById('root'))

The "App" component instantiates the Apollo client and inserts it into React system as a provider.

src/App.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
import ApolloClient from 'apollo-boost'
import React from 'react'
import { ApolloProvider } from 'react-apollo'
import Header from './containers/Header'
import MainSection from './containers/MainSection'

const client = new ApolloClient({
  uri: 'http://localhost:3000'
})

const App = () => (
  <ApolloProvider client={client}>
    <div>
      <Header />
      <MainSection />
    </div>
  </ApolloProvider>
)

export default App

The markup of the app mostly follows TodoMVC with Redux code from Testing Redux Store blog post I have written earlier. The Todos.jsx is fetching the initial list of Todo items for example, and then creating a pure functional component TodoItem.jsx for each returned item.

src/Todos.jsx
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
const Todos = () => (
  <Query
    query={gql`
      {
        allTodos {
          id
          title
          completed
        }
      }
    `}
  >
    {({ loading, error, data }) => {
      if (loading) return <p>Loading...</p>
      if (error) return <p>Error :(</p>
      return (
        <ul className='todo-list'>
          {data.allTodos.map(todo => (
            <TodoItem todo={todo} key={todo.id} />
          ))}
        </ul>
      )
    }}
  </Query>
)
export default Todos

If we start the GraphQL backend and the application, and browse to localhost:1234 we can see the initial items.

Initial todos

You can see that the app renders the loaded items correctly, and that there is the correct GraphQL query at the start of the application.

First test

Let us confirm that our application renders the initial list of Todos correctly. I will use Cypress.io to write my end-to-end tests. First, because it is incredibly powerful. Second, because I spent last year and a half making it more powerful and developer-friendly.

I will install Cypress test runner using npm i -D cypress and will write my first test - just checking if two items load initially. You can find this code in this pull request

cypress/integration/spec.js
1
2
3
4
5
/// <reference types="cypress" />
it('loads 2 items', () => {
  cy.visit('/')
  cy.get('.todo-list li').should('have.length', 2)
})

A few notes about the above test code

  • I have added the special comment /// <reference types="cypress" /> that allows editors that understand this comment syntax (like VSCode, WebStorm) to load TypeScript definitions included with Cypress NPM module. Then if I hover over Cypress commands, I get to see code completion and documentation. For example when I am hovering over should('have.length', 2) assertion, VSCode editor shows information about this specific assertion. More information.

Intelligent help

  • The test visits / url, because I have placed the http://localhost:1234 base url in cypress.json settings file
cypress.json
1
2
3
{
  "baseUrl": "http://localhost:1234"
}
  • The test passes, all seems good

Two items test

Continuous Integration

Before writing any more tests, I will set up continuous integration server to make sure each commit is fully tested. There are Cypress CI examples for many systems, but the simplest one to use is via Cypress CircleCI Orb. I will add the repo bahmutov/todo-graphql-example to CircleCI build and will drop this circle.yml file into my repo

circle.yml
1
2
3
4
5
6
7
version: 2.1
orbs:
  cypress: cypress-io/[email protected]
workflows:
  build:
    jobs:
      - cypress/run

Note: in order to use 3rd party orbs (like the Cypress CircleCI Orb), you need to go to the organization (or user) Security settings on CircleCI page and enable this setting

Two items test

The configuration file uses the [Cypress Circle Orb][cypress orb registry] that comes from CircleCI registry. Because I am using a specific version cypress-io/[email protected] of the orb, I am isolated from any configuration changes until I decide to upgrade the orb. The orb takes care of installing Cypress, caching its binary and running cypress run command - I don't have to configure these things from my project!

Hmm, but how will the CI know that it needs to

  • start GraphQL endpoint with npm start command
  • then start the application using npm run app and wait for port 1234 to respond
  • before running Cypress tests?

To start the server in the background before running the tests, and wait for port 1234 to respond, we can use parameters defined in the Cypress orb. I will add another script to the package.json to start both the GraphQL endpoint and the bundler

package.json
1
2
3
4
5
6
7
{
  "scripts": {
    "start": "json-graphql-server db.js",
    "app": "parcel serve src/index.html",
    "server": "npm start & npm run app"
  }
}

The Circle config will start the server (in the background) and wait for local url http://localhost:1234 to respond before starting the Cypress tests

circle.yml
1
2
3
4
5
6
7
8
9
version: 2.1
orbs:
  cypress: cypress-io/[email protected]
workflows:
  build:
    jobs:
      - cypress/run:
          start: npm run server
          wait-on: http://localhost:1234

Great, the test passes on CircleCI - see the result here todo-graphql-example/3

Testing the loading message

When Todos.jsx is loading the initial set of todos, it is showing the "Loading..." message.

Todos.jsx
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
const Todos = () => (
  <Query
    query={gql`
      {
        allTodos {
          id
          title
          completed
        }
      }
    `}
  >
    {({ loading, error, data }) => {
      if (loading) return <p>Loading...</p>
      if (error) return <p>Error :(</p>
      ...

Let us test it. We need to delay the server response somehow in order for this message to appear. We need network control which Cypress provides except for window.fetch calls currently. The network rewrite is in progress which will add support for window.fetch spying and stubbing. For now, we can force GraphQL request from Apollo client to drop from window.fetch to using XMLHttpRequest protocol that is stubbed. Let us do this

cypress/integration/spec.js
1
2
3
4
5
6
7
8
9
it.only('shows loading message', () => {
  cy.visit('/', {
    onBeforeLoad (win) {
      // force Apollo client to use XHR
      delete win.fetch
    }
  })
  cy.get('.todo-list li').should('have.length', 2)
})

Notice in the screenshot below that when Apollo client uses XHR to make GraphQL request, Cypress "sees" it. The XHR details appear in the Command Log on the left, and if you click on the (XHR) command it shows the details of the call in the DevTools console.

XHR GraphQL call

Also, do you see that Cypress is showing DOM snapshot when you click on the (XHR) command? Because Cypress takes DOM snapshots while running the tests, it can travel back in time and show how the application looked during each command of the test. In this case, our application did in fact show "Loading..." message, but it was kind of quick. Let us delay the server response and confirm that the user interface shows this message. To delay the response I need to stub it, and return some mock data. I will use cy.route for it

cypress/integration/spec.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
it.only('shows loading message', () => {
  cy.server()
  cy.route({
    method: 'POST',
    url: 'http://localhost:3000/',
    delay: 1000,
    status: 200,
    response: {
      errors: [],
      data: {
        allTodos: []
      }
    }
  })

  cy.visit('/', {
    onBeforeLoad (win) {
      // force Apollo client to use XHR
      delete win.fetch
    }
  })
  cy.contains('.main', 'Loading...').should('be.visible')
})

The test passes - and I can check that the Loading... goes away too. Because React re-renders the affected DOM tree, the element "Loading..." should disappear from the DOM

1
2
3
4
...
cy.contains('.main', 'Loading...').should('be.visible')
// and then it should disappear from the DOM
cy.contains('.main', 'Loading...').should('not.exist')

Loading shows up and disappears

note: checking the loader using its text content is less maintainable than using a test data attribute. See Selecting Elements section in our Best Practices guide.

While we are working with XHR stub, we can move the data to be returned by the stub out from the spec file and into its own fixture file. I will create cypress/fixtures/empty-list-no-errors.json with the response

cypress/fixtures/empty-list-no-errors.json
1
2
3
4
5
6
{
  "errors": [],
  "data": {
    "allTodos": []
  }
}

In the cy.route response I can use the name of the fixture file directly

1
2
3
4
5
6
7
cy.route({
  method: 'POST',
  url: 'http://localhost:3000/',
  delay: 1000,
  status: 200,
  response: 'fixture:empty-list-no-errors'
})

The test passes the same way.

Adding Todo mutation

Let us add a new todo. We will wire the TodoTextInput.jsx component to send a mutation to the GraphQL endpoint.

TodoTextInput.jsx
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
import gql from 'graphql-tag'
import { Mutation } from 'react-apollo'

// utility to generate random ids for new items
function randomId () {
  return Number(
    Math.random()
      .toString()
      .substr(2, 10)
  )
}

// GraphQL mutation, we expect just "id" back
const ADD_TODO = gql`
  mutation AddTodo($id: ID!, $title: String!) {
    createTodo(id: $id, title: $title, completed: false) {
      id
    }
  }
`

// now the component
export default class TodoTextInput extends Component {
  handleSubmit (addTodo, e) {
    const text = e.target.value.trim()
    if (e.which === 13) {
      addTodo({
        variables: {
          id: randomId(),
          title: text
        }
      })
      if (this.props.newTodo) {
        this.setState({ text: '' })
      }
    }
  }

  render () {
    return (
      // extract mutation from GraphQL client
      <Mutation mutation={ADD_TODO}>
        {addTodo => (
          <input
            className={classnames({
              edit: this.props.editing,
              'new-todo': this.props.newTodo
            })}
            type='text'
            placeholder={this.props.placeholder}
            autoFocus
            value={this.state.text}
            onBlur={this.handleBlur.bind(this)}
            onChange={this.handleChange.bind(this)}
            onKeyDown={this.handleSubmit.bind(this, addTodo)}
          />
        )}
      </Mutation>
    )
  }
}

Whenever there is keyDown event, if the key is Enter (which: 13), we call the mutation ADD_TODO passed as an argument to the handleSubmit method. The mutation can be seen in the Network tab

AddTodo mutation

Notice that the text of the query is sent separately from the variable values. Also notice that the application is NOT showing the new Todo item "baz" in the list. Because we just sent it to the server, but have not told the Apollo Client to refresh the list. The easy way to refetch the data after sending a mutation is to ... tell the client to refetch a specific query after sending a mutation!

Here is our query to get all todo items - I am exporting it now:

Todos.jsx
1
2
3
4
5
6
7
8
9
export const ALL_TODOS = gql`
  query allTodos {
    allTodos {
      id
      title
      completed
    }
  }
`

And I am adding refetchQueries property to the Mutation node

TodoTextInput.jsx
1
2
3
4
5
6
7
8
import { Mutation } from 'react-apollo'
import { ALL_TODOS } from '../Todos'

...
render () {
  <Mutation mutation={ADD_TODO} refetchQueries={[{ query: ALL_TODOS }]}>
  ...
}

The new item will now appear in the list when we enter it in the input field. The Network DevTools tab shows the mutation followed by the query all todos fetch.

AddTodo mutation followed by AllTodos query

Network control limits

Our previous tests were rudimentary. Cypress network stubbing is working in the browser by spying and stubbing the XMLHttpRequest object, and it is very limited. For example, one cannot selectively stub calls based on request body - only based on the HTTP method and url. So our stubbing can tell apart calls like these

  • cy.route('GET', '/todos') from cy.route('POST', '/todos') because they have different HTTP method
  • cy.route('POST', '/todos') from cy.route('POST', '/todos/1') because they have different url

But all GraphQL requests share the same method and url - so they all look the same to the cy.route. Only the request bodies are different - and cy.route does not look at request body when matching the route 🙁.

Future enhancement #687 will allow much more flexible stubbing, but for now we need a different work around.

Run json-graphql-server in the browser

If we have our simple GraphQL server based on JSON object using json-graphql-server, then we can load the same server right in the browser, and mock HTTP calls directly using another library!

So, we will load the json-graphql-server directly from the test, and we will use fetch-mock to replace window.fetch inside application's iframe with a polyfill that will point at that in-browser GraphQL server.

spec.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import fetchMock from 'fetch-mock'
import JsonGraphqlServer from 'json-graphql-server'
// load JSON for GraphQL server
const data = require('../../db')

it('in browser fetch mock', () => {
  cy.visit('/', {
    onBeforeLoad (win) {
      const server = JsonGraphqlServer({ data })
      win.fetch = fetchMock.sandbox()
      // all GraphQL queries go to this endpoint
      win.fetch.post('http://localhost:3000/', server.getHandler())
    }
  })

  cy.get('.todo-list li')
    .should('have.length', 2)
})

The test works - our GraphQL server is running in the browser!

in browser GraphQL json server

Testing GraphQL logic

Now that we can just load our "backend" server right in the browser under test, we can write complex tests. But first, we should move the server setup logic into the common hook to run before each test.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
beforeEach(() => {
  cy.visit('/', {
    onBeforeLoad (win) {
      const server = JsonGraphqlServer({ data })
      win.fetch = fetchMock.sandbox()
      // all GraphQL queries go to this endpoint
      win.fetch.post('http://localhost:3000/', server.getHandler())
    }
  })
})

it('in browser fetch mock', () => {
  cy.get('.todo-list li').should('have.length', 2)
})

We can confirm that a new item gets added to the list - all using in-browser JSON GraphQL server

1
2
3
4
5
6
7
it('adds an item', () => {
  cy.get('.todo-list li').should('have.length', 2)
  cy.get('.new-todo').type('new todo{enter}')
  cy.get('.todo-list li')
    .should('have.length', 3)
    .contains('new todo')
})

Adding todo test using in-browser server

Note: our beforeEach contains a bug. Right now we are loading a single object data that the JsonGraphlServer will mutate if a test adds a new item for example. Thus to make the tests independent we need to clone this object before passing it to the constructor.

1
2
3
4
5
onBeforeLoad (win) {
  // avoid mutating global data singleton
  const copied = Cypress._.cloneDeep(data)
  const server = JsonGraphqlServer({ data: copied })
}

Next, let's confirm that GraphQL requests really happen when we add a new Todo item. We need a reference to the win.fetch mock sandbox. We can save it in the scope as let fetches variable.

1
2
3
4
5
6
7
8
9
10
11
12
let fetches

beforeEach(() => {
  cy.visit('/', {
    onBeforeLoad (win) {
      const server = JsonGraphqlServer({ data })
      fetches = win.fetch = fetchMock.sandbox()
      // all GraphQL queries go to this endpoint
      win.fetch.post('http://localhost:3000/', server.getHandler())
    }
  })
})

Our test checks the fetches.calls() array's length after the DOM gets updated, using cy.then(cb) to schedule the assertion.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
it('tracks number of GraphQL calls', () => {
  // just loads all todos
  expect(fetches.calls()).to.have.length(1)

  cy.get('.todo-list li').should('have.length', 2)
  cy.get('.new-todo').type('new todo{enter}')
  cy.get('.todo-list li')
    .should('have.length', 3)
    .contains('new todo')
    .then(() => {
      // mutation to add todo + query all todos again
      expect(fetches.calls()).to.have.length(3)
    })
})

What if we want to see and test the actual GraphQL queries from our application? No problem, we can extract the important GraphQL information from each call and assert the expected data. We just need to see what the call fetches.lastCall() returns. Because Cypress runs the real browser, just open the DevTools, add debugger keyword to pause the execution and inspect the returned value:

Inspecting GraphQL call recorded by fetch mock

Perfect, we can write a tiny helper function to extract GraphQL operation name and variables - the important parts of the query. To make accessing nth call easier we need a helper method, and it should make it convenient to grab calls that have happened last.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
/**
 * Parses array returned by "fetch-mock" to get GraphQL information
 */
const extractGraphQL = call => {
  // only interested in request body from [url, body] arguments
  const [, request] = call
  // parse query into JSON and pick two properties
  return Cypress._.pick(JSON.parse(request.body), [
    'operationName',
    'variables'
  ])
}
/**
 * Extracts GraphQL object from fetch-mock calls
 *
 * @param {number} k index of the call to return, pass -1 to get the last call
 */
const nthGraphQL = (k = -1) =>
  cy.then(() => {
    const calls = fetches.calls()
    const nthCall = Cypress._.nth(calls, k)
    if (!nthCall) {
      throw new Error(`Cannot find GraphQL call #${k}`)
    }

    return extractGraphQL(nthCall)
  })

Now we can confirm the calls that our web application sends to GraphQL endpoint

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
it('uses expected GraphQL operations', () => {
  // during the application load, app queries all todos
  nthGraphQL().should('deep.equal', {
    operationName: 'allTodos',
    variables: {}
  })

  cy.get('.todo-list li').should('have.length', 2)
  cy.get('.new-todo').type('new todo{enter}')

  cy.get('.todo-list li')
    .should('have.length', 3)
    .contains('new todo')

  // addTodo mutation call
  nthGraphQL(-2).should('deep.equal', {
    operationName: 'AddTodo',
    variables: {
      title: 'new todo'
    }
  })

  // allTodos query call
  nthGraphQL(-1).should('deep.equal', {
    operationName: 'allTodos',
    variables: {}
  })
})

Hmm, the test fails - because our application generates a random ID for each item.

Random ID is a problem for deep equality

Luckily, we can make our test deterministic using the same approach as in this post. We can simply reach into the application's context and override Math.random method during tests. Then each new item will get a nice deterministic id our tests can compare against.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
it('uses expected GraphQL operations', () => {
  // application's random generator ignores first two digits
  // so our fake ids will be with 100, 101, 102, ...
  let counter = 10100
  cy.window()
    .its('Math')
    .then(Math => {
      cy.stub(Math, 'random').callsFake(() => counter++)
    })

  // during the application load, app queries all todos
  nthGraphQL().should('deep.equal', {
    operationName: 'allTodos',
    variables: {}
  })

  cy.get('.todo-list li').should('have.length', 2)
  cy.get('.new-todo').type('new todo{enter}')

  cy.get('.todo-list li')
    .should('have.length', 3)
    .contains('new todo')

  // addTodo mutation call
  nthGraphQL(-2).should('deep.equal', {
    operationName: 'AddTodo',
    variables: {
      id: 100, // id is no longer random
      title: 'new todo'
    }
  })

  // allTodos query call
  nthGraphQL(-1).should('deep.equal', {
    operationName: 'allTodos',
    variables: {}
  })
})

We could even assert that the random generator stub was really called, if we want, because Cypress keeps track of all cy.spy and cy.stub methods.

Finished test

Reset fetch mock on page reload

If we decide to test how the web application "remembers" the new item on window reload, we will hit a problem. Imagine we add a new test:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
it('shows new item after reload', () => {
  // starts with new item
  cy.get('.todo-list li').should('have.length', 2)
  cy.get('.new-todo').type('new todo{enter}')

  // now has 3 items
  cy.get('.todo-list li')
    .should('have.length', 3)
    .contains('new todo')

  // shows 3 items after the user reloads the page?
  cy.reload()
  // still 3 items after page reload
  cy.get('.todo-list li').should('have.length', 3)
})

The test fails to load mocked GraphQL server - because cy.reload deletes our stubbed window.fetch method we have set in cy.visit call. We need to set window.fetch = ... again, but cy.reload does not take an options object with onBeforeLoad callback like cy.visit. How can we attach the mocked fetch to the window object?

By using cy.on('window:before:load', ...) event. Here is an updated test that passes

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
it('shows new item after reload', () => {
  // starts with new item
  cy.get('.todo-list li').should('have.length', 2)
  cy.get('.new-todo').type('new todo{enter}')

  // now has 3 items
  cy.get('.todo-list li')
    .should('have.length', 3)
    .contains('new todo')

  // shows 3 items after the user reloads the page?

  // currently deletes the window.fetch mock
  // so we need to set it again before the window loads
  cy.on('window:before:load', win => {
    // fetches was created in `cy.visit` callback
    win.fetch = fetches
  })
  cy.reload()
  // still 3 items after page reload
  cy.get('.todo-list li').should('have.length', 3)
})

Reload test

There is an entire catalogue of evens in Cypress documentation.

Related info