process

How to write end-to-end test using app and api actions

How to bypass user interface when writing end-to-end tests.

Note: the source code for this blog post is in repo cypress-io/cypress-example-realworld in the pull request #55.

End-to-end browser tests do not have to go every time through the DOM interface to exercise the web application. In fact, doing so would make the end-to-end tests terribly slow and inefficient. In this post I will show a concrete example that bypasses the HTML interface for all the but the first test, yet keeps covering the same amount of code (measured by collecting end-to-end code coverage), and runs much much faster.

First test

Our target application is a clone of the Conduit blog web application, and we want to confirm that we can write a new article. Using Cypress Test Runner we can write our first "writes a post" test.

new-post-spec.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
describe('New post', () => {
  beforeEach(() => {
    cy.task('cleanDatabase')
    cy.registerUserIfNeeded()
    cy.login()
  })

  it('writes a post', () => {
    // I have added "data-cy" attributes
    // following Cypress best practices
    // https://on.cypress.io/best-practices#Selecting-Elements
    cy.get('[data-cy=new-post]').click()

    cy.get('[data-cy=title]').type('my title')
    cy.get('[data-cy=about]').type('about X')
    cy.get('[data-cy=article]').type('this post is **important**.')
    cy.get('[data-cy=tags]').type('test{enter}')
    cy.get('[data-cy=publish]').click()

    // changed url means the post was successfully created
    cy.location('pathname').should('equal', '/article/my-title')
  })
})

The test passes

Passing Cypress test that writes a post

The above test is already pretty solid:

  • the initial state is set using beforeEach callback, ensuring the test starts every time from a clean slate
  • the test uses data-cy attributes to find elements following Best Practices

Because we can measure code coverage from Cypress tests, just by running this single test we get 54.2% of all front-end code statements covered.

New post test covers a lot of front-end code

Second test

Great, writing a new post works. Let's see if we can comment on a post. Hmm, to add a comment we need a blog post. We can copy the above test and just add a few additional Cypress commands to add a comment.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
it('writes a post and comments on it', () => {
  cy.get('[data-cy=new-post]').click()

  cy.get('[data-cy=title]').type('my title')
  cy.get('[data-cy=about]').type('about X')
  cy.get('[data-cy=article]').type('this post is **important**.')
  cy.get('[data-cy=tags]').type('test{enter}')
  cy.get('[data-cy=publish]').click()

  // changed url means the post was successfully created
  cy.location('pathname').should('equal', '/article/my-title')

  // comment on the post
  cy.get('[data-cy=comment-text]').type('great post 👍')
  cy.get('[data-cy=post-comment]').click()

  cy.contains('[data-cy=comment]', 'great post 👍').should('be.visible')
})

The test runs creating a new blog post and then commenting on it.

Writing a post and commenting on it

After this test finishes, the total code coverage increases by 3%

Code coverage after the above test

We can see the new lines covered in the "Article" reducer, the "ADD_COMMENT" action has been covered by the new commands.

Additional line covered by the test

Nice.

Page object

Yet, there is a problem. The second test "writes a post and comments on it" is exactly 70% line for line matching the first test "writes a post". We are creating the post by clicking and typing on the page - repeating exactly the same page actions as the first test. What have we learned from about 2 seconds it takes to type the new post (just like a real user would type character by character), that we don't know already from the first test?

Nothing.

So to remove the duplicate lines, people write Page Objects, a wrapper around the HTML and the elements of the a particular page. We avoid the code duplication by having our tests call into the Page Object wrapper. In our case, the test would be

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
it('writes a post (via page object) and comments on it', () => {
  // page object encapsulating code for writing a post
  // by executing page commands = DOM actions
  const editor = {
    writeArticle () {
      cy.get('[data-cy=new-post]').click()

      cy.get('[data-cy=title]').type('my title')
      cy.get('[data-cy=about]').type('about X')
      cy.get('[data-cy=article]').type('this post is **important**.')
      cy.get('[data-cy=tags]').type('test{enter}')
      cy.get('[data-cy=publish]').click()

      // changed url means the post was successfully created
      cy.location('pathname').should('equal', '/article/my-title')
    }
  }

  // use "Editor" page wrapper to write a new post
  editor.writeArticle()

  // comment on the post
  cy.get('[data-cy=comment-text]').type('great post 👍')
  cy.get('[data-cy=post-comment]').click()

  cy.contains('[data-cy=comment]', 'great post 👍').should('be.visible')
})

The above test works, and removes code duplication. Every time you need an article to test commenting on it, or to test how an article can be deleted, or how a user can like it - every test can just call editor.writeArticle() and be done.

App action

Let's see why the Page Object is less than ideal.

  • The commands going through the DOM are slow. We will see how to avoid it later.
  • Opening the editor and typing into the input boxes again and again from every test does not help us test better, because it is just redundant commands that do the same thing over and over.
  • The Page Object is an extra layer of code that does not benefit the users and is built on top of the HTML, and can be only tested at runtime without any static tools help

Once we have "writes a post" test that goes through the DOM to confirm that a user can write a new article, there is no point doing it again. Instead we can directly create an article by accessing the underlying application code and calling its methods - I call this approach "app actions", and argue in the blog post Stop using Page Objects and Start using App Actions that this approach saves you time and removes an unnecessary level of code sitting on top of the HTML. And this is not about code duplication - yes, you could factor out writing the post into a single reusable function (this would be a Page method). The point is that we want to avoid going through the DOM completely to perform an action that we have already tested!

In practice, this means we need to access the underlying web application to create the post somehow. This is how we do it.

First, study the application code to see how the UI components trigger actions. In our case the Editor.js component submits an article via an "agent" object reference.

src/components/Editor.js
1
2
3
4
5
6
7
8
9
10
11
12
13
import agent from '../agent'
// Editor component
this.submitForm = ev => {
  ev.preventDefault()
  const article = {
    title: this.props.title,
    description: this.props.description,
    body: this.props.body,
    tagList: this.props.tagList
  }
  const promise = agent.Articles.create(article)
  this.props.onSubmit(promise)
}

Ok, so if we could call agent.Articles.create method directly from our Cypress test, we could create an article almost instantly. Let's pass this agent reference from the application to the test via window object.

src/index.js
1
2
3
4
import agent from './agent'
if (window.Cypress) {
  window.agent = agent
}

The test can grab the window object, then its agent property and call the application action.

1
2
3
4
5
6
7
8
9
10
it('writes a post (via app action) and comments on it', () => {
  const article = {
    title: 'my title',
    description: 'about X',
    body: 'this post is **important**.',
    tagList: ['test']
  }
  cy.window().its('agent.Articles')
    .invoke('create', article) // resolves with new article object
})

I can also see by inspecting the code that the Editor executes the following code after the agent submits the article

1
2
3
case ARTICLE_SUBMITTED:
  const redirectUrl = `/article/${action.payload.article.slug}`;
  return { ...state, redirectTo: redirectUrl };

Ok, we can do the same thing - we can use the result returned by the agent to get the new article's slug and redirect to the article url. Thus the full test should redirect to the article url like this

1
2
3
4
5
6
7
8
9
10
11
cy.window().its('agent.Articles')
  .invoke('create', article) // resolves with new article object
  .its('article.slug')
  .then(slug => {
    cy.visit(`/article/${slug}`)
  })
// comment on the post
cy.get('[data-cy=comment-text]').type('great post 👍')
cy.get('[data-cy=post-comment]').click()

cy.contains('[data-cy=comment]', 'great post 👍').should('be.visible')

The test runs and passes - creating the new post instantly.

Creates new post by calling app code

If we run both tests "writes a post" and "writes a post (via app action) and comments on it" togher - we get exactly the same code coverage percentage 57.3% - because we literally covered the same statements as before - we just did not cover some of the twice.

Api action

When we bypass the DOM to set our state (we need an article) before testing a feature (like adding a new comment), we do not even have to use the application code. By studying the Network tab and the application code we can see the HTTP POST request that happens when a new article is sent to the server.

POST article XHR call

Super, we can execute the same call ourselves. Here is the custom command (for reusability) and the test

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
Cypress.Commands.add('postArticle', fields => {
  checkArticle(fields)
  const jwt = localStorage.getItem('jwt')
  expect(jwt, 'jwt token').to.be.a('string')

  cy.request({
    method: 'POST',
    url: `${apiUrl}/api/articles`,
    body: {
      article: fields
    },
    headers: {
      authorization: `Token ${jwt}`
    }
  })
})

it('writes a post (via API) and comments on it', () => {
  const article = {
    title: 'my title',
    description: 'about X',
    body: 'this post is **important**.',
    tagList: ['test']
  }
  cy.postArticle(article)
    .its('body.article.slug')
    .then(slug => {
      cy.visit(`/article/${slug}`)
    })
  // comment on the post
  cy.get('[data-cy=comment-text]').type('great post 👍')
  cy.get('[data-cy=post-comment]').click()

  cy.contains('[data-cy=comment]', 'great post 👍').should('be.visible')
})

Super, the test passes, and we have a reusable custom command cy.postArticle we can use any time we need an article to comment on, or to like, or to delete - thus all our end-to-end tests can run fast, yet because we have already tested creating a new article, the code coverage stays complete.

For me, the rule of thumb is:

  • when testing feature A, use the DOM just like a real user would.
  • when testing feature B, that needs something feature A does, bypass the DOM when achieving part A. Instead call the application code directly, just like the UI component that implements feature A would. This saves a lot of time, making your tests fly, yet does not diminish the test code coverage.