products

Be careful when running all specs together

A common mistake when using beforeEach hooks in Cypress specs

Example application

In our example application we have two spec files and a support file.

1
2
3
4
5
6
7
8
repo/
  cypress/
    integration/
      spec-a.js
      spec-b.js
    support/
      index.js
  cypress.json

The spec files have two tests each.

cypress/integration/spec-a.js
1
2
3
4
5
6
7
8
9
context('spec a', () => {
  it('works', function () {
    expect(1).to.equal(1)
  })

  it('works 2', function () {
    expect(1).to.equal(1)
  })
})
cypress/integration/spec-b.js
1
2
3
4
5
6
7
8
9
context('spec b', () => {
  it('works', function () {
    expect(1).to.equal(1)
  })

  it('works 2', function () {
    expect(1).to.equal(1)
  })
})

Support file

The support file is initially empty. Let's add a single console.log message.

cypress/support/index.js
1
console.log('support file')

The support file runs in the browser, right before the spec file runs. We can see the support file and the spec file downloaded by the test runner in the DevTools console (blue arrow).

Support file

The two scripts (support file and the spec file) are requested by the test runner via XHR calls and then evaluated.

Support and spec files

The above download mechanism is just an implementation detail. In effect it means the test runner is evaluating scripts in this order:

1
2
<script src="support/index.js"></script>
<script src="integration/spec-a.js"></script>

We can see that both files were bundled using Cypress' built-in preprocessor.

Support file bundle

Spec file bundle

Which means our tests are really running the following concatenated script:

1
2
3
4
5
6
7
8
9
10
11
12
// support/index.js
console.log('support file')
// integration/spec-a.js
context('spec a', () => {
  it('works', function () {
    expect(1).to.equal(1)
  })

  it('works 2', function () {
    expect(1).to.equal(1)
  })
})

Before hooks

Let's place a hook into support file and a hook into the spec file.

cypress/support/index.js
1
2
3
4
console.log('support file')
before(() => {
  console.log('support file: before hook')
})
cypress/integration/spec-a.js
1
2
3
4
5
6
7
8
9
10
11
12
before(() => {
  console.log('spec-a file: before hook')
})
context('spec a', () => {
  it('works', function () {
    expect(1).to.equal(1)
  })

  it('works 2', function () {
    expect(1).to.equal(1)
  })
})

The test spec-a.js runs and the DevTools console prints the expected output

1
2
3
support file
support file: before hook
spec-a file: before hook

This makes sense - the support file comes first, thus its hook is executed before the spec file's hook. Similarly, if we add a hook to spec-b.js it will execute in the same order

cypress/integration/spec-b.js
1
2
3
4
5
6
7
8
9
10
11
12
before(() => {
  console.log('spec-b file: before hook')
})
context('spec b', () => {
  it('works', function () {
    expect(1).to.equal(1)
  })

  it('works 2', function () {
    expect(1).to.equal(1)
  })
})

DevTools console messages:

1
2
3
support file
support file: before hook
spec-b file: before hook

Before hooks when running all specs

Great, but what happens when the user selects "Run all specs" button?

Run all specs button

We see 4 tests finish in the Test Runner, and in the DevTools Network tab we can see the support file plus each spec file requested by the test runner.

Specs requested

The scripts are then evaluated one after another, which is equivalent to running the following test code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
console.log('support file')
before(() => {
  console.log('support file: before hook')
})
before(() => {
  console.log('spec-a file: before hook')
})
context('spec a', () => {
  it('works', function () {
    expect(1).to.equal(1)
  })

  it('works 2', function () {
    expect(1).to.equal(1)
  })
})
before(() => {
  console.log('spec-b file: before hook')
})
context('spec b', () => {
  it('works', function () {
    expect(1).to.equal(1)
  })

  it('works 2', function () {
    expect(1).to.equal(1)
  })
})

The console prints the messages we expect

1
2
3
4
support file
support file: before hook
spec-a file: before hook
spec-b file: before hook

Good, no surprises here, but notice that all hooks executed before all tests. This is noticeable when adding a log message in each test.

all before hooks ran before the tests

If you assume that spec-b: before hook runs AFTER tests from spec-a.js but before tests in spec-b.js, you might get a wrong result here.

BeforeEach hook

Let's switch from before to beforeEach hook.

cypress/support/index.js
1
2
3
4
console.log('support file')
beforeEach(() => {
  console.log('support file: beforeEach hook')
})
cypress/integration/spec-a.js
1
2
3
4
beforeEach(() => {
  console.log('spec-a file: beforeEach hook')
})
...
cypress/integration/spec-b.js
1
2
3
4
beforeEach(() => {
  console.log('spec-b file: beforeEach hook')
})
...

When we run a single spec, the DevTools show the following messages.

Support and spec-a with beforeEach hooks

Spec-b runs the same way by itself.

Now let's run all specs together. Just like before, this will be equivalent to executing this concatenated script.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
console.log('support file')
beforeEach(() => {
  console.log('support file: beforeEach hook')
})
beforeEach(() => {
  console.log('spec-a file: beforeEach hook')
})
context('spec a', () => {
  it('works', function () {
    expect(1).to.equal(1)
  })

  it('works 2', function () {
    expect(1).to.equal(1)
  })
})
beforeEach(() => {
  console.log('spec-b file: beforeEach hook')
})
context('spec b', () => {
  it('works', function () {
    expect(1).to.equal(1)
  })

  it('works 2', function () {
    expect(1).to.equal(1)
  })
})

Do you see the problem? Look at the printed console messages.

All beforeEach hooks executed for each test

Every hook has run for every test. We probably did not mean for the beforeEach hook from spec-b.js to run before every test from spec-a.js, right? But because they were all in the same script at the root level, all three hooks are executed for every test.

Solution

  1. Never use "Run all specs" button. In fact, when you execute cypress run, Cypress never runs all specs together. Instead it executes "support file + spec-a", then it separately executes "support file + spec-b" scripts.

  2. Be wary of placing before or beforeEach hooks at the root level, instead prefer moving them into describe and context suites. This isolates the hooks, limiting them to the tests in that suite.

cypress/support/index.js
1
2
3
4
console.log('support file')
beforeEach(() => {
  console.log('support file: beforeEach hook')
})
cypress/integration/spec-a.js
1
2
3
4
5
6
7
8
9
10
11
12
13
context('spec a', () => {
  beforeEach(() => {
    console.log('spec-a file: beforeEach hook')
  })

  it('works', function () {
    expect(1).to.equal(1)
  })

  it('works 2', function () {
    expect(1).to.equal(1)
  })
})
cypress/integration/spec-b.js
1
2
3
4
5
6
7
8
9
10
11
12
13
context('spec b', () => {
  beforeEach(() => {
    console.log('spec-b file: beforeEach hook')
  })

  it('works', function () {
    expect(1).to.equal(1)
  })

  it('works 2', function () {
    expect(1).to.equal(1)
  })
})

Isolated hooks

When we place the hooks into a suite like above, they apply correctly to the tests inside their suite.