process

Mock system APIs

Do not mock your internal modules, mock system APIs instead!

When you write your module (let's say a Node JavaScript one), you often expose public API, but internally your code accesses the outside world via own code. Testing the public API is the best, but often it is hard, since your code interacts with the file system, user or network.

How do you test the exposed API when you need to mock the rest of the system? I think it is preferable to NOT mock your own code modules, but to mock the interface to the external world.

In principle, if your public API calls two internal functions fn1 and fn2, which in turn call Node system libraries (like fs or http), then you probably should stub the Node system libraries and NOT the internal functions fn1 and fn2.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
   public API     <-----------------------+ Testing public API
        ^                                      (preferred)
+-------|-----------------------+
|       |                       |
|       +                       |
|                               |
|     Your top level code       |
|                               |
|                               |
+------------------+------------+ <-------+ Mocking internal modules
|                  |            |             (hard, can change)
|                  |            |
|     Your fn1     |  Your fn2  |
|                  |            |
|        +         |     +      |
+--------|---------+-----|------+
         |               |
+--------|---------------|------+ <-------+ Mocking system APIs
|        +               +      |           (surprisingly easy,
|  OS / Node stable public API  |            do not change often)
+-------------------------------+

hint the above diagram was created using AsciiDraw

Example

My examples comes from my own library wrapping the Git tool ggit. The library has many public functions in its public API, methods like blame and commits. A typical use case is someone writing a Node program showing history for each line of a tracked file

1
2
3
4
5
6
7
8
9
10
11
12
const blame = require('ggit').blame
blame(filename, lineNumber).then(function (info) {
  /*
    info is an object with fields like
    { commit: '6e65f8ec5ed63cac92ed130b1246d9c23223c04e',
      author: 'Gleb Bahmutov',
      committer: 'Gleb Bahmutov',
      summary: 'adding blame feature',
      filename: 'test/blame.js',
      line: 'var blame = require(\'../index\').blame;' }
  */
});

How can we test this public function from ggit? Well, src/blame.js makes use of two calls internally - one is to internal module ./exec.js which routes to Node's built-in child_process.exec, and another directly to Node's built-in fs.existsSync function. Do we somehow overwrite calls to ./exec.js with stubs controlled from the tests? That is difficult, and requires exposing ./exec.js module and being able to overwrite it on the fly. One can use helpers that clear Node module cache and then force loading ./exec.js again, but substituting the stubbed version. One could use proxyquire or really-need to do this, and the test would look like this

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
const reallyNeed = require('really-need')
describe('blame', () => {
  const filename = 'foo.txt'
  const line = 10
  beforeEach(() => {
    // need to clear './blame.js' from module cache
    // need to clear './exec.js' from module cache
    // load mocked version of './exec.js'
    // somehow mock function that calls `fs.existsSync` inside ./blame.js
  })
  it('returns blame information for given line', () => {
    return blame(filename, line)
      .then(info => {
        // check info object
      })
  })
})

Painful, isn't it? This mocking targets immediate dependencies of the module under test, but their are internal modules that should be replaceable easily. Yet now, the test makes it hard to test ./exec.js or replacing it with a different, better module, like execa. It is hard, because changing it requires reworking lots and lots of complicated tests that mostly setup mocks of our unstable code! The person doing the refactoring has to update tests and make sure the Node module cache is used correctly, and the modules under test are loaded in the right order. I often feel like pulling so much dead weight when refactoring tests, just like this tractor.

Updating tests often feels like pulling heavy weight

Mocking system libraries

Instead of mocking (or stubbing) the immediate internal modules around the module under test, let us mock the system APIs our code is expected to use. For example, instead of mocking ./exec.js function, we can mock the child_process.exec that will be called at some point in order for our module to work.

Mocking a simple method

First, let me show the simple stubbing of system dependency - overwriting fs.existsSync call. We can use the excellent Sinon library to do this quickly.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
const sinon = require('sinon')
// we need access to "fs" object
const fs = require('fs')
describe('blame', () => {
  const filename = 'foo.txt'
  const line = 10
  beforeEach(() => {
    sinon.stub(fs, 'existsSync').withArgs(filename).returns(true)
  })
  afterEach(() => {
    // remove sinon's stubs
    fs.existsSync.restore()
  })
  it('returns blame information for given line', () => {
    return blame(filename, line)
      .then(info => {
        // check info object
      })
  })
})

All we have do to make sure the call fs.existsSync('foo.txt') returns true for fake file foo.txt is to stub it.

1
sinon.stub(fs, 'existsSync').withArgs(filename).returns(true)

Of course after we are done with a test we should restore the original fs.existsSync method. Or we can only stub the first call - but we should still cleanup the stub after the test

1
2
3
4
sinon.stub(fs, 'existsSync')
  .withArgs(filename)
  .onFirstCall()
  .returns(true)

Similarly, we can stub a property or promise-returning method

1
2
3
sinon.stub(object, "propertyName", newValue)        // mock property value
sinon.stub(object, "methodName")
  .resolves(newValue) // mock promise-returning method

For full documentation see sinon stubs

Mocking event emitter

Mocking Node's child_process.spawn method is slightly more complex, because it returns a ChildProcess event emitter. But I have written a utility that does exactly that - stubs spawn or exec system functions. I even made stub-spawn-once testing-friendly because it automatically cleans up each stub after using it once.

For example, the ./blame.js calls ./exec.js with the following command (assuming "foo.txt" filename and line number 10)

1
git blame --porcelain -L 10,10 foo.txt

Then git blame command produces something that looks like this

1
2
3
4
5
6
7
8
9
10
11
12
6f272d6aef8a1d19e559792d9493d07d4218ea09 10 10 1
author Gleb Bahmutov
author-mail <[email protected]>
author-time 1499625750
author-tz -0400
committer Gleb Bahmutov
committer-mail <[email protected]>
committer-time 1499625750
committer-tz -0400
summary this is initial commit
boundary
filename foo.txt

Our test then can mock both fs.existsSync and child_process.exec Node built-in API functions like this

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
const sinon = require('sinon')
// we need access to "fs" object
const fs = require('fs')
const {stubExecOnce} = require('stub-spawn-once')
// common-tags is a great library for dealing with whitespace
// in ES6 template literals
const {stripIndent} = require('common-tags')
describe('blame', () => {
  const filename = 'foo.txt'
  const line = 10
  beforeEach(() => {
    sinon.stub(fs, 'existsSync').withArgs(filename).returns(true)
    const cmd = 'git blame --porcelain -L 10,10 foo.txt'
    const blameOutput = stripIndent`
      // the mock output shown above
    `
    stubExecOnce(cmd, blameOutput)
  })
  afterEach(() => {
    // remove sinon's stubs
    fs.existsSync.restore()
    // stubExecOnce removes itself after single use
  })
  it('returns blame information for given line', () => {
    return blame(filename, line)
      .then(info => {
        // check info object
      })
  })
})

Note that we no longer care or depend on our internal modules. Instead we only care about interacting with the operating system. This is much easier to maintain. Think about a developer updating a test for ./blame.js, he or she can see the expected Git command, and its mock output, and can even run the command in the terminal!

And if we change the internal ./exec.js code or even replace it with another library for running shell scripts, our tests still going to execute the same operating system calls, and should be able to parse the same command output. Truly the ./blame.js test is now testing only a single source file and not several.

Examples

Take a look at these spec files showing mocking stable Node API calls, rather than internal modules.

  • ggit/blame-spec.js

    The above example with snap- and schema-shot testing (see the "Bonus" section below)

  • cypress-io/env-or-json-file

    Stubs file load like

    1
    2
    3
    4
    5
    sinon.stub(fs, 'existsSync').withArgs(fullPath).returns(true)
    sinon
      .stub(fs, 'readFileSync')
      .withArgs(fullPath, 'utf8')
      .returns(configString)

  • generator-node-bahmutov

    Shows how easy it is to stub HTTP(s) requests using nock

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    const nock = require('nock')
    const description = 'cool project, bro'

    beforeEach(() => {
      nock('https://api.github.com')
        .get('/repos/no-such-user/does-not-exist')
        .reply(200, { description })
    })

    it('can mock non-existent repo', () => {
      const url = '[email protected]:no-such-user/does-not-exist.git'
      return repoDescription(url).then(text => {
        la(description === text, 'wrong description returned', text)
      })
    })

    Every nock intercept is automatically removed after single use, so nothing needs to be cleaned up.

and lots more examples in their own repo node-mock-examples.

hint if you do not know what HTTP requests are actually happening, run the tests with DEBUG=nock.* ... or just DEBUG=nock.interceptor ... environment variables to see Nock's messages.

Final thoughts

  • Mocking your own internal modules is hard: they are cached by the Node module system, unstable, often undocumented.
  • Mocking OS calls is easier; API is stable and well documented
  • Libs like sinon and stub-spawn-once make stubbing API methods a breeze

Bonus

I love using snapshot testing to quickly assert everything about received result. Just wrap the promise returned from the call blame(filename, line)

1
2
3
4
5
6
7
8
9
const snapshot = require('snap-shot')
describe('blame', () => {
  beforeEach(() => {
    // ...
  })
  it('returns blame information for given line', () =>
    snapshot(blame(filename, line))
  })
})

Similarly, we can skip mocking even OS results by using schema snapshots in this case. We are interested in the schema of the result, rather than exact values.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
const schemaShot = require('schema-shot')
describe('blame', () => {
  // no setup is necessary!
  it('returns blame information for this file', () =>
    schemaShot(blame(__filename, 1))
  })
})
/*
saves schema:
$ cat __snapshots__/blame-spec.js.schema-shot
exports['gets blame for 1 line of this file 1'] = {
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "object",
  "properties": {
    "commit": {
      "type": "string",
      "required": true
    },
    "author": {
      "type": "string",
      "required": true
    },
    "committer": {
      "type": "string",
      "required": true
    },
    "summary": {
      "type": "string",
      "required": true
    },
    "filename": {
      "type": "string",
      "required": true
    },
    "line": {
      "type": "string",
      "required": true
    }
  },
  "additionalProperties": false,
  "list": false,
  "example": {
    "commit": "adfb30d5888bb1eb9bad1f482248edec2947dab6",
    "author": "Gleb Bahmutov",
    "committer": "Gleb Bahmutov",
    "summary": "move blame test",
    "filename": "spec/blame-spec.js",
    "line": "\tconst la = require('lazy-ass');"
  }
}
*/

Happy testing!