Picking JavaScript testing framework

Which JavaScript testing framework do you recommend?

Short answer: pick Mocha for BDD, pick QUnit for TDD.

Long story: I answered this question many times. Both at MathWorks, when trying to pick a good framework for unit testing, and for personal projects. I looked in detail at features, releases, API, etc. Most often, I picked QUnit - it is well known and has lots of plugins. But it is not the full story.

QUnit

QUnit has not been updated in a long time after reaching 1.0 version. The latest patch 1.1.14 has been released in January 2014, and there have been no API changes. Meanwhile, I had to write several plugins to add features my projects required.

• qunit-promises adds quick promise value assertions.
• qunit-once adds setup and tear down methods that run once before and after all tests in the module.
• qunit-inject adds dependency injection from modules to unit tests without extra closures.
• qunit-helpful rewrites assertions to include the expression statement to remove need to add extra text.

I strongly believe that better async testing (promise support) and single module setup / tear down methods should be part of the core API, not user space.

At this point, I am not using QUnit on node at all. Instead I use my own gt - it runs QUnit tests on Node natively, including code coverage support via istanbul. It also has much reacher API (badly described I must admit) than QUnit - better async and external process testing, even JSON logging testing via bunyan-gt.

Mocha instead of Jasmine

Jasmine is the most popular BDD testing framework, developed at PivotalLabs. It is currently going through a large rewrite, almost reaching 2.x.x version. An interesting feature is its inclusion of spies.

Still, Jasmine async support sucks. The current 1.x.x version is just awful enough that we skip writing async tests! The second generation will become slightly simpler by adding done argument to each unit test (and setup and tear down functions). Today this feels like catching up to yesterday's level.

Take a look at Mocha. It has the same BDD syntax as Jasmine, but offers perfect async promise support. Here is how one can test a promise-returning function

describe('#find()', function(){
  it('respond with matching records', function(){
    return db.find({ type: 'User' }).should.eventually.have.length(3);
  })
});

Beautiful, isn't it? Same story with setup and tear down functions - just return a promise, and the framework will automatically work.

Mocha has another feature that I now appreciate more and more - it is not bundled with any matchers or assertions. You can use any assertion library to check values. To me this is important because I can use my own lazy assertions. This makes our spec and production code look almost the same, generates helpful context information on test failures, and helps with API documentation, see this post.

form-spec.js

var la = require('lazy-ass');
var check = require('check-more-types');
var form = require('./form-config-names');
describe('form config names', function () {
  it('is a function', function () {
    la(check.fn(form));
  });
});

If you need spies in Mocha, I would use sinon.js - it is excellent, provides any spying feature you can think of, and just works.

Other frameworks

If you need to test UI elements, I would take a look at Joel Feenstra's jstestr.

If you develop grunt plugins, nodeunit is the way to go. But the API is pretty limited, so I have not seen any browser library use it.

Conclusion

For testing under node, I would use gt or Mocha. For testing under browser, I would first try testing under node using gt + benv as described in this post. If this does not work, I would use Mocha + Karma test runner.

Update 1

In large projects, it is more likely different teams will pick different testing frameworks. For example, the front end team might pick Jasmine because of the simple Karma integration. The server team might pick QUnit for their tests. I suggest not to fight this and allow different frameworks and unit tests to coexist. In this case, you can use Good Examples approach to allow anyone to add unit tests quickly even if not very familiar with a particular framework.

Update 2

For most Nodejs projects a good unit testing framework plus NPM test scripts is enough to avoid needing grunt or gulp. For example, I like Mocha and install it as a dev dependency in my individual packages.

npm install --save-dev mocha

The install adds Mocha bin file to the list of the package tools, which you can check

$ ls ./node_modules/.bin
_mocha  mocha

Any tool that prefers global installation and has bin file will have a shortcut there. The good thing about these shortcuts - they can be used in the npm script commands directly. For example let us run mocha as the test command

package.json

"scripts": {
  "test": "mocha -R spec src/**/*-spec.js"
}

We can execute mocha unit tests simply by running npm test command.

What if we want mocha to keep watching the source files and rerun unit tests on changes? We can just pass an argument through the test command from the command line without modifying the package.json file

npm test -- --watch

Now Mocha test framework will rerun the unit tests on any change until you press Ctrl+C to kill the process. To simply running unit tests I wrote npm-quick-run - it runs scripts by prefix and passes arguments by adding -- automatically. Thus the same command as above could be shorter

nr t --watch

Update 3

I tried a new test runner called Ava on a small project condition-node-version. You can see the test file here. There is nothing special there yet, but Ava really shines in 2 aspects (at least according to the blog posts and documentation)

• All tests run in parallel and even in separate processes to speed things up. I did not see the speed up, since my unit tests are tiny and do not have any IO; Mocha probably would be faster for small simple tests.
• Ava handles ES6/ES7 natively, transpiling your unit tests. Thus you can return a promise, a generator or use async/await keywords in your unit tests.

// Ava example test: ES7 + Promise is native
test('bar', async t => {
    const bar = Promise.resolve('bar').then(delay(200));
    t.is(await bar, 'bar');
});

The second point is very important if your code base already uses ES6/ES7; I feel like Ava is perfect for projects that look towards the future of JavaScript. On the other hand, it is still missing some features I love about Mocha - like the built-in 2 second timeout for each unit test.

Update 4 - Rocha

Sometimes the tests are not properly isolated from each other. A test leaves some data behind, which allows other tests to pass. This is a bad practice, because it makes refactoring and moving tests around a gamble. In order to flush out these test run order dependencies I wrote Rocha (pronounced Rokka) - a Node wrapper around Mocha. Rocha randomizes the order of unit tests inside each suite before running the tests. If the tests fail, the order will be saved onto a disk, and the same test run order will be used next time. If the tests pass, the saved order file will be deleted and the order will be picked next time randomly again.

The random test ordering hopefully reveals the tests that leave data behind.

Update 5 - compiling ES6 tests for Mocha

If you write main code that is transpiled from ES6 or only uses ES5, but your tests use ES6, they probably do not run on Node 0.12; and setting up transpile for unit tests seems like extra work to me. Luckily it is simple to compile tests on the fly. Install Babel and its preset

npm install --save-dev babel-register babel-preset-es2015

Then add the compiler option to the Mocha command inside package.json

{
  "scripts": {
    "test": "mocha --compilers js:babel-register"
  }
}

Set the preset in .babelrc file

{
  "presets": ["es2015"]
}

See Babel Mocha docs and bahmutov/little-store as an example project.