- Intro
- Functional approach to the boilerplate removal
- Removing boilerplate in asynchronous code
- Boilerplate in the unit tests
- Boilerplate and ES6
- Additional tools
- Conclusions
You can find the slides to go with this blog post here
Intro
We write too much code. Each day we pump hundreds if not thousands of new lines, trying to add new features, fix bugs or improve test coverage. This is not just simple code duplication - this is a code that takes more characters than necessary to achieve its goals.
Each new variable, function and branch can interact with any other entity, all the way up
the lexical scope chain until the global
object. In the following code, there are
only a few separate variables: a
, b
, add
and an anomyous functional expression.
var add = function (a, b) {
return a + b;
};
The four entities can interact with the other three, leading to 4x3 possible connections. As the number of tokens in the program grows to N, there are N x (N-1) interactions, which makes it O(n^2) in possible complexity. Every potential interaction makes the code more brittle, harder to understand, test and write. The complexity hides defects, logical contradictions, and ultimately errors. We can say that
Errors = more code^2
or in its more famous notation e = mc^2
Let me show how to remove a lot of boilerplate code from your programs, while preserving or even clarifying its meaning. We are going to use a couple of approaches to remove the boilerplate code: partial application of arguments (built into the JavaScript language), functional programming, the intelligent order of arguments in the API functions, promises. Our ultimate goal is to remove code, while making the its meaning clear and unambiguous.
Example: forming full path to a file
1 | // verbose approach |
Every time we need a full file path relative to the current directory, we can potentially trip over
3 items: path
variable, join
method, __dirname
argument. We can remove the first item easily
because method path.join
does not use this
anywhere.
1 | // removed object variable |
This is better, but we still need to remember to pass the argument __dirname
. We now can use the fact
that Function.prototype.bind
can do the context binding and partial application.
1 | // clean code |
The code clearly expresses what we are trying to compute for each filename. We removed
or at least localized references to the path.join
method, and only used the __dirname
variable once.
Functional approach to the boilerplate removal
Partial application
The above example used partial application - creating a new function from an existing one by binding some of the arguments to the values we already know.
// function join(a, b)
var relativePath = path.join.bind(null, __dirname);
// relativePath(a=__dirname, b)
We can provide the first value to the path.join
method, and then call it again, providing the other
parts of the path. The new function relativePath
will wait to be called again, and then combines
the arguments before calling the path.join
. This is different from the default argument value - we bind
the value at runtime, not when writing the function
// bind at runtime
var relativePath = path.join.bind(null, __dirname);
// not the same as
// default value at write time
function join(a: __dirname, b);
The default left to right partial application works great if the function's signature was designed with this use case in mind. In general I try to follow this advice when designing an API
Place on the left arguments that are likely to be known first
For example, updating user profile information method might need both the user id and new information
1 | function updateUserInfo(userId, newInfo); |
It is likely that we know the user id when the user successfully logs in. Then we can make a new method
for updating the user method using the standard .bind
method
1 | // user-service.js |
Selective application
If we know the value of an argument that is on the right position, we can still do
partial application, but we must use a helper library (please don't write partialRight
yourself,
there are already lots of libraries for this). For example, Lodash has
_.partialRight function
1 | ['1', '2', '3'].map(parseInt); // [1, NaN, NaN] |
What if we want to bind arguments by position? Lodash can leave empty places now
1 | ['1', '2', '3'].map(_.partial(parseInt, _, 10)); // [1, 2, 3] |
Even less boilerplate if you use spots which does both argument binding with placeholders and context bind
1 | var S = require('spots'); |
Authenticating users in Express routes example:
1 | // same callbacks to check if the user is logged in and authorized |
Selective application by name
Sometimes we wish to bind values by argument name, and not by its position, similarly to the AngularJS dependency injection mechanism. We can easily do this using heroin
1 | // divide by 10 |
Selective application by property name
As the number of arguments grows, it becomes quite difficult to manage them. Any function with more than 2 arguments I switch to using an options object. We can partially apply properties in the options object too using obind
1 | var obind = require('obind'); |
Point-free programming
Let us go back to updating the user information example
1 | // user-service.js |
Notice that function onSubmitted
takes a single argument and simply calls updateUser
function with the same argument. We don't need a separate onSubmitted
!
1 | $('form').on('submit', updateUser); |
By adapting the updateUserInfo
function and using it as a callback, we have achieved a
point-free programming style - avoiding using unnecessary arguments and variables.The
values are simply passed from one function to another, but hidden from our program's eyes.
You can read more about the point-free programming in JavaScript here. The main difficulty is usually matching and adapting the signatures of functions that have to work together. Read [Adapted point-free callbacks][../adapted-point-free-callbacks/] for more examples.
For example, the above example most likely has a first argument that makes our point-free programming hard to achieve
1 | // user-service.js |
Luckily, just like we can bind known values, we can ignore some arguments at known positions using ignore-argument
1 | // user-service.js |
ignoreArgument
returns a new function that will call your function but will ignore some of the
outer arguments
1 | $('form').on('submit', ignoreArgument(updateUser, true)); |
Another example: configuring and returning a service object to make requests to specific API end points.
1 | function initDataService(urls) { |
can be expressed much shorter
1 | function initDataService(urls) { |
Working with arrays
Imagine we deal with a collection of numbers and need to multiply and print them. We could just write a loop
1 | var numbers = [3, 1, 7]; |
A lot of boilerplate in keeping an index variable, just to iterate over an array. We can use EcmaScript5 array methods
1 | function mul(a, b) { |
We just wrote a lot more boilerplate, right? Yes, but we can also reuse all these short simple and easy
to test functions. We can also change the order of arguments when calling mul
and apply partial application
to derive point-free style.
1 | function mul(a, b) { |
I sometimes prefer to stress the code readability and natural expression of intent
1 | function mul(a, b) { |
Functional libraries and currying
Array.prototype.map
and Array.prototype.forEach
are nice, but before ES5 people have been writing
map
and other operations as functions combined into libraries (like Underscore and Lodash). These libraries
have hundreds of operations on lists, replacing lots of custom code.
1 | var numbers = [3, 1, 7]; |
We are using Lodash partial method instead of built-in
bind
to save a few characters and make the meaning clearer. If we often have to apply partial
application to our functions, we might make them curried - have the partial application from left
to right built right in.
1 | var numbers = [3, 1, 7]; |
Let us look at another fragment: the _.map
function. It has a signature that is very similar to
the native Array.prototype.map
1 | function cb(item, index, array) { ... } |
The array
variable is at the first position, and the callback is at the second position.
While this is very close to the native function, in practice this breaks the principle that
allows us to quickly remove the boilerplate code by applying the partial application from
the left. To remind us:
Place on the left arguments that are likely to be known first
The array of items is probably generated dynamically and is only known at runtime.
The callback function on the other hand is often known statically! If we want to be able
to quickly bind arguments in the map
, we should have the callback function placed first,
and the array argument second. This is exactly what the functional library
Ramda takes as its main principle. Together with every function
it exports being curried by default, it makes an excellent tool to remove boilerplate code.
1 | var numbers = [3, 1, 7]; |
We used the built-in curried R.multiply
function to create new function inline.
We also have every array iteration function accept callback at the first position, thus
we can even make steps more explicit
1 | var numbers = [3, 1, 7]; |
Composition
The last line in the Ramda example shows something interesting. We have achieved a series of functions that execute each other with the dynamic data only appearing once at the very end. When functions call each other and pass the result back to the outer function, it is a functional composition operation. We can create a single function that performs the same computation and call it with the data.
1 | var numbers = [3, 1, 7]; |
I prefer to work with the reverse of the composition, in order to make the code read as natural as possible
1 | var numbers = [3, 1, 7]; |
I prefer creating small utility functions by currying inline right inside the pipeline, even it is clear what they do. This is how much of my code looks today.
1 | var numbers = [3, 1, 7]; |
A nice utility if I want to debug the intermediate steps in the pipeline is R.tap. The callback function will be passed the value, and the same value will be passed unchanged down the pipeline.
1 | var numbers = [3, 1, 7]; |
You can see another example of the regular code transformed to a pipeline in Imperative to compose example, and see why Ramda wins in my eyes compared to Lodash.
Simplifying logical operations
Ramda also has a set of logical operations that can be used to remove a lot of if - else
branches. For example, we might need complicated precondition to continue inside a function
1 | var numbers = [1, 5, -3, 2, 0, null, 0, 7]; |
Each condition should be a small reusable function
1 | var numbers = [1, 5, -3, 2, 0, null, 0, 7]; |
We can combine AND
predicates using Ramda's allPass function
1 | var numbers = [1, 5, -3, 2, 0, null, 0, 7]; |
Finally, we can make a reusable function to select positive odd numbers
1 | var numbers = [1, 5, -3, 2, 0, null, 0, 7]; |
The final code is very short, testable and hard to misunderstand. Similarly, one can
refactor OR conditions, or even if-else
blocks using
the ifElse function.
Removing boilerplate in asynchronous code
NodeJs asynchronous operation convention is to pass a callback function, leading to verbose, error-prone code. For example writing a simple function to print all the JavaScript files in the current folder requires a lot of boilerplate code (even when using a helper module to fetch the files list)
1 | var glob = require('glob'); |
Each step needs to check if the previous one returned an error, not forget to return, etc. We can do better by using promises, and especially multiple useful methods present in the major libraries, like Q and Bluebird. For example we can convert a node callback to a promise returning method quickly.
1 | var getJavaScriptFiles = require('q') |
Pretty cool, and by using .done()
to finish the chain of promises we are making
sure that any error thrown from any internal function is not quietly swallowed.
A lot of code that uses promises still suffers from boilerplate. Let us take this example. Two asynchronous functions, then print the result and verify it.
1 | var Q = require('q'); |
First, let us start the promise chain from a value without the boilerplate code
1 | var Q = require('q'); |
Second, any function that is NOT the first call, will be part of the promise by default and does not need to make a new promise itself.
1 | var Q = require('q'); |
or even
1 | var Q = require('q'); |
Third, let us remove the return
statement from the print
function, using R.tap
1 | var R = require('ramda'); |
Third, Ramda works great with promise chains, just like it works great with ordinary functions.
1 | var R = require('ramda'); |
The clarity of asynchronous pipelines becomes very obvious in any real-world application. For the most power, check Bluebird .map function that allows to process a list of promises with a given concurrency limit
1 | fs.readdirAsync('.') |
Imagine how much boilerplate this single method removes!
Boilerplate in the unit tests
Use a better framework
If you use Jasmine or QUnit, consider using Mocha unit testing framework. I prefer Mocha because it understands the async promise-returning code natively. Just return a promise from a unit test to let the framework know what to do.
1 | QUnit.test('a test', function(assert) { |
Compare this to Mocha - just return the promise
1 | it('works', function () { |
Write a wrapper that removes the boilerplate
You can also write your own API- or library-specific wrappers for removing unit test boilerplate. For example, we wrote ng-describe that removes a LOT of boilerplate code from AngularJS unit tests
1 | // typical AngularJS unit test |
The same unit test using ng-describe
cuts out all verbose repetitive parts
1 | ngDescribe({ |
Do not think that just because a popular library has a tool (like AngularJs that has angular-mocks for testing), that this tool is the ultimate perfection. You might be able to write a tool that perfectly solves your particular use case.
Remove boilerplate from assertions
The assertions inside unit tests can have a lot of boilerplate if you try to make sure that a failed assertion provides plenty of context to allow quick debugging. For example, just using a predicate generates an empty exception message.
1 | it('does something', function () { |
test "does something" failed
Error:
At most we get the source line from the stack. What is the assertion that failed? What was the value of foo
that we were comparing to the string bar
? We usually write this manually.
1 | it('does something', function () { |
test "does something" failed
Error: expected foo to equal "bar"
We still don't know the value of foo
, so let us add it to the message. Most assertion libraries only
allow a single message argument after the predicate, thus we are forced to concatenate.
1 | it('does something', function () { |
test "does something" failed
Error: expected foo something to equal "bar"
Because foo
could be a complex object, we need to stringify it better to avoid [object Object]
text
1 | it('does something', function () { |
This code is slow, verbose, and forces us to repeat the assertion expect(foo).toEqual(bar)
inside
the error message. We solved this problem in two parts:
lazy assertions
First, we wrote an assertion library that allows unlimited number of arguments after the predicate. The arguments are intelligently stringified if the predicate is false, thus avoiding the performance penalty, yet providing a lot of context.
1 | // require('lazy-ass'); |
See the library at lazy-ass
source code rewriting
Second, in our unit tests we rewrite the source code on the fly to generate the assertion message.
1 | var helpfulDescribe = require('lazy-ass-helpful'); |
When the code runs, the helpfulDescribe
re-evaluates the callback tests
after rewriting
its source. It inserts the predicate expression as string and the values of all variables into the
assertion. The above example thus becomes at runtime
1 | var helpfulDescribe = require('lazy-ass-helpful'); |
Every assertion is expanded to provide values of variables automatically. Check out more examples in lazy-ass-helpful
Boilerplate and ES6
ES6 (or EcmaScript2015) adds a lot of syntactic suger. Some of it is useless in my opinion, like
class
keyword. The most interesting in my opinion are the new concepts from ES6
that remove the boilerplate.
For example, the rest parameter can remove a lot of boilerplate
when getting the list of arguments as an array.
1 | function f(x, y) { |
Using lodash toArray
1 | function f(x, y) { |
Using ES6 rest parameter
1 | function f(x, y, ...a) { |
Similarly, ES6 allows to specify default values for arguments on the right
1 | function f (x, y, z) { |
Computed property names
In ES5 we had to add properties that were computed as separate steps
1 | var foo = '...'; |
In ES6 we can just list them when creating the object
1 | var foo = '...'; |
The property name foo
will be computed.
Parameter context matching
When passing options object, the new parameter matching might be useful.
1 | // in both cases add({ a: 10, b: 20 }) |
Arrow functions
Shorthand for small anonymous functions: preserves this
, returns the last value automatically.
1 | numbers.map(function (v) { return v + 1; }); |
I am not sure this is a great feature. The functions are anonymous, making debugging more difficult. I also prefer using utility functions that are tested as part of the library. For example
1 | var R = require('ramda'); |
Small arrow functions can be useful, but at the cost of readability and testability in my personal opinion. Ramda allows simpler (in my opinion) point-free code.
Additional tools
- changed-complexity reports the difference in complexity in the changed file before a commit. At least answers the questions: how many lines of code I have added or removed?
- jscpd detects the copied code fragments in your repo.
Conclusions
We can remove a lot of custom code using the following principles
- writing small testable functions with a single clear goal
- designing the signatures to allow quick and natural partial application or currying
- combining small functions into pipelines using standard composition
- using promises and helper methods from promise libraries to handle asynchronous code
- designing helper methods for unit testing our code
- using the new ES6 constructs
I hope that you take my advice to heart and start removing more code every day. To help, I tagged more of my blog posts that deal with the boilerplate code, see the list at glebbahmutov.com/blog/tags/boilerplate/.