process

Replacing the wheels on the running car

How to use semantic versioning with external services.

Despite what people might think, NPM has a huge advantage over other systems - it uses semantic versions when installing dependencies. If your module depends on another module "foo", you have to specify which particular version or range of versions is acceptable.

1
2
3
4
5
{
  "dependencies": {
    "foo": "0.1.0"
  }
}

I suppose that your project has unit tests, and once they all pass, you assume that "foo@1.0.0" is the dependency you know works reliably. The "foo" package can have other releases, of course, and you can look them up, I advise using my available-versions for this

1
2
3
4
5
6
7
8
9
10
$ npm i -g available-versions
$ vers [email protected]
foo since 1.0.0
--------------------------
version  age      dist-tag
-------  -------  --------
1.0.1    a month
1.0.2    a month
1.1.0    19 days
1.2.0    10 days  stable

Just by looking at the new versions, we can see that there are 2 patch releases 1.0.1 and 1.0.2 that probably have bug fixes. Then there are two new feature releases that preserve backward compatability 1.1.0 and 1.2.0 - in fact I look at the versions not as absolute, but as difference from the current one

  • changed.same.same = breaking API change, danger!
  • same.changed.same = new feature, yum!
  • same.same.changed = bug fix, thank you, kind developer!

I can easily control which version of "foo" I use - I list its version in the package.json file, I can use strict version (without any ^, ~ or * ranges), and I can test new versions at my leisure before switching (I recommend using next-update).

Service dependencies

The "static" installed dependencies above are simple. But we have another type of dependencies that are much much harder to control - the external 3rd party services we call at runtime. For example, if your application calls the GitHub API, then you are using a 3rd party service. My next-update tool calls next-update.herokuapp.com to display the update statistics for a package. If you split your application into a pool of micro services, each one is likely to call others many many times.

Versioning service dependencies is an open field; there are no standards and people have very strong opinions what method is the best. Main methods are: including API version in the request URL, or setting a special header or even including the desired version in the "Accept" header. All these schemes in my opinion make the problem much harder by being the opposite of the NPM's solution and be giving the service too much work to do. Let us take a look at how we can solve the service versioning in the similar fashion to NPM.

  1. The dependency package "foo" should NOT implement multiple API versions simultaneously. Instead, it publishes a new version to the registry. It is up to NPM installer to pick the correct version depending on what is specified in the package.json. Similarly, it is a burden on the service author to support multiple versions simultaneously. Instead a deploy process should allow running multiple service versions in parallel.
  2. Usually our module gets the service URL injected somehow instead of hardcoding it. Maybe it is a config file or an environment variable - all we have is an URL to call. How do we know if this is the right service? We cannot know, instead the service can return its own version with the response. Then the client can check the received version against the expected service version or range.

The second point is important - we do not have the control over the deployment in a distributed heterogeneous environment. The most we can do is to make a request, hope it gets to the right service, and check the response - is the executor service the right one? How do we know it is a right one for us? By testing of course! Just like we did in unit testing, once we have tested and confirmed that [email protected] is acceptable, we set its version in the package.json. Same with the external services. Once we have tested GitHub API and confirmed that v3 works for us, we should validate every GitHub response to make sure that we are getting v3 responses. If we all of the sudden get v2 or v4 response - something is wrong!

A service can embed name and version in the response headers, here is my koa-version-header for example. Any request gets the information necessary to validate if the service is acceptable.

1
2
3
4
5
6
7
8
const koa = require('koa')
const serviceVersion = require('koa-version-header')
const app = koa()
app.use(serviceVersion())
app.use(function * () {
  this.body = 'ok'
})
app.listen(3000)
1
2
3
4
5
6
7
8
9
$ http localhost:3000
HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 2
Content-Type: text/plain; charset=utf-8
Date: Wed, 23 Mar 2016 19:37:26 GMT
X-Service-Name: koa-version-header
X-Service-Version: 0.0.0-semantic-release
ok

From an application you can validate the version using semantic versioning package

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
// list run time service dependencies
// https://github.com/npm/node-semver#advanced-range-syntax
const serviceDependencies = {
  'test-service': '~2.3.0'
}
axios.get(url)
  .then((response) => {
    const serviceName = response.headers['x-service-name']
    const serviceVersion = response.headers['x-service-version']
    console.log(`got response from ${serviceName}@${serviceVersion}`)
    const expectedVersion = serviceDependencies[serviceName]
    if (typeof expectedVersion === 'string') {
      console.log(`is this service version acceptable? We need ${expectedVersion}`)
      // https://github.com/npm/node-semver#ranges-1
      const satisfies = semver.satisfies(serviceVersion, expectedVersion)
      console.log(`satisfies? ${satisfies}`)
    }
  })

The above example is from this demo

To simplify checking I implemented the version validation for requests that use the axios library. Other libraries allow other interceptors, and if the excellent nock implements andCallThrough feature, then you could validate the service requests from any library.

If the service does NOT satisfy our requirement - well, a runtime exception should probably be sent to a crash monitoring service. Point is that something went wrong - maybe the deploy failed and an old external service is still responding to the calls, or maybe the external service has been upgraded but not the client application.

Conclusion

During npm install you have the power to tell NPM which version of a dependency is acceptable. If a specific version is unavailable, the install fails. Similarly, during execution, if a semantically acceptable version of an external service is unavailable, you should be notified and fail. By embedding the version information in the response as opposed to request you decouple the client from the service itself; the service should not have the burden of implementing multiple versions.

If testing and upgrading NPM module versions is like switching parts when assembling a car, validating versions in a deployed system is like replacing a wheel on a moving car - hard, but might be the only way to keep all the parts the same except for a specific one.

Resources