Jan 30 2016

Microservices with fuge

Building microservices with Fuge

This is one of 3 blog posts

Lately, I wanted to get better at DevOps, and especially with container-based NodeJS development. When I heard about the new tool fuge to help quickly develop microservices using containers, I have decided to give it a try. If running Docker is still too much trouble, take a look at my tool quickly.

Addition service

My first goal is to write a small service to compute sum of two numbers.

npm install -g fuge
md add-service; cd add-service
$ fuge help
usage: fuge <command> <options>
available commands
generate [system | service] - generate a system or an additional system service
build - build a system by executing the RUN commands in each services Dockerfile
pull - update a system by attempting a git pull against each service
run <compose file> - run a system
preview <compose file> - preview a run command for a system
shell <compose file> - start an interactive shell for a system
help - show this help

Fuge can build and run both services and websites, and I will start with a service.

$ fuge generate service
? Service name (servicefed7pg): addition
 create package.json
 create service.js
 create README.md
 create test/serviceTest.js
 create Dockerfile
 create .jshintrc
 create .istanbul.yml
 create .gitignore

I had to wait while the NPM install finishes (make sure to set npm set progress=false before running NPM install due to the progress bar bug).

The fuge command created a new folder with Dockerfile, README, package, etc. - all the scaffolding necessary to start the service. The main file has the stub code for the service itself, based on Seneca micro services toolkit.

service.js

'use strict';
var seneca = require('seneca')();
seneca.use('env-plugins');
seneca.add({role: 'addition', cmd: 'action1'}, function(args, callback) {
  callback(null, {data: 'data'});
});
seneca.add({role: 'addition', cmd: 'action2'}, function(args, callback) {
  callback(null, {data: 'data'});
});
seneca.listen({host: process.env.SERVICE_HOST, port: process.env.SERVICE_PORT});
module.exports.seneca = seneca;

I will replace the stub code with simple add and sub operations.

service.js

seneca.add({role: 'addition', cmd: 'add'}, function(args, callback) {
  callback(null, {data: args.a + args.b});
});
seneca.add({role: 'addition', cmd: 'sub'}, function(args, callback) {
  callback(null, {data: args.a - args.b});
});

Unit testing the service

The package.json created by fuge already has a few utility commands, and even some simple unit tests. Unfortunately, it relies on global jshint command, so I had to install it locally to be able to run the lint and test steps.

$ npm install --save-dev jshint
$ npm run lint

Then I updated the unit tests in test/serviceTest.js

process.env.SERVICE_HOST = 'localhost';
process.env.SERVICE_PORT = 3001;
var test = require('tape');
var seneca = require('../service').seneca;
test('add test', function(t) {
  t.plan(2);
  seneca.act({role: 'addition', cmd: 'add', a: 2, b: 3}, function(err, result) {
    t.equal(err, null);
    t.equal(5, result.data);
  });
});

When running the unit tests one can see that it actually creates a service bound to the socket, etc - exercising the full micro service round trip.

$ npm test
> [email protected] test /microservices/add-service/addition
> jshint **/*.js && tape test/*Test.js
2016-01-30T20:48:22.934Z 20r2obxkxj6j/- INFO    hello   Seneca/
2016-01-30T20:48:23.223Z 20r2obxkxj6j/- INFO    listen  {host:localhost,port:3001}  
TAP version 13
# add test
ok 1 should be equal
ok 2 should be equal

The unit test also shows how a micro service client calls it

1
2
3

seneca.act({role: 'addition', cmd: 'add', a: 2, b: 3}, function(err, result) {
    // result.data should be a + b
});

This seems very verbose, so in the future I plan to apply partial application to the seneca.act method, and to convert the Node-style callback to promise-returning method. For example, using object binding with obind I could create a function to just execute add method with just the operands.

var obind = require('obind');
var add = obind(seneca.act.bind(seneca),
  { role: 'addition', cmd: 'add' });
test('add test', function(t) {
  t.plan(2);
  add({a: 2, b: 3}, function(err, result) {
    t.equal(err, null);
    t.equal(5, result.data);
  });
});

Then we can convert add into Promise-returning function

var obind = require('obind');
var blue = require('bluebird');
var add = obind(blue.promisify(seneca.act, { context: seneca }),
  { role: 'addition', cmd: 'add' });
test('add test', function(t) {
  t.plan(1);
  add({a: 2, b: 3})
    .then(function (result) {
      t.equal(5, result.data);
    })
    .done();
});

The transformation makes using the services a lot more conveniently in my opinion. See related blog post Promisify Seneca microservice

Running the service

Let us first run the service directly as a Seneca micro service. I added a command to the package.json file for simplicity

"run": "SERVICE_HOST=localhost SERVICE_PORT=3001 node service.js"

Then execute npm run run and from another terminal window I can send a command

$ curl -d '{"role":"addition","cmd":"add","a":2,"b":3}' http://localhost:3001/act
{"data":5}

I prefer using httpie which makes JSON POST requests simpler

$ http http://localhost:3001/act role=addition cmd=add a=2 b=3
{
    "data": "23"
}

Hmm, we are getting back the concatenated string, not added numbers. Let us ensure that our service adds numbers.

service.js

seneca.add({role: 'addition', cmd: 'add'}, function(args, callback) {
  callback(null, {
    data: Number(args.a) + Number(args.b)
  });
});

Now we are getting the result as a number

$ http http://localhost:3001/act role=addition cmd=add a=2 b=3
{
    "data": 5
}

Running inside Docker container

In order to run the service inside a Docker container, fuge needs to create a system. Let us create a system - it will include a server, and two services, just like our addition service above.

fuge generate system

We can start the service right now and see the dummy actions execute

fuge run ./fuge/compose-dev.yml
open localhost:10000

We can execute the methods by going through the website, but I really want to try our addition service. We need to edit the fuge/compose-dev.yml file and replace the listed services with

compose-dev.yml

frontend:
  build: ../site/
  container_name: site
addition:
  build: ../addition/
  container_name: addition

Then we need to edit site/api/services.js and call addition service. The best part about fuge is that we can start it and then it keeps a watch on our files and restarts the services on file changes. Thus we can for example change the micro services the server will call to just addition and the server will restart. Edit site/api/services.js and remove 2 default services in favor of single addition. Add endpoint (we are working with a Hapi api here)

seneca.client({
    host: process.env.PROXY_HOST, 
    port: process.env.addition_PORT, 
    pin: {role: 'addition'}
});
module.exports = function(server) {
  server.route({
    method: 'GET',
    path: '/addition/add/{a}/{b}',
    handler: function(request, reply) {
      var options = {
        role: 'addition',
        cmd: 'add',
        a: request.params.a,
        b: request.params.b
      };
      seneca.act(options, function(err, res) {
        reply({result: err ? 'error' : res, err: err});
      });
    }
  });
  ...
};

Notice that most parameters are passed from the environment variables already, and thus the server to micro service bridge is easy to run from a container. After modifying the HTML of the page, we can call the right route, which calls the right micro service addition and prints the result JSON back in the page.

$ fuge run ./fuge/compose-dev.yml
compiling...
starting proxy...
  proxy frontend 10000 -> 192.168.0.8:20000
  proxy addition 10001 -> 192.168.0.8:20001
running: frontend
running: addition
running: __proxy
[frontend - 47688]: 160130/224457.928, [response], http://0.0.0.0:20000: get /addition/add/10/4 {} 200 (33ms) 
[frontend - 47688]: 160130/224500.049, [response], http://0.0.0.0:20000: get /addition/sub {} 200 (16ms)

fuge addition

Another nice feature of fuge is the shell. It is specifically for starting / stopping / controlling the individual micro services. For example, let us start just the addition

$ fuge shell fuge/compose-dev.yml 
compiling...
starting proxy...
  proxy frontend 10000 -> 192.168.0.8:20000
  proxy addition 10001 -> 192.168.0.8:20001
starting shell..
? fuge> ps
name           type     status   watch  tail  count
frontend       process  stopped  yes    yes   0    
addition       process  stopped  yes    yes   0
? fuge> start addition
running: addition
[addition - 48339]: ... INFO    listen  {host:0.0.0.0,port:20001}

We can try the addition micro service from another shell

$ http http://localhost:20001/act role=addition cmd=add a=2 b=3
{
    "data": 5
}

Nice! We have a UI front end for simple testing, and independent micro services, all controlled from a single tool, and each part (the front end server, addition micro service) has its own Docker file. For example, here is the addition Dockerfile

FROM node
ADD ./package.json /
RUN npm install
ADD . /
CMD node service.js

Deployment to "production"

We can easily develop our system, without running multiple terminals, and with quick service reload (via fuge shell). Now let us switch to using Docker (via docker-compose) and run the same multi-container setup.

I will show how to run the system in a hosted environment in the next blog post.

Better world by better software

Gleb Bahmutov PhD

Our planet 🌏 is in danger

Act today: what you can do

Microservices with fuge

Building microservices with Fuge

Addition service

Unit testing the service

Running the service

Running inside Docker container

Deployment to "production"