Playing with PouchDB

I am playing with a cool Node / browser NoSQL database called PouchDB. It works as a uniform interface across all modern desktop and mobile browsers by utilizing IndexedDB / WebSQL technologies, and works under Node too. It can also proxy requests to a CouchDB database, the database behind NPM registry.

Simple example

Here is how simple it is to get started with PouchDB under Node.

npm install pouchdb --save

A simple source to insert an item and then print all items

simple.js

var PouchDB = require('pouchdb');
var db = new PouchDB('my_database');
var todo = {
  _id: new Date().toISOString(),
  title: 'an example todo',
  completed: false
};
db.put(todo, function (err, result) {
    console.log('Successfully saved a todo!', result.id);
    db.allDocs({ include_docs: true, descending: true }, function (err, result) {
      console.log('fetched', result.rows.length, 'items');
      console.log(result.rows);
    });
  });

The output (we are using timestamps as ids)

Successfully saved a todo! 2015-03-23T19:44:45.106Z
fetched 1 items
[ { id: '2015-03-23T19:44:45.106Z',
    key: '2015-03-23T19:44:45.106Z',
    value: { rev: '1-cf1d4cfef10eca49836f072566e29fbe' },
    doc: 
     { title: 'an example todo',
       completed: false,
       _id: '2015-03-23T19:44:45.106Z',
       _rev: '1-cf1d4cfef10eca49836f072566e29fbe' } } ]

Notice that this created a folder locally called my_database with all data stored (with revisions) in binary format.

If we run this problem several times we will insert a new item into the store every time. Thus on the second run you should inspect seeing a list of 2 items, then 3 items, etc.

Destroying a db in the browser

If you want to remove an existing pouch database stored on the client, execute the destroy call

(new PouchDB('db-name')).destroy().then(console.log.bind(console), console.error.bind(console));
// prints {ok: true}

Simple example using memory only

For unit testing we might want to avoid writing anything to the disk. The PouchDB uses adapters to perform low-level database operations. There is an adapter for keeping data in memory, without leaving any traces on the disk. The memory adapter is perfect for unit testing. First install memdown, then perform same operations npm install memdown --save

memory-only

var PouchDB = require('pouchdb');
var db = new PouchDB('my_database', { db: require('memdown') });
// the rest of the code is unchanged

Running this example always starts from empty (new) database, guaranteeing the unit tests will execute in a clean environment.

Promise-based API

The included PouchDB api is node-like callback based. Every function accepts a callback with the error (if any) at the first position and data at the second. I prefer using promises to callbacks, thus I convert each method to a promise-returning one using Q.nbind method

promises

var Q = require('q');
var PouchDB = require('pouchdb');
var db = new PouchDB('my_database');
var dbPut = Q.nbind(db.put, db);
var dbAllDocs = Q.nbind(db.allDocs, db);
function addTodo(text) {
  var todo = {
    _id: new Date().toISOString(),
    title: text,
    completed: false
  };
  return dbPut(todo).then(function (result) {
    console.log('Successfully posted a todo!', result.id);
  });
}
function printTodos() {
  return db.allDocs({ include_docs: true, descending: true })
    .then(function (doc) {
      console.log(doc.rows);
    });
}
addTodo('first todo')
  .then(printTodos)
  .done();

If we need to insert several todos, I prefer to link them to existing promise chain and using partial application instead of extra functions

Q()
  .then(addTodo.bind(null, 'first todo'))
  .then(addTodo.bind(null, 'second todo'))
  .then(printTodos)
  .done();

Promisfying all methods

Q api has a function to promisify a single method, while another library of promises bluebird has a method to promisify all object's methods at once. For each method foo.bar(..., callback(err, ...)) it will create a promise-returning foo.barAsync() method.

promisfy all db methods

var Promise = require('bluebird');
var PouchDB = require('pouchdb');
var db = new PouchDB('my_database', { db: require('memdown') });
// adds new ...Async methods that return promises
Promise.promisifyAll(db);

function addTodo(text) {
  var todo = {
    _id: new Date().toISOString(),
    title: text,
    completed: false
  };
  return db.putAsync(todo).then(function (result) {
    console.log('Successfully posted a todo!', result.id);
  });
}
function printTodos() {
  return db.allDocsAsync({ include_docs: true, descending: true })
    .then(function (doc) {
      console.log(doc.rows);
    });
}
Promise.resolve() // same as Q()
  .then(addTodo.bind(null, 'first todo'))
  .then(addTodo.bind(null, 'second todo'))
  .then(printTodos)
  .done();

Removing a little bit of boilerplate

We can avoid all partial application calls like addTodo.bind(null, 'first todo') using a curried function, like I described in Remove boilerplate from promise chains.

// same DB code
var R = require('ramda');
var addTodo = R.curryN(2, function addTodo(text) {
  var todo = {
    _id: new Date().toISOString(),
    title: text,
    completed: false
  };
  return db.putAsync(todo).then(function (result) {
    console.log('Successfully posted a todo!', result.id);
  });
});
// same printTodos
Promise.resolve()
  .then(addTodo('first todo'))
  .then(addTodo('second todo'))
  .then(printTodos)
  .done();

The curried version of addTodo is a pleasure to work with in the middle of the promise chain.

Removing a lot of boilerplate using generators

Promises are nice, but we can do better using generators. Instead of creating and chaining promises, I am going to yield individual asynchronous functions into a runner function co. First, I am converting a function that takes a callback into a thunk using thunkify. A thunk just shifts the last callback argument into new function

thunks

// original db.put
db.put(item, function (err, result) { ... });
// thunk method bound to the original object
var thunkify = require('thunkify');
var dbPut = thunkify(db.put).bind(db);
dbPut(item)(function (err, result) { ... });

We just need to return thunks from db.put and db.allDocs methods.

generators

var PouchDB = require('pouchdb');
var db = new PouchDB('my_database', { db: require('memdown') });
var thunkify = require('thunkify');
var dbPut = thunkify(db.put).bind(db);
var dbAllDocs = thunkify(db.allDocs).bind(db);
var co = require('co');
function addTodo(text) {
  var todo = {
    _id: new Date().toISOString(),
    title: text,
    completed: false
  };
  return dbPut(todo);
}
function printTodos() {
  return dbAllDocs({ include_docs: true, descending: true });
}
co(function *() {
  yield addTodo('first todo');
  yield addTodo('second todo');
  var result = yield printTodos();
  return result.rows;
}).then(function (result) {
  console.log(result);
}, function (err) {
  console.error(err.stack);
});

We need to run this code using node --harmony index.js flag (or use io.js). It prints two JSON documents

[ { title: 'second todo',
    completed: false,
    _id: '2015-03-24T03:52:11.439Z',
    _rev: '1-92c5d74cc0e408c35d332572c5fa1337' },
  { title: 'first todo',
    completed: false,
    _id: '2015-03-24T03:52:11.428Z',
    _rev: '1-f90e06ba4ee3b752fe476e397d8ef993' } ]

We are going to insert two items, then grab results - and the sequence is written linearly, while the execution is asynchronous. The yielded and continued values are controlled by the library function co

co(function *() {
  yield addTodo('first todo');
  yield addTodo('second todo');
  var result = yield printTodos();
  return result.rows;
});

The co function returns a promise that is resolved with the last returned value.

Cleaner syntax by yielding promises

We can apply the same good code refactoring principles to the generators as to our usual code. Plus instead of creating the thunks we can directly yield promises. To run the generator function we can you another method from Bluebird API Promise.coroutine

promises and coroutine

var PouchDB = require('pouchdb');
var db = new PouchDB('my_database', { db: require('memdown') });
var Promise = require('bluebird');
Promise.promisifyAll(db);
function addTodo(text) {
  var todo = {
    _id: new Date().toISOString(),
    title: text,
    completed: false
  };
  return db.putAsync(todo);
}
function getTodos() {
  return db.allDocsAsync({ include_docs: true, descending: true });
}
function * add2TodosAndReturnAll() {
  yield addTodo('first todo');
  yield addTodo('second todo');
  var result = yield getTodos();
  return result.rows;
}
var start = Promise.coroutine(add2TodosAndReturnAll);
start()
  .then(function (result) {
    console.log('result', result);
  }, function (err) {
    console.error(err.stack);
  });

I like this approach for only using a single external promise library "bluebird".

Removing boilerplate using ES7 await / async

We can go one step further and use a new pair of keywords that will probably be a part of the EcmaScript7 standard. While await/async feature is in flux, one can already use it for experiments. Instead of a yield I am using the await keyword, and instead of the generator function, I have marked a function with the keyword async. Each individual DB operation is returning a promise created via promisifyAll method.

async-await.js

var PouchDB = require('pouchdb');
var db = new PouchDB('my_database', { db: require('memdown') });
var Promise = require('bluebird');
Promise.promisifyAll(db);
function addTodo(text) {
  var todo = {
    _id: new Date().toISOString(),
    title: text,
    completed: false
  };
  return db.putAsync(todo);
}
function printTodos() {
  return db.allDocsAsync({ include_docs: true, descending: true })
}

use await/async to remove boilerplate code

// most important function
export default async function () {
  let first = await addTodo('first todo');
  console.log('Successfully posted a todo!', first.id);
  let second = await addTodo('second todo');
  console.log('Successfully posted a todo!', second.id);
  let result = await printTodos();
  console.log(result.rows);
  return result.rows;
}

To run this example under Node 0.12 I created a second file and installed es6-module-loader as a dependency

async-await-app.js

var System = require('es6-module-loader').System;
System.traceurOptions = {
  asyncFunctions: true
};
System.import('./async-await').then(function (loaded) {
  loaded.default();
});

Then I ran it using regular Node command node async-await-app.js. The ES7 code from async-await.js was automatically transpiled using traceur on module load.