Extra FP for RxJS

I have read an excellent gentle introduction to reactive programming using RxJS library. The blog post is Getting Started With Rx.js: A Gentle Introduction by Jaime González García and the example is posted on jsFiddle for playing around with. It is embedded below; click on "Result" tab and click the "load more" button. It should load a new random twitter account on each click.

I liked this article and the example, it clearly shows the event processing steps; from clicking on the button to showing the result (even if I would prefer using something like virtual-dom to update the page efficiently).

Let us look at the example's code and see if we can "improve" it a little bit.

let randomPlayer$ = Rx.Observable
  .fromEvent(btnLoadMore, 'click')
  .map( _ => 'https://api.github.com/users')
  .startWith('https://api.github.com/users')
  .flatMap(url => Rx.Observable.fromPromise(fetch(url)))
  .flatMap(r => Rx.Observable.fromPromise(r.json()))
  .map(pickRandomItem)

The refactorings I am going to do should serve as learning exercises, and might not be needed in this particular case. Yet they fight the whatever little boilerplate code exists even in this short program. I hate boilerplate and always look for ways to shorten and clarify every piece of software I write.

Functions instead of data

First, let us eliminate the duplicate GitHub API urls. We map every button click to the user list url string, and we also start the stream with the same url. Let us move the url into its own variable.

const url = 'https://api.github.com/users'
let randomPlayer$ = Rx.Observable
  .fromEvent(btnLoadMore, 'click')
  .map( _ => url)
  .startWith(url)
  .flatMap(url => Rx.Observable.fromPromise(fetch(url)))
  .flatMap(r => Rx.Observable.fromPromise(r.json()))
  .map(pickRandomItem)

Look at the .map(_ => url) step - it ignores the input arguments and just returns the same value every time. Let us create a utility function that just returns the given value when called.

const always = x => _ => x
const url = 'https://api.github.com/users'
let randomPlayer$ = Rx.Observable
  .fromEvent(btnLoadMore, 'click')
  .map(always(url))
  .startWith(url)
  .flatMap(url => Rx.Observable.fromPromise(fetch(url)))
  .flatMap(r => Rx.Observable.fromPromise(r.json()))
  .map(pickRandomItem)

The function always takes an argument value and returns another function. The returned function does nothing but returns the original value when called. If you do not feel comfortable with expressing the meaning of always as a one liner (I often feel lost in one liners), write it down explicitly.

const always = x => _ => x
// same as
function always(x) {
  return function () {
    return x
  }
}

Using helper always we can replace hardcoded url with clear code - we always map a click to the url. We can even refactor always use and "hide" the url inside.

const always = x => _ => x
const toUrl = always('https://api.github.com/users')
let randomPlayer$ = Rx.Observable
  .fromEvent(btnLoadMore, 'click')
  .map(toUrl)
  .startWith(toUrl())
  .flatMap(url => Rx.Observable.fromPromise(fetch(url)))
  .flatMap(r => Rx.Observable.fromPromise(r.json()))
  .map(pickRandomItem)

.map(...) requires a function, thus we give it toUrl, while .startWith(...) needs an actual value, thus we give it toUrl().

You can find the updated example at 52mwv7gn/1/ and below

I often return a function to access the data rather than the data in my code. This helps hide the implementation details as well as protects the data from mutations. For example instead of returning a user object and letting the outside code read its properties (and thus being able to modify the user object) I can return a function that allows reading a particular property but not setting it.

// using objects directly
function getUser() {
  var user = ... // somehow get user
  return user
}
// client code
var user = getUser()
console.log('user email', user.email)
user.email = undefined // !

// returning a function to limit access to the object
const prop = o => name => o[name]
// same as
function prop(o) {
  return function (name) {
    return o[name]
  }
}
function getUser() {
  var user = ... // somehow get user
  return prop(user)
}
// client code
var user = getUser()
console.log('user email', user('email'))
// user.email = undefined no longer works!
// if internal user object's structure changes, no problem
// we can just return a different function and the
// client code will still work

Shorten promise boilerplate

Let us shorten the next two steps - making an Ajax request for data and extracting the JSON from the response.

.flatMap(url => Rx.Observable.fromPromise(fetch(url)))
.flatMap(r => Rx.Observable.fromPromise(r.json()))

Notice that the argument name to both .flatMap calls does not matter, it is only used for clarity. Thus in reality this code has a very strong boilerplate smell. If we replace the argument with letter "x" then 45 characters out of 50 in the two lines are the same!

.flatMap(x => Rx.Observable.fromPromise(fetch(x)))
.flatMap(x => Rx.Observable.fromPromise(x.json()))

Let us look at the first call's callback. This is the (almost) the simplest function composition example one can find:

x => Rx.Observable.fromPromise(fetch(x))
     ------------------------- -----
          f                      g  (x)

Every time we see function f applied to the result of calling another function g to argument x we can replace f(g(...)) with equivalent composed function

// assume we have "compose" helper function
var composed = compose(f, g)
composed(x) // is same as (f(g(x)))

Writing the "compose" helper fuction in this case is very simple - we are ignoring context and only assume a single argument

function compose(f, g) {
  return function (x) {
    return f(g(x))
  }
}
// same as
const compose = (f, g) => x => f(g(x))

Now let us replace first .flatMap with shorter equivalent code. We can even just provide the callback without explicit url parameter, achieving the pointfree programming style

const ajax = compose(Rx.Observable.fromPromise, fetch)
...
  .startWith(toUrl())
  .flatMap(ajax)
  .flatMap(x => Rx.Observable.fromPromise(x.json()))

You can try the code at jsfiddle.net/h75k3y8z/1/ or below

Let us simplify the second .flatMap callback. Instead of simple composition f(g(x)) it has a method call on the argument f(x.g()). We need to compose f with invoking a method on a object. We first know the name of the method json but no the object yet - the object will be passed to the callback. Let us write simple helper function to invoke a method given just its name on an object passed later.

function method(name) {
  return function(object) {
    return object[name]()
  }
}
// same as
const method = name => object => object[name]()
// use like this
var user = {
  email: function () { return '[email protected]' }
}
user.email()   // '[email protected]'
var getEmail = method('email')
getEmail(user) // '[email protected]'

method('email') just creates a function that is waiting for an actual object. In our RxJS example we need a function that will invoke json() on the given response object.

.flatMap(x => Rx.Observable.fromPromise(x.json()))
// same as
.flatMap(x => Rx.Observable.fromPromise(method('json')(x)))
// same as
const json = method('json')
.flatMap(x => Rx.Observable.fromPromise(json(x)))

Now we have simple function composition again with first function being Rx.Observable.fromPromise and the second function being json function. Applying compose again makes the callback go away.

const ajax = compose(Rx.Observable.fromPromise, fetch)
const json = compose(Rx.Observable.fromPromise, method('json'))
...
  .startWith(toUrl())
  .flatMap(ajax)
  .flatMap(json)

The example still works like before, we just replaced the code with equivalent shorter code

Using external library

We have only introduced 3 helper functions above

const always = x => _ => x
const compose = (f, g) => x => f(g(x))
const method = name => object => object[name]()

The functions are so short it is hard to make mistakes in them ;) Yet if you start using little helper functions like these you immediately will want more of them. You will want more helpers for different situations (like grabbing the property of an object instead of calling a method), and more power in each helper (like composing more than two functions or passing arguments to the method call). Writing these helpers and testing them inside each function quickly becomes a chore. Luckily there are entire libraries of these functions, lodash and Ramda being my favorite.

For the example above, Ramda has everything we need right out of the "box". Let us add the library to the fiddle (using CDN) and replace our custom code with Ramda's functions. The syntax almost maps one to one, except for method helper - Ramda needs to know the number of arguments expected by the method, thus we pass 0 first.

const toUrl = R.always('https://api.github.com/users')
const ajax = R.compose(Rx.Observable.fromPromise, fetch)
const json = R.compose(Rx.Observable.fromPromise, R.invoker(0, 'json'))

We run the new code ... and it does not work. The browser console shows an error

Uncaught TypeError: Failed to execute 'fetch' on 'Window': parameter 2 ('init') is not an object.

What is this? Well, R.compose is more powerful than our custom code before; it passes all arguments along. For example, if we create a little wrapper function to see what arguments are passed to fetch (which is the new built-in HTML5 API function) we can see those.

const ajax = R.compose(Rx.Observable.fromPromise, function (url) {
 console.log(arguments)
 return fetch(url)
})

Run the code again and see in the browser console the following

["https://api.github.com/users", 1, FlatMapObservable]

Just like Array.map, Observable.flatMap passes to each callback the item, the index and the "list" (in this case Observable instance) itself. When using arrays, this leads to some madness (and an excellent interview question).

In our case, fetch gets extra arguments and breaks. But does not fetch only expect a single argument? After all, if we print fetch.length we get 1! Not really, the fetch API allows additional options object as the second argument

fetch(url, {
  credentials: 'include'
})

Thus fetch is not ignoring the second argument and breaks if the argument is an iteration index for example. We should really tell fetch to ignore everything but the first argument - we should make fetch unary. We can write a little helper ourselves easily or use Ramda's function

function unary(f) {
  return function (x) {
    return f(x)
  }
}
// our unary is same as R.unary
const ajax = R.compose(Rx.Observable.fromPromise, R.unary(fetch))

Now the code works as expected!

We often have to do this function signature "massaging" when adapting the expected arguments to what is really passed into the function.

Compose promises instead of flap mapping

We have two chained steps to .flatMap in our stream, and they deal with a single request: making an ajax request (returns a promise) and then extracting JSON result (also returns a promise in the new HTML5 API). We can combine the promises using .then() and use a .flatMap instead.

const ajaxP = R.unary(fetch)
const jsonP = R.invoker(0, 'json')
function users(url) {
  return Rx.Observable.fromPromise(
    ajaxP(url).then(jsonP)
  )
}
...
  .startWith(toUrl())
  .flatMap(users)
  .map(pickRandomItem)

Even better, Rx.Observable.flatMap works directly with a promise object, without mapping into Observable first

const ajaxP = R.unary(fetch)
const jsonP = R.invoker(0, 'json')
function users(url) {
  return ajaxP(url).then(jsonP)
}
...
  .startWith(toUrl())
  .flatMap(users)
  .map(pickRandomItem)

Finally, we have a promise-returning function ajaxP chained to jsonP. The result of ajaxP is passed to jsonP and then the result is returned (inside the resolved promise). This is very similar to the function composition, only the passing the result along is different. Instad of simplifying f(g(x)) into compose(f, g)(x) promise composition combines the promise-returning functions using .then method like this f(x).then(g) = composeP(f, g)(x)

Ramda has promise composition under R.composeP

const ajaxP = R.unary(fetch)
const jsonP = R.invoker(0, 'json')
const users = R.composeP(jsonP, ajaxP)
...
  .startWith(toUrl())
  .flatMap(users)
  .map(pickRandomItem)

The promise composition works from right to left (first ajax, then json). Our original code was easier to read left to right since it was ajaxP(url).then(jsonP). There is an equivalent helper method R.pipeP which is just like compose but expects its arguments in left to right order. I find it usually easier to read "piped" code than "composed" code. Both produce the same result

Conclusion

Functional reactive programming is all about using little helpers to get rid of unnecessary code in your reactive streams. Writing your own helper functions is a great exercise and a perfect first step to shorter and clearer code. Then switch to using a dedicated library and get rid of most of the code in your application.