products

Run Express server in your browser

Plus how to run a web application when the JavaScript itself is disabled*.

The problem - web application loading times

Don't you hate the waiting period when a web application loads? I have been chasing faster boot up times every since Angular plus React equals Speed revisited friendly competition dialogue with Dave Smith which started at ng-conf. I believe that once the application has started, both Angular and React are pretty similar in performance; but React is definitely faster to boot than Angular 1.x, and it remains so despite my best efforts.

Yet, even React has a noticeable delay between the page load and the application's first full rendering. Take a look at the following video showing the simplest possible web application implemented using React - see the initial markup shift when the application renders itself?

You can pick your favorite framework or library and see how fast does it start at TodoMVC.com - some are faster than others, but none shows the fully rendered page on load - which is fine, these are web application after all.

For any application more complicated than a single list, the boot times are longer. My favorite Todo application - the PivotalTracker takes a few seconds to load the same unchanged list of tasks every time I open the browser tab! Why can't it store the items in the browser's localStorage and recreate even the stale page right away? It is unlikely I as a user will want to interact with the application in the first 500ms, I probably just want to read the content; so why do I need the fully interactive application right away?

Instant application by intercepting HTML in ServiceWorker

I have visited this question in Instant web application blog post. The idea, implemented in my bottle-service library is to store both the data and the last rendered HTML every time the web application renders something. Then on startup we can show the last rendered HTML right away and then update it if the data changes. The extra trick which completes the "instant feel" is storing the rendered HTML inside a ServiceWorker, which allows to intercept and update the page HTML before it gets to the browser. Here is the result - the application feels like it appears with NO delay, even as I intentionally slow down the JavaScript code load by 500ms.

You can try the demo application yourself using Chrome or Opera browsers at instant-todo.herokuapp.com.

The ServiceWorker code that intercepts and modifies the HTML before the browser sees it is in dist/bottle-service.js

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
// intercept HTML request and return updated HTML
self.addEventListener('fetch', function (event) {
  if (!isIndexPageRequest(event)) {
    return fetch(event.request)
  }
  event.respondWith(
    fetch(event.request)
      .then(function (response) {
        // get raw HTML text from response
        // update the web application area with previous HTML
        // in variable pageHTml
        var responseOptions = {
          status: 200,
          headers: {
            'Content-Type': 'text/html charset=UTF-8'
          }
        }
        return new Response(pageHtml, responseOptions)
      })
  )
})

The instant page content achieved!

Server-side rendering

I have never been a big fan or server-side rendering, because I know how painful Atlassian Jira feels - every click forces a full page reload, often for no reason! Yet from the search engine optimization and performance stand point, the server-side rendering has its advantages.

  • The full page arrives in its complete form to the user's browser. The rendering is instant
  • The server can render pages faster than the browser - the client libraries do not have to be loaded every time to render the page

In have tried to shoehorn client side frameworks to run on the server, for example Angular 1.x. Some frameworks make it really simple: virtual-dom is one of such frameworks that works equally well on the server and on the client.

Here is one possible implementation of the TodoMVC using server-side rendering. Every time I add an item, mark it done or delete it the entire page is reloaded! The page behaves very well, eventhough it is rendered somewhere on a remote server. In the end I reload the page several times - notice how fast the complete page appears - there is no blinking at all.

You can try the application yourself at todomvc-express.herokuapp.com; its source is at bahmutov/todomvc-express.

One interesting feature of this implementation is that the page itself disallows ALL JavaScript (using Content-Security-Policy meta tag). It works perfectly fine with form submissions and plain HTML5 (although submitting a form on checkbox click is a pain!)

For example, here is the code for a entering a new item. It is sent to the server using POST method when the user clicks "Enter" - notice the markup's simplicity

1
2
3
<form action="/" method="post">
  <input class="new-todo" placeholder="What needs to be done?" autofocus="" name="what">
</form>

There is a variety of backend implementations you can check out at www.todobackend.com - the server-side rendering has been around a log longer than the client-side JavaScript rendering!

Connecting the Express and ServiceWorker "dots"

As I was playing with an Express server rendering Todo application, I have noticed a similarity to the ServiceWorker code. Here is the Express code that handles HTTP requests src/start.js

1
2
3
4
5
6
7
8
var express = require('express')
var app = express()
var http = require('http')
var server = http.createServer(app)
server.listen(process.env.PORT || 3000)
var host = server.address().address
var port = server.address().port
console.log('TodoMVC server listening at http://%s:%s', host, port)

The Express "app" is a function that has only 2 arguments, traditionally called req and res

1
function app(req, res) { /* express js processing */ }

The req is an instance of Node ClientRequet and the res is an instance of Node http Response. The Express logic reads all input data from the request object and writes everything that needs to be sent to the client into the response object. Both can be used as streams.

The ServiceWorker "fetch" interceptor on the other hand expects an event with a request property that is an object of type [Request](https://developer.mozilla.org/en-US/docs/Web/API/Request. It resolves the fetch with an object of type Fetch API Response.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
self.addEventListener('fetch', function (event) {
  // read everything from event.request
  event.respondWith(new Promise(function (resolve) {
    // return page or data
    var options = {
      status: 200,
      headers: {
        'Content-Length': ...,
        'Content-Type': ...
      }
    }
    resolve(new Response(pageHTML, options))
  }))
})

Notice the similarities? The main logic needs the request data and should output data we can package up for specific case. Either output Node http Response or Fetch API Response.

This made me think - I have previously bundled up Angular 1.x to run inside a web worker

Can I bundle the entire Express application to run inside ServiceWorker, intercepting the requests and "server-side" rendering the responses?

Express inside ServiceWorker

Turns out it is very simple to do using browserify. I just need to patch the environment a little to provide the missing objects, like XMLHttpRequest and setImmediate. I also needed to make a mock Request object an Express can understand, and read data back from the Response it generates. You can find the entire code in the src/server.js in the repo bahmutov/express-service

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
var express = require('express')
var app = express()
var url = require('url')
self.addEventListener('fetch', function (event) {
  const parsedUrl = url.parse(event.request.url)
  var req = {
    url: parsedUrl.href,
    method: event.request.method,
    body: body,
    headers: {
      'content-type': event.request.headers.get('content-type')
    },
    unpipe: function () {}
  }
  var res = { /* our mock Node http Server Response object */ }
  event.respondWith(new Promise(function (resolve) {
    app(req, res)
  }))
})

In order to wait for Express application to finish and the response to be complete, we overwrite the method res.end

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
function endWithFinish (chunk, encoding) {
  const responseOptions = {
    status: res.statusCode || 200,
    headers: {
      'Content-Length': res.get('Content-Length'),
      'Content-Type': res.get('Content-Type')
    }
  }
  if (res.get('Location')) {
    responseOptions.headers.Location = res.get('Location')
  }
  if (res.get('X-Powered-By')) {
    responseOptions.headers['X-Powered-By'] = res.get('X-Powered-By')
  }
  // send response back from ServiceWorker
  resolve(new Response(chunk, responseOptions))
}
var res = { /* our mock Node http Server Response object */ }
res.end = endWithFinish

That is it, the Express is running inside the ServiceWorker, rendering pages just like a normal remove server would do. You can see the live demo (use Chrome or Opera browser) running at https://express-service.herokuapp.com/. The source for the demo is in bahmutov/todomvc-express-and-service-worker. To generate the service bundle, the code could not be simpler, see index.js

1
2
3
const expressService = require('express-service')
const app = require('todomvc-express')
expressService(app)

To bundle, I use command npm run build that runs browserify and outputs dist/index.js (pretty heavy file at almost 2 MB unminified!) using browserify index.js -o dist/express-service.js. The index page itself only registers the service worker and reloads, letting the Express running in the ServiceWorker to take over

dist/index.html
1
2
3
4
5
6
7
8
9
10
11
12
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <title>TodoMVC in ServiceWorker</title>
</head>
<body>
  <p>Registering the service worker</p>
  <script src="client.js"></script>
</body>
</html>

If you open the browser DevTools while running the application you can see the pages being served from the ServiceWorker

screenshot

Here is the application in action, notice it is pretty snappy!

The application still says "Server-side" because I am using the complete TodoMVC Express without ANY modifications.

How to run a web application when the JavaScript itself is disabled*?

I have a demo TodoMVC running inside the ServiceWorker, and here an interesting thing. After the original service worker registration script runs, you can use the application with JavaScript completely disabled! The rendered pages do not have any client-side code.

* - the very first application load time still requires JavaScript.

Update

There is a proposal to load service worker via <link ...> or page header, like this

<link rel="serviceworker" scope="/" href="/sw.js">

See Chrome status. This would be great!

Discussion

I believe that running Express as a web application is an interesting experiment. A typical Express app has advantages over a web application - performance, simple to unit test. We could even make a hybrid - for clients with browsers that support ServiceWorkers, we could load the server into the ServiceWorker, and for every one else we could use server-side render from a remote server.

I was thinking how to call the server running in the browser. Typically, it would be called "universal" JavaScript, but I think this is a little different. We had to bundle and patch the environment for the server code to run in the different environment. I propose we call it "boxable" code; meaning the code that can be placed in a box. This is also similar to "black box" and "white box" testing; meaning we had to patch the environment in the box (the "white" box). We also had to convert environment-specific inputs and outputs before passing them into the box - the "black" box approach.