process

Hydrate your apps

Remove the empty page flicker on web application startup.

I love declarative web application frameworks like Angular and Vue.js - my personal website glebbahmutov.com is a Vue app! Nothing is easier than starting a new application by writing HTML markup with a few additional directives. For example to show a list of items, one could simply write

1
2
3
4
5
6
7
8
<div id="app">
  <ul>
    <li v-for="todo in todos">
      <span v-text="todo.text"></span>
      <button v-on:click="removeTodo($index)">X</button>
    </li>
  </ul>
</div>

Beautiful, simple and powerful. But this suffers from several problems, all because the initial page markup is NOT what we want to show. We want to NOT show the above template, instead we want to show the generated HTML after the application has started, something like this

1
2
3
4
5
6
7
8
9
10
11
12
<div id="app">
  <ul>
    <li>
      <span>Clean room</span>
      <button>X</button>
    </li>
    <li>
      <span>Learn Italian</span>
      <button>X</button>
    </li>
  </ul>
</div>

The gap between the time the browser shows the initial page and the time the application starts and updates the page with the real content is very noticeable and annoying.

I dislike the following 3 problems

  • the initial template is visible before the web app takes over
  • an empty space is visible before the web app runs
  • any content after the web app suddenly moves down

Any network delay makes the problems more obvious, and of course the browser caching makes the problems less apparent, but does not completely or even reliably removes it. Even when the data and the framework code have been loaded, the web application might need a lot of time to actually compute and render the live DOM. I have done a lot of experiments, trying to figure out how to speed up the initial loading of Angular 1 applications, all without much success.

Remove the initial template (simple)

We can solve the flash of the template at the beginning quickly. Just hide the div where the application's template is located. Make it visible once the web application has started.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
<style>
.hidden {
  visibility: hidden;
  display: none;
}
</style>
<body>
</body>
  <div id="app" class="hidden">
    <ul>
      <li v-for="todo in todos">
        <span v-text="todo.text"></span>
        <button v-on:click="removeTodo($index)">X</button>
      </li>
    </ul>
  </div>
</body>

We should remove the "hidden" class when the application is ready. In Vue, it is simple

1
2
3
4
5
6
new Vue({
  el: '#app',
  ready: function () {
    document.getElementById('app').classList.remove('hidden')
  }
})

The sudden DOM update looks weird if there is any content after the div, as shown below.

Is there something we can render instead of blank space at the startup?

Hydrated web application

I was inspired by the FeatherApp - a very lightweight application where the first two features are

  • Initial render of clientside components to static HTML at build time. So the browser gets "pre-rendered" HTML.
  • JS "taking over" once loaded in the browser.

While this is a great feature: generate the partial HTML as output of the build step, it is difficult to make useful for two reasons

  1. Only some web frameworks (React, Cycle) can easily render HTML without a full browser.
  2. We probably don't have good or relevant data yet to render during the build time.

When you think about what would be nice to render immediately for a particular user, one answer comes to mind: render the last viewed page and continue work after the app loads. This solves the above two problems nicely

  1. Every framework supports it - because the HTML to be saved and restored is already inside the browser.
  2. The rendered content is tailored to a specific user - this is what the user saw at the last saved checkpoint.

Before going into the implementation specifics, see the results in the video below. For the demo purposes, the web framework library is only loaded after 2 second delay.

Even before the app JavaScript loads and runs, the page appears. It appears very very quickly, and the footer text is not shifting wildly around the page.

Implementation details

The main application has two parts: the index HTML page and the src/app.js, taken pretty much verbatim from vuejs.org/guide. The application code loads at the bottom of the body.

1
2
3
4
5
6
7
<div id="app" class="hidden">
    Enter todo: <input v-model="newTodo" v-on:keyup.enter="addTodo"
      placeholder="learn Italian" title="Enter text and press Enter">
    ...
</div>
<script src="https://cdn.jsdelivr.net/vue/1.0.11/vue.js"></script>
<script src="src/app.js"></script>

The app code is just a single component attached to #app

1
2
3
4
5
6
7
8
9
10
11
12
// src/app.js
new Vue({
  el: '#app',
  data: {
    newTodo: '',
    todos: []
  },
  methods: {
    addTodo: function () { ... }
    removeTodo: function () { ... }
  }
})

The hydration feature is implemented in a single file src/hydration.js. It is a factory function that creates a single object called bottle and adds it to the window object for simplicity. The application code can do the following with the bottle

  • bottle.open() - called on page load, loads saved HTML snapshot and places into the page. This is the "dry" HTLM shown to the user while the application is loading. The bottle is opened automatically when the script is loading.

  • bottle.drink() - called by the application when it has fully loaded and wants to replace the "dry" HTML with actual live application. In Vue this command can be called from the ready callback

1
2
3
4
ready: function () {
  bottle.drink()
  document.getElementById('app').classList.remove('hidden')
}
  • bottle.refill() - can be called multiple times by the app to save DOM snapshots to be used on next load. Think every time we add a todo, or remove one, if we reload the page we want to show the updated HTML. For example we want to save both the todos list and the rendered HTML after adding a new one
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
// app methods fragment
save: function save () {
  localStorage.setItem('todos', JSON.stringify(this.todos))
  setTimeout(function () {
    // we should save the HTML after it has been rendered
    bottle.refill()
  }, 0)
},
addTodo: function () {
  var text = this.newTodo.trim()
  if (text) {
    this.todos.push({ text: text })
    this.newTodo = ''
  }
  this.save()
}

Finally, there is a utility method bottle.recycle() that deletes whatever HTML snapshot is stored in the localStorage.

For clarity, the demo has built-in console log messages and user interface popups, plus a blue overlay shown while the application is "dry" and the user cannot interfact with the input field or buttons.

To enable hydration, I include the src/hydration.js right after the web application element, but before anything else. The configuration is passed to the hydration function via script tags.

1
2
3
4
5
6
7
8
<div id="app">
...
</div>
<script src="src/hydration.js"
  id="app"
  verbose="true"
  on="hydrate"
  verbose-ui="true"></script>

There is a little bit more complexity and extra calls just to enable conditional hydration based on the check box - this is a demo comparing the original app vs hydrated one, after all! I invite you to see the results for yourself at glebbahmutov.com/hydrate-vue-todo.

Shortcomings

The hydration has its own shortcomings.

  1. The user cannot interact meaningfully with the page while the "dry" HTML is shown. In my opinion this is not a huge deal, because in most cases the first couple of seconds I am reading the website, not actively using it. Showing the loading message / overlay gives the user an idea that the app is read-only.
  2. The stored HTML cannot be too massive due to localStorage limitation (about 5MB limit).
  3. The "dry" HTML is extra work for the browser. Not a lot of extra work, because it is rendered only once, but it is saved multiple times - every time the web app thinks it makes sense to have a saved checkpoint.

You might think that direct pasting of HTML from the localStorage into the DOM is a security risk. I think the risk is minimal. First, this HTML comes directly from the user's own DOM. Second, the other websites do not have access to the localStorage snapshot - only the domain that wrote it there has access. Third, the HTML snapshots are compatible with the Content-Security-Policy - your website can have the inline JavaScript disabled and the hydration would still work; my personal website glebbahmutov.com has very strict policy and works just fine.

Where to go from here

  • Checkout the tiny library for app hydration I wrote hydrate-app. Let me know if it works or not.
  • The Feather App - my inspiration
  • Offline-First resources - a huge list of resources for people who want the web application to work even if they are offline. Nothing specific about web app hydration though.

Ideas to explore

The ServiceWorker feature allows to cache and returns quickly HTML and JavaScript (or even some unexpected value, see service-turtle). What if instead of using localStorage to store the HTML snapshots, we sent the HTML to the ServiceWorker, who would cache it? Basically, it will rewrite the page HTML, and on the next load would return the modified HTML - making the dynamic page appear static!

The current DOM snapshot is being replaced by the newly initialized and rendered application. This is probably fine for Angular, Vue, but React could patch it up from the VirtualDom, thus removing the need to actually swap one HTML with another. Might be worth exploring.