Aug 17 2015

Express sessions

Using and observing ExpressJS sessions from the client code.

This is a few notes on sessions in ExpressJS server and how to observe / debug them. You can find the companion code at github.com/bahmutov/express-sessions-tutorial. Several checkpoints are tagged, you can play with the code by cloning the repo and going to any of them

git clone https://github.com/bahmutov/express-sessions-tutorial
cd express-sessions-tutorial
npm install
git checkout step-0

step-0: ExpressJS server with local sessions save

Start a new project using npm init and install expressjs

npm install --save express

Create main file server.js

server.js

var app = require('express')();
app.get('/', function (req, res) {
  res.send('hi there');
});
var server = app.listen(3000, function () {
  var host = server.address().address;
  var port = server.address().port;
  console.log('Example app listening at http://%s:%s', host, port);
});

We can run the server, but it does nothing, not even serving stating index page yet. Just answering 'Hi there' to any request.

Before we proceed, I would add nodemon to automatically restart the server on any source file changes.

npm install --save-dev nodemon

Create watch script in the package.json

package.json

1
2
3

"scripts": {
    "watch": "nodemon server.js"
}

Now start the server via nodemon that watches source files and restarts the server whenever you make local file modifications

npm run watch

You can make a curl http://localhost:3000/ request from another terminal, I prefer using httpie.

This is the output

$ http http://localhost:3000/
HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 8
Content-Type: text/html; charset=utf-8
Date: Mon, 17 Aug 2015 18:46:06 GMT
ETag: W/"8-/TPi6K08sb3T6o9WM/z1xw"
X-Powered-By: Express
hi there

step-1 - add session persistance

Whenever a new browser connects the server, we can initialize a session cookie. Using this cookie we can determine if this is the same user making multiple visits for example. Let us install a couple of modules for handling the sessions and for storing them into the file system

npm install --save express-session session-file-store morgan

express-session parses session cookie, validates it, etc.
session-file-store saves the session object in JSON files in the local folder (sessions by default). This is not as advanced as other sessions stores, but great for debugging and learning
morgan is the usual logger middleware for Express

Add the session middleware to the server stack

server.js

var app = require('express')();
app.use(require('morgan')('dev'));
var session = require('express-session');
var FileStore = require('session-file-store')(session);
app.use(session({
  name: 'server-session-cookie-id',
  secret: 'my express secret',
  saveUninitialized: true,
  resave: true,
  store: new FileStore()
}));
app.get('/', function (req, res) {
  res.send('hi there');
});
...

You can find the session options in the express-session documentation.

The behavior of the application has not changed, but the session middleware now adds a new property to the req object passed to each middleware callback after it. Let us print the session property

server.js

...
app.use(session({
  ...
}));
app.use(function printSession(req, res, next) {
  console.log('req.session', req.session);
  return next();
});
...

If you make a curl request, the session object will be printed to the standard output

req.session { cookie:
   { path: '/',
     _expires: null,
     originalMaxAge: null,
     httpOnly: true } }
GET / 200 7.493 ms - 8

And the curl will print almost the same output, except it has additional property - the server-session-cookie-id cookie.

$ http http://localhost:3000/
HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 8
Content-Type: text/html; charset=utf-8
Date: Mon, 17 Aug 2015 18:58:04 GMT
ETag: W/"8-/TPi6K08sb3T6o9WM/z1xw"
X-Powered-By: Express
set-cookie: server-session-cookie-id=s%3ASacx84tlozr...; Path=/; HttpOnly
hi there

This cookie was generated automatically by the express-session middleware and is unique to the browser session. If you open http://localhost:3000/ in the browser and inspect the "Resources / Cookies", you will find it present too.

You can also look at the sessions/ local folder to see individual JSON file with session info, for example

$ cat sessions/cgMyI0mMSGB_zSmsy5WoLuArZvWSgIok.json
{
  "cookie": {
    "originalMaxAge": null,
    "expires": null,
    "httpOnly": true,
    "path": "/"
  },
  "__lastAccess": 1439838463473
}

The cool thing is, express-session automatically looks at the cookie server-session-cookie-id, and it is not present, creates new one, creates new session object and sets it on the request. If there is a cookie, it is parsed, and then the right session is fetched from the session store.

step-2 - count session views

We can add more information to the session object, and it will be serialized just the same. For example we can can keep the number of views in the session.

server.js

app.use(session({
  ...
  resave: true
}));
app.get('/', function initViewsCount(req, res, next) {
  if (typeof req.session.views === 'undefined') {
    req.session.views = 1;
    return res.end('Welcome to the file session demo. Refresh page!');
  }
  return next();
});

If we execute several curl requests, because the session cookie is not present, each will initialize a new counter.

$ http http://localhost:3000/
HTTP/1.1 200 OK
Connection: keep-alive
Date: Mon, 17 Aug 2015 19:28:48 GMT
Transfer-Encoding: chunked
X-Powered-By: Express
set-cookie: server-session-cookie-id=s%3A65byNjKYiFbnKl...; Path=/; HttpOnly
Welcome to the file session demo. Refresh page!

We can open the session folder and find the new file to see that the session object does in fact have a views property

$ cat sessions/y6qdHmRw5eu1OCr0JYpzoMkb_QPxJCYJ.json
{
  "cookie": {
    "originalMaxAge": null,
    "expires": null,
    "httpOnly": true,
    "path": "/"
  },
  "views": 1,
  "__lastAccess": 1439839726898
}

If we access the url through the browser, because of the cookie persistance, we will not generate a new session. Let us increment the counter for sessions that already have views.

server.js

app.get('/', function initViewsCount(req, res, next) {
  if (typeof req.session.views === 'undefined') {
    req.session.views = 0;
    return res.end('Welcome to the file session demo. Refresh page!');
  }
  return next();
});
app.get('/', function incrementViewsCount(req, res, next) {
  console.assert(typeof req.session.views === 'number',
    'missing views count in the session', req.session);
  req.session.views++;
  return next();
});
app.use(function printSession(req, res, next) {
  console.log('req.session', req.session);
  return next();
});
app.get('/', function sendPageWithCounter(req, res) {
  res.setHeader('Content-Type', 'text/html');
  res.write('<p>views: ' + req.session.views + '</p>\n');
  res.end();
});

We are going to increment the counter, and embed it in the page. The console shows the session for each page refresh

Example app listening at http://:::3000
req.session { cookie:
   { path: '/',
     _expires: null,
     originalMaxAge: null,
     httpOnly: true },
  __lastAccess: 1439840352586,
  views: 2 }
GET / 200 13.147 ms - -
req.session { cookie:
   { path: '/',
     _expires: null,
     originalMaxAge: null,
     httpOnly: true },
  __lastAccess: 1439840375987,
  views: 3 }
GET / 200 2.239 ms - -

Nice!

step-3 - same session from the command line

The browser can send the same session cookie server-session-cookie-id with each request. Can we send do the same form the command line? The curl help page explains how the cookies can be stored in a "jar". First we start the "cookie jar"

curl --cookie-jar cookies http://localhost:3000/
cat cookies
# Netscape HTTP Cookie File
# http://curl.haxx.se/docs/http-cookies.html
# This file was generated by libcurl! Edit at your own risk.
#HttpOnly_localhost FALSE   /   FALSE   0   server-session-cookie-id    s%3AW_fN2t8ImZPyn_gZH...

On the next request, use this file again and the session will be valid

$ curl --cookie cookies http://localhost:3000/
<p>views: 1</p>
$ curl --cookie cookies http://localhost:3000/
<p>views: 2</p>
$ curl --cookie cookies http://localhost:3000/
<p>views: 3</p>

Saving / sending the same cookies using httpie is even simpler. Just specify the same filename for storing / loading

$ http --session=my http://localhost:3000/
HTTP/1.1 200 OK
Connection: keep-alive
Date: Tue, 18 Aug 2015 02:05:43 GMT
Transfer-Encoding: chunked
X-Powered-By: Express
set-cookie: server-session-cookie-id=s%3AvK-ddDZo3jK6BrQDhtS...; Path=/; HttpOnly
Welcome to the file session demo. Refresh page!

$ http --session=my http://localhost:3000/
HTTP/1.1 200 OK
Connection: keep-alive
Content-Type: text/html
Date: Tue, 18 Aug 2015 02:05:47 GMT
Transfer-Encoding: chunked
X-Powered-By: Express
<p>views: 1</p>

$ http --session=my http://localhost:3000/
HTTP/1.1 200 OK
Connection: keep-alive
Content-Type: text/html
Date: Tue, 18 Aug 2015 02:05:48 GMT
Transfer-Encoding: chunked
X-Powered-By: Express
<p>views: 2</p>

The file ~/.httpie/sessions/localhost_3000/my.json contains the following

{
    "__meta__": {
        "about": "HTTPie session file",
        "help": "https://github.com/jkbr/httpie#sessions",
        "httpie": "0.8.0"
    },
    "auth": {
        "password": null,
        "type": null,
        "username": null
    },
    "cookies": {
        "server-session-cookie-id": {
            "expires": null,
            "path": "/",
            "secure": false,
            "value": "s%3AvK-ddDZo3jK6BrQDhtS..."
        }
    },
    "headers": {}
}

step-4 - same session across Node requests

Let us execute GET requests not from a browser or a command line tool, but from a client Node program. For example, we can use request-promise

npm install --save request-promise

The write a simple client in a separate file, for example client.js

var rp = require('request-promise');
rp('http://localhost:3000/')
  .then(console.dir)
  .catch(console.error);

Each run of the program will return new session

$ node client.js
'Welcome to the file session demo. Refresh page!'
$ node client.js
'Welcome to the file session demo. Refresh page!'

We need to store the cookies and send them with each request, at least while the application is running. Just using several requests does not work - the cookie is not stored / saved with each request.

client.js

var rp = require('request-promise');
function requestPage() {
  return rp('http://localhost:3000/');
}
requestPage()
  .then(console.dir)
  .then(requestPage)
  .then(console.dir)
  .then(requestPage)
  .then(console.dir)
  .catch(console.error);

$ node client.js
'Welcome to the file session demo. Refresh page!'
'Welcome to the file session demo. Refresh page!'
'Welcome to the file session demo. Refresh page!'

Luckily, it is simple to configure the request code to save / send cookies. Just set the default options jar (as in "cookie jar") to true.

client.js

var rp = require('request-promise').defaults({
  jar: true
});
...

$ node client.js
'Welcome to the file session demo. Refresh page!'
'<p>views: 1</p>\n'
'<p>views: 2</p>\n'

Can we inspect the saved cookies, similarly to how we saved the cookies into the plain local files using curl and httpie tools? Yes, but we need to use a different cookie implementation.

npm install --save tough-cookie-filestore

Due to the bug we need to create an empty output file first to store future cookies

touch cookies.json

Now create a cookie jar backed by a plain JSON file in our code

var FileCookieStore = require('tough-cookie-filestore');
var requestPromise = require('request-promise');
var rp = requestPromise.defaults({
  jar: requestPromise.jar(new FileCookieStore('cookies.json'))
});
function requestPage() {
  return rp('http://localhost:3000/');
}
requestPage()
  .then(console.dir)
  .then(requestPage)
  .then(console.dir)
  .then(requestPage)
  .then(console.dir)
  .catch(console.error);

We now can execute multiple requests using the same session and inspect the cookies

$ node client.js
'Welcome to the file session demo. Refresh page!'
'<p>views: 1</p>\n'
'<p>views: 2</p>\n'
$ node client.js
'<p>views: 3</p>\n'
'<p>views: 4</p>\n'
'<p>views: 5</p>\n'

cookies.json

$ c cookies.json
{"localhost":{"/":{"server-session-cookie-id":{
    "key":"server-session-cookie-id",
    "value":"s%3AqSHbMfG4G0SIl1pdI...",
    "domain":"localhost",
    "path":"/",
    "httpOnly":true,
    "hostOnly":true,
    "creation":"2015-08-20T21:42:46.878Z",
    "lastAccessed":"2015-08-20T21:42:46.878Z"}
}}}

step-5 - Running ExpressJS in HTTPS mode

We will create a self-signed certificate to allow our test server to work over HTTPS locally. I followed the Express over HTTPS instructions. The initial command to create the certificate can be found in package.json under npm run make-certificate. It generates cert.pem and key.pem files. We need to load these files when starting the Express server.

server.js

var https = require('https');
var fs = require('fs');
var app = require('express')();
...
var server = https.createServer({
  key: fs.readFileSync('key.pem'),
  cert: fs.readFileSync('cert.pem')
}, app).listen(3000, function () {
  var host = server.address().address;
  var port = server.address().port;
  console.log('Example app listening at https://%s:%s', host, port);
});

When I tried running the server, I got error:0907B068:PEM routines:PEM_READ_BIO_PRIVATEKEY:bad password read, so I removed the passphrase (using npm run remove-passphrase). Then start the server node server.js and open the browser

open https://localhost:3000/

After the security warning you should see the HTTPS website with the session counter

step-6 - update client code

Because we use self-signed certificate, we need to update our Node client code to allow connecting. Otherwise we get a [RequestError: Error: self signed certificate]

client.js

var FileCookieStore = require('tough-cookie-filestore');
var requestPromise = require('request-promise');
var rp = requestPromise.defaults({
  strictSSL: false, // allow us to use our self-signed cert for testing
  rejectUnauthorized: false,
  jar: requestPromise.jar(new FileCookieStore('cookies.json'))
});
...

You should be able to connect to the server and see the incremented counter again

$ node client.js
'<p>views: 9</p>\n'
'<p>views: 10</p>\n'
'<p>views: 11</p>\n'

Similarly, other CLI tools should skip the certificate validation when trying to connect to our local server. For example, httpie

http --verify=no --session=my https://localhost:3000/

step-7 - passing Referer header from the server to the next request

Sometimes as a security measure, the server checks Referer header included in the client's request to make sure it matches whatever the server sent before. Let us simulate this in the server code

app.get('/', function sendPageWithCounter(req, res) {
  res.setHeader('Content-Type', 'text/html');
  res.setHeader('Referer', 'server referer');
  res.write('<p>views: ' + req.session.views + '</p>\n');
  res.end();
});

We can even print this header in the middleware

app.use(function printSession(req, res, next) {
  console.log('req.session', req.session);
  console.log('req header referer', req.header('Referer'));
  return next();
});

The client application code needs to read the header Referer (the headers are case-insensitive) from each response and pass it with the next request to the server. We can modify the client code to achieve this; but we now need to deal with full response, not just the body

var rp = requestPromise.defaults({
  strictSSL: false, // allow us to use our self-signed cert for testing
  rejectUnauthorized: false,
  jar: requestPromise.jar(new FileCookieStore('cookies.json'))
});
function requestPage(previousResponse) {
  var referer = previousResponse ? previousResponse.headers.referer : null;
  if (previousResponse) {
    console.log('previous response referer "%s"', referer);
  }
  return rp({
    url: 'https://localhost:3000/',
    resolveWithFullResponse: true,
    headers: {
      referer: referer
    }
  });
}
requestPage()
  .then(function (response) {
    console.log(response.body);
    return requestPage(response);
  })
  .then(function (response) {
    console.log(response.body);
    return requestPage(response);
  })
  .catch(console.error);

The server will print its own value whenever receiving a client's request

GET / 200 0.773 ms - -
req.session { cookie:
   { path: '/',
     _expires: null,
     originalMaxAge: null,
     httpOnly: true },
  views: 40,
  __lastAccess: 1440172136711 }
req header referer server referer

Nice.

CSRF protection

You can add "Cross Site Request Forgery" (CSRF for short) protection on top of the session using csurf module. See an example in the express-sessions-tutorial repo.

Bonus

Whenever you set cookies in Express, you can pass additional options. For example you can disable JavaScript access (via document.cookie property), limit cookies to specific domain or same site (see MDN Cookies).

app.use(session({
  name: 'server-session-cookie-id',
  ...
  cookie: {
    sameSite: true,
    domain: 'my.company.com'
    // session cookie is httpOnly by default
  }
}));
var csrfProtection = csrf({
  cookie: {
    key: '_csrf',   // cookie name
    sameSite: true, // never send outside with CORS
    httpOnly: true, // do not put this cookie into document.cookie
    domain: 'my.company.com' // limit cookie to 'my.company.com' and subdomains
  }
})

Note that SameSite is only supported by the Chrome browser right now.

Additional resources

Example code used in this post
ExpressJS and PassportJS Sessions Deep Dive
Solid ExpressJS server

Better world by better software

Gleb Bahmutov PhD

Our planet 🌏 is in danger

Act today: what you can do

Express sessions

Using and observing ExpressJS sessions from the client code.

step-0: ExpressJS server with local sessions save

step-1 - add session persistance

step-2 - count session views

step-3 - same session from the command line

step-4 - same session across Node requests

step-5 - Running ExpressJS in HTTPS mode

step-6 - update client code

step-7 - passing Referer header from the server to the next request

CSRF protection

Bonus

Additional resources