Triple Tested Static Site Deployed to GitHub Pages Using GitHub Actions

How to test static sites three times before and after deployment to GitHub pages.

This blog post shows something I have wanted to achieve for a while: deploying a fully tested static site or a web application and then testing the deployed site.

Introduction

My previous attempts used a combination of continuous integration (CI) systems and deployment tools; in all cases the result is a workflow that was unnecessary complex and hard to understand. You can see these attempts for yourself from these blog posts:

This blog post shows how to deploy a static site or a web apps to GitHub Pages while testing the site three times using GitHub Actions:

  • first time running site in development mode
  • second time by building the production bundles and serving the site using a static server
  • third time after deploying the site to GitHub Pages

Source code: you can find the code for this blog post in bahmutov/triple-tested. You can see the finished (well-tested) version of the site deployed at https://glebbahmutov.com/triple-tested/.

Let's roll!

The static site

As an example, I created a static blog site using VuePress. First, let's install it using NPM

1
2
3
$ npm init --yes
$ npm i -D vuepress
+ [email protected]

In package.json we can add commands to run the local development server and build the production site:

1
2
3
4
5
6
7
8
9
10
{
"name": "triple-tested",
"scripts": {
"docs:dev": "vuepress dev docs",
"docs:build": "vuepress build docs"
},
"devDependencies": {
"vuepress": "1.4.0"
}
}

Then I have added a new folder called docs with a few Markdown files - these files will be converted into HTML pages using VuePress settings from docs/.vuepress folder.

1
2
3
4
5
6
7
8
9
10
11
12
13
$ ls -la docs/
total 16
drwxr-xr-x 5 gleb staff 160 Mar 22 17:23 .
drwxr-xr-x 13 gleb staff 416 Mar 22 17:30 ..
drwxr-xr-x 4 gleb staff 128 Mar 21 14:51 .vuepress
-rw-r--r-- 1 gleb staff 101 Mar 22 17:28 README.md
-rw-r--r-- 1 gleb staff 152 Mar 22 17:29 about.md

$ cat docs/.vuepress/config.js
module.exports = {
title: 'Triple Tested',
description: 'Example static site'
}

The README.md file has the content for the main or index page

docs/README.md
1
2
3
4
5
6
7
8
# Main page
> This is an experiment

- does it work?
- maybe
- hope so!

Go to [about page](./about)

The about.md file has a link back to the main page

docs/about.md
1
2
3
4
5
# About

This is a static blog used as example in [bahmutov/triple-tested](https://github.com/bahmutov/triple-tested) repo.

Back to the [main page](/)

We can start the local development server that bundles the Markdown files and serves the HTML pages using npm run docs:dev command. The vuepress dev docs command serves the content at port 8080 by default.

1
2
$ npm run docs:dev
success [17:38:19] Build 62f9fb finished in 116 ms! ( http://localhost:8080/ )

Open that URL in your browser and find a nice, clean static site

Main page to About page and back

The development server watches the source files, and automatically rebuilds the pages on changes. It also forces the browser to reload the page.

Development process reloads the page when Markdown file changes

Tip: the above example used relative link [about page](./about), but according to VuePress docs you should use the file path with extension [about page](./about.md) to generate the correct link.

Building the production site

We cannot run the npm run docs:dev in production though: we need to convert the Markdown into finalized HTML pages. VuePress has the build command that does the production build and optimizes the HTML, CSS and JavaScript asset bundles.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
$ npm run docs:build

> [email protected] docs:build /Users/gleb/git/triple-tested
> vuepress build docs

wait Extracting site metadata...
tip Apply theme @vuepress/theme-default ...
tip Apply plugin container (i.e. "vuepress-plugin-container") ...
tip Apply plugin @vuepress/register-components (i.e. "@vuepress/plugin-register-components") ...
tip Apply plugin @vuepress/active-header-links (i.e. "@vuepress/plugin-active-header-links") ...
tip Apply plugin @vuepress/search (i.e. "@vuepress/plugin-search") ...
tip Apply plugin @vuepress/nprogress (i.e. "@vuepress/plugin-nprogress") ...

✔ Client
Compiled successfully in 7.25s

✔ Server
Compiled successfully in 4.58s

wait Rendering static HTML...
success Generated static files in docs/.vuepress/dist.

The build command prepares the static site for deployment - assuming the hosting platform "simply" serves the produced folder docs/.vuepress/dist.

1
2
3
4
5
6
7
$ ls -la docs/.vuepress/dist
total 16
drwxr-xr-x 5 gleb staff 160 Mar 22 17:57 .
drwxr-xr-x 4 gleb staff 128 Mar 22 17:56 ..
-rw-r--r-- 1 gleb staff 2990 Mar 22 17:57 about.html
drwxr-xr-x 5 gleb staff 160 Mar 22 17:57 assets
-rw-r--r-- 1 gleb staff 2508 Mar 22 17:57 index.html

In fact, we can serve the dist folder ourselves to see how it looks. Let's use a quick static server utility called serve:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
$ npm i -D serve
+ [email protected]
$ npx serve docs/.vuepress/dist

┌────────────────────────────────────────────────┐
│ │
│ Serving! │
│ │
│ - Local: http://localhost:5000 │
│ - On Your Network: http://10.0.0.124:5000 │
│ │
│ Copied local address to clipboard! │
│ │
└────────────────────────────────────────────────┘

Open localhost:5000 in the browser - it should look and work exactly the same way as the development server, and perhaps even faster, since it was optimized for production and does not include the extra development code.

Production static site served from dist/.vuepress/dist folder

Deploying to GitHub Pages

Now that we have built such a beautiful content, let's host it somewhere. I will use GitHub Page to host. I love using GH Pages for static pages, even this blog you are reading is hosted there! Any public GH repository can push HTML files into gh-pages branch, and the GH Pages serve it.

I could push the dist folder to gh-pages branch manually, but I want to automate this process. Thus I have created a GitHub Actions workflow file .github/workflow/ci.yml that checks out the code, installs NPM dependencies, builds the production site and deploys the dist folder to the gh-pages branch where GitHub Pages host it:

.github/workflow/ci.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
name: ci
# only deploy site from master branch
on:
push:
branches:
- master
jobs:
test:
runs-on: ubuntu-latest
steps:
# https://github.com/actions/checkout
- name: Checkout 🛎️
uses: actions/[email protected]
with:
persist-credentials: false

- name: NPM install 📦
uses: bahmutov/[email protected]

- name: Build site 🏗
run: npm run docs:build

# https://github.com/marketplace/actions/github-pages-action
- name: Deploy 🚀
uses: peaceiris/[email protected]
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./docs/.vuepress/dist

Note: if you have never used GitHub Actions, spend 30 minutes watching my recent talk GitHub Actions in Action and browse the slides, or read the blog post Trying GitHub Actions.

In the above ci.yml file we are using 3rd party GitHub action from repo peaceiris/actions-gh-pages and pass it two parameters. The secrets.GITHUB_TOKEN is created automatically by GitHub workflow; this token allows the action to push a new commit back to the gh-pages branch if there are changed HTML files. The public_dir parameter tells the action which folder we want to deploy.

Warning ⚠️: if the first deploy fails, read this issue - there seems to be a bug with GH Pages and the GITHUB_TOKEN on first deploy. I pushed the code manually once using deploy-gh-pages and my personal GitHub access token GH_TOKEN=... npx deploy-gh-pages ./docs/.vuepress/dist. After that the ci.yml was able to push the new commits automatically.

You can see the workflow ci run at GitHub inside the repo's Actions tab

Workflow "ci" has successfully finished

Before we look at the deployed site, let's go back to the local development mode and make sure it works correctly.

Local testing

In a section above, I have shown navigation from the Main page to the About page and back by clicking on the links.

Main page to About page and back

What if we accidentally change the filename about.md to info.md? We will have a broken link, which leads to frustrated users. Let's prevent this by adding an end-to-end Cypress test. First, we will install Cypress test runner and a useful utility start-server-and-test to help run and stop multiple processes.

1
2
3
$ npm i -D cypress start-server-and-test
+ [email protected]
+ [email protected]

When we work locally we want to run the VuePress develop command, wait for the server to respond at port 8080, then we can open Cypress. Here are the scripts:

package.json
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
{
"scripts": {
"docs:dev": "vuepress dev docs",
"docs:build": "vuepress build docs",
"cy:open": "cypress open",
"cy:run": "cypress run",
"dev": "start-test docs:dev 8080 cy:open"
},
"devDependencies": {
"cypress": "4.2.0",
"serve": "11.3.0",
"start-server-and-test": "1.10.11",
"vuepress": "1.4.0"
}
}

Whenever we need to start everything locally, we will use npm run dev script. Let's write a Cypress test.

cypress/integration/spec.js
1
2
3
4
5
6
7
8
9
10
11
12
13
/// <reference types="cypress" />
describe('Triple tested', () => {
it('has index and about pages', () => {
cy.visit('/')
cy.contains('h1', 'Main page')
cy.contains('hope so!')
cy.contains('a', 'about page').click()
cy.url().should('match', /about/)
cy.contains('h1', 'About')
cy.contains('a', 'main page').click()
cy.url().should('not.match', /about/)
})
})

The test does not hardcode the URL to visit - instead the command cy.visit('/') relies on baseUrl set in the cypress.json file

cypress.json
1
2
3
{
"baseUrl": "http://localhost:8080"
}

Start the development server and open Cypress with NPM script

1
2
3
4
5
$ npm run dev
# start-test docs:dev 8080 cy:open
# - runs "docs:dev" script
# - waits for port 8080 to respond
# - opens Cypress application

The test in spec.js runs very quickly

Cypress test going from Main to About page and back using links

Tip: if we only wanted to check the local Markdown links, we could just do so, read the blog post Check links in your Markdown documents. But in this blog post we want to write other kinds of functional tests.

While we are at it, we can verify the "Search" behavior. VuePress automatically adds search widget with page titles. Users can go to a page by typing the title:

Going to the About page using the search

Let's write a test. First, we need to select the search input box. Here is the relevant markup:

HTML markup for search widget

Tip: I have described how to test VuePress Search widget in details in the blog post VuePress and some Cypress end-to-end testing tips

1
2
3
4
5
6
7
8
9
10
11
12
13
it('finds About page using search', () => {
cy.visit('/')
cy.get('.search-box input').type('about')
// suggestions list appears
cy.get('.suggestions li').should('be.visible')
// and should have at least 1 item
.and('have.length.gte', 1)
// and the first search result is our "About" page
.first()
.contains('.page-title', 'About').click()
// check we are on the right page
cy.title().should('contain', 'About')
})

The test runs and confirms the search is working

The search test finds the About page

Local test on CI

Now that we have end-to-end tests, let's run them on GitHub Actions CI. We could extend the ci.yml with commands to start the development server in the background and run Cypress tests like this:

1
2
3
4
5
6
7
8
- name: NPM install 📦
uses: bahmutov/[email protected]

- run: npm run docs:develop &
- run: npm run cy:run

- name: Build site 🏗
run: npm run docs:build

Or we could use start-server-and-test utility on CI by defining one more script:

package.json
1
2
3
4
5
6
7
{
"scripts": {
"docs:dev": "vuepress dev docs",
"cy:run": "cypress run",
"dev:ci": "start-test docs:dev 8080 cy:run"
}
}

Where the cypress run command just runs Cypress headlessly.

ci.yml
1
2
3
4
5
6
7
- name: NPM install 📦
uses: bahmutov/[email protected]

- run: npm run dev:ci

- name: Build site 🏗
run: npm run docs:build

The above approach is imperfect. The action step bahmutov/[email protected] does NOT understand how to cache Cypress binary correctly, leading to slow or failing builds. To greatly simplify running Cypress on GitHub Actions CI, we have written cypress-io/github-action. This action "knows" how to install Cypress and NPM dependencies, cache them, start the server, and run the tests. Let's use this action:

ci.yml
1
2
3
4
5
6
7
8
- name: Install and test 📦✅
uses: cypress-io/[email protected]
with:
start: 'npm run docs:dev'
wait-on: 'http://localhost:8080'

- name: Build site 🏗
run: npm run docs:build

The GitHub Actions execute and pass

Cypress tests successfully ran on GitHub

We can see the logs from the npm run docs:dev command followed by the Cypress' terminal output.

Cypress GitHub Action logs

Nice, our static site will work correctly when deployed; we have tested it!

Or will it?

The problem with deployed site

We have tested the site on GitHub CI by running the VuePress development server. Then we built the site and pushed the ./docs/.vuepress/dist folder to GitHub Pages. Everything must be good, right? Let's check.

I have a custom domain glebbahmutov.com configured with GitHub Pages via CloudFlare. My personal repository bahmutov/bahmutov.github.io is hosted at https://glebbahmutov.com/ and every repo <name> will be available under https://glebbahmutov.com/<name>/ sub path. Thus we can open https://glebbahmutov.com/triple-tested/ and see our blog.

Let's open that URL ...

The deployed site is terribly broken

Oh no, the site ... does not look good. Click on the "About" page link - we can browser to it. But going to the "Main" page does not work. Instead it redirects to the top level domain https://glebbahmutov.com.

Link to the main page is wrong

Open the DevTools Network tab and reload the main page - you will see that only the index.html really loads, all other links fail to load.

Network tab shows that resources fail to load

Clicking on any resource shows the full path - and it looks wrong, because it does not include /triple-tested/ folder.

Failed CSS resource

Let's look at the source of the page - notice that all resource links start with /assets/..., which means they are served from the root of the domain.

Deployed site HTML

Each resource link "forgets" to include the base url of the site - the subfolder /triple-tested part. We should have read the entire VuePress deployment guide that explains GitHub deployment in details:

VuePress deployment guide to GitHub Pages

Ok, that can be done. But before we fix the problem, we need to

...

write a test for it.

Testing the deployed site

As you saw above, things can go wrong when deploying a site. The problems that arise are outside of source code of the site itself. It was not the link [about page](./about) from the Main page to the About page that we have tested. It was not the Search widget either. No, instead we have discovered a hosting configuration problem. If instead of our small static site example, we had a complex web application, we could have wrong environment variables, an invalid database connection string, an unreachable proxy address - all the things that worked perfectly locally, but were set up incorrectly in production.

To really validate the deployed production site, we need to run a quick smoke end-to-end test, just to see if the entire stack works together.

Remember how we set the base url to test in cypress.json? It is pointing at http://localhost:8080, but we can overwrite it from command line or using an environment variable. Let's see if our current tests catch the errors with the deployed site. Open Cypress GUI locally, but pass the full production site URL:

1
2
3
$ npx cypress open --config baseUrl=https://glebbahmutov.com/triple-tested
# or equivalent
$ CYPRESS_baseUrl=https://glebbahmutov.com/triple-tested npx cypress open

Then run the same tests against it.

Running end-to-end tests against the deployed site

The "finds About page using search" test catches the broken site - because the JavaScript bundle for the search widget did not load.

Tip: notice the 404 HTTP status codes Cypress shows in its terminal output - meaning the page under test fails to load a lot of its resources! That's a red flag.

1
2
3
4
5
6
7
8
9
10
11
GET /__cypress/iframes/integration/spec.js 200 1.025 ms - 629
GET /__cypress/tests?p=cypress/integration/spec.js-331 200 1.689 ms - -
GET /triple-tested/ 200 2.891 ms - -
GET /assets/css/0.styles.59ba12ca.css 404 22.829 ms - -
GET /assets/js/app.b08d2fd8.js 404 26.132 ms - -
GET /assets/js/5.620a0c16.js 404 31.183 ms - -
GET /assets/js/2.28d5c6fe.js 404 32.865 ms - -
GET /assets/js/3.d052a4f0.js 404 32.142 ms - -
GET /assets/js/4.5f60d4ea.js 404 28.854 ms - -
GET /assets/js/6.fa5f4b7d.js 404 30.396 ms - -
GET /triple-tested/about 200 92.170 ms - -

Why did the first test "has index and about pages" pass? I remember it had an assertion after navigating back from the "About" to the "Main" page... It should have failed.

1
2
cy.contains('a', 'main page').click()
cy.url().should('not.match', /about/)

Hmm, look at the time traveling debugger - the assertion passes, but for a wrong reason - the site's url is https://glebbahmutov.com that does not match the regular expression /about/, so the test "thinks" everything is peachy.

The negative assertion against the URL passes

You have to be careful about negative assertions like should('not.match', ...), because they can be passing but for completely unexpected (and wrong) reasons. Let's add a positive assertion that checks something on the page that only the right Main page has.

1
2
3
4
cy.contains('a', 'main page').click()
cy.url().should('not.match', /about/)
// only our Main page has <h1>Main page</h1>
cy.contains('h1', 'Main page')

The test fails correctly.

The first test correctly fails

Tip: Cypress is a functional testing tool. Thus if the elements are present on the page, Cypress will happily click on the buttons, navigate using links, check the DOM and urls, etc. And it won't "notice" that the page's styles are missing, and everything looks broken. For that you need to add visual testing; luckily it not too difficult to do.

Testing the deployed site from CI

We cannot rely on manual testing of the deployed site. We need to automatically test the site from CI after it has been deployed. Since we deploy the site to GitHub Pages from GitHub Actions, let's write another action workflow that would be triggered after the deploy finish. I will create a new file .github/workflows/deployed.yml with another GitHub workflow. Creating a separate workflow file is a great for keeping your CI files simple to understand and use.

1
2
3
4
5
6
7
8
9
10
11
12
13
name: deployed
on:
status:
branches:
- master
jobs:
test-deployed-page:
if: github.event.context == 'github/pages' && github.event.state == 'success'
runs-on: ubuntu-latest
steps:
- run: echo "gh-pages 📑 built successfully ✅"
- name: show deployed page
run: curl -s https://glebbahmutov.com/triple-tested

This workflow only runs when a commit on the master branch has its status updated. GitHub Pages bot sets this status when it starts building the pages, and when it finishes the deployment. We can use GitHub Actions expressions to skip the job test-deployed-page unless the event that triggered the workflow tells us the GitHub Pages deploy was a success.

Tip: you can dump the entire GH event object and inspect all the fields in order to write the if: ... expressions. Here is a dummy workflow job that simply prints the event (it is a huge JSON object)

1
2
3
4
5
6
7
show-event:
runs-on: ubuntu-latest
steps:
- name: Dump GitHub context
env:
GITHUB_CONTEXT: ${{ toJson(github) }}
run: echo "$GITHUB_CONTEXT"

Let's see this workflow in action. I have added line another one 👆 the docs/README.md file and have commited and pushed the code.

1
2
3
...
Go to [about page](./about)
another one 👆
1
2
3
4
5
$ git add docs/README.md
$ git commit -m "another one"
[master 1e8d247] another one
1 file changed, 1 insertion(+)
$ git push

We can see the 3 workflows at GitHub: the original ci.yml on push to master branch, followed by the skipped deployed.yml workflow (because the event status is pending), and then the full deployed workflow runs when the page has been deployed.

The workflows: one builds and deploys, the other runs after the deploy

We can click on the finished deployed workflow and look at the output of the curl -s https://glebbahmutov.com/triple-tested command. The site has been updated with new text

The static site has been updated

Now that we have the workflow that runs AFTER the deploy, let's run Cypress tests there, using the same cypress-io/github-action code, but just pointing the tests at the external site.

1
2
3
4
5
6
7
8
9
10
11
12
13
- name: show deployed page
run: curl -s https://glebbahmutov.com/triple-tested/

# we need our source code that has package.json and Cypress specs
- name: Checkout 🛎️
uses: actions/[email protected]
with:
persist-credentials: false

- uses: cypress-io/[email protected]
name: test deployed site
with:
config: baseUrl=https://glebbahmutov.com/triple-tested/

Note: if we only change the deployed.yml file, the deploy event WILL NOT be triggered - because the VuePress content has stayed the same! Thus we need to change some text in the docs folder in order to trigger a new deploy and see the tests in action. I will remove another one 👆 line and push again.

The CI workflows shows the deployed workflow failing - because the tests we have added detect the broken static site

The deployed workflow fails as expected

Click on the deployed workflow to see both Cypress tests failing as expected.

Both tests failed against the production site

We have automated the deployment and testing the production site. At least now we know when the deployment went badly and corrective measures should be taken.

Fixing the deploy url

After reading the VuePress deploy guide one more time, I open the docs/.vuepress/config.js file and add the base parameter:

docs/.vuepress/config.js
1
2
3
4
5
module.exports = {
title: 'Triple Tested',
description: 'Example static site',
base: '/triple-tested/'
}

The base parameter also changes the local url when using npm run docs:dev script, thus we update cypress.json (and related start-server-and-wait parameters)

cypress.json
1
2
3
{
"baseUrl": "http://localhost:8080/triple-tested",
}
.github/workflows/ci.yml
1
2
3
4
5
- name: Install and test 📦✅
uses: cypress-io/[email protected]
with:
start: 'npm run docs:dev'
wait-on: 'http://localhost:8080/triple-tested/'

Let's push the commit, the local tests pass, then the deploy happens. And now the post-deploy Cypress tests pass!

Post-deploy CI job passes

We can check the site at GitHub Pages - it is working.

The site is working correctly

Great!

Preventing broken deployments

Can we prevent a broken site from being deployed to production in the first place? Could we have caught an invalid or missing base configuration?

When we tested the site in the section Local test on CI, we used the development VuePress server via vuepress dev docs command. The served site passed the tests, then we built the production version and deployed it. We then ran the production site - which failed at first. We should have tested the production version before deployment in a way similar to the production site. I have already showed that we can serve the dist folder using serve module: $ npx serve docs/.vuepress/dist.

Let's simulate the broken build again by removing the base property from the file docs/.vuepress/config.js:

docs/.vuepress/config.js
1
2
3
4
5
6
module.exports = {
title: 'Triple Tested',
description: 'Example static site',
// removing "base" property to break production site
// base: '/triple-tested/'
}

To simulate sub path, we can serve the bundle from a nested folder.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# build the production site
$ npm run docs:build
$ mkdir output
$ cp -r docs/.vuepress/dist output/triple-tested

# let's look at the "production" folder
$ ls -la output
total 0
drwxr-xr-x 3 gleb staff 96 Mar 23 22:02 .
drwxr-xr-x 14 gleb staff 448 Mar 23 22:02 ..
drwxr-xr-x 5 gleb staff 160 Mar 23 22:02 triple-tested

$ ls -la output/triple-tested/
total 16
drwxr-xr-x 5 gleb staff 160 Mar 23 22:02 .
drwxr-xr-x 3 gleb staff 96 Mar 23 22:02 ..
-rw-r--r-- 1 gleb staff 2990 Mar 23 22:02 about.html
drwxr-xr-x 5 gleb staff 160 Mar 23 22:02 assets
-rw-r--r-- 1 gleb staff 2512 Mar 23 22:02 index.html

Now let's serve the output folder

1
2
3
4
5
6
7
8
9
10
11
12
$ npx serve output

┌────────────────────────────────────────────────┐
│ │
│ Serving! │
│ │
│ - Local: http://localhost:5000 │
│ - On Your Network: http://10.0.0.124:5000 │
│ │
│ Copied local address to clipboard! │
│ │
└────────────────────────────────────────────────┘

Open http://localhost:5000/triple-tested in the browser to see exactly the same broken site as we had in production before.

Accessing the sub path site locally shows missing resources

Cypress catches the broken site

1
$ CYPRESS_baseUrl=http://localhost:5000/triple-tested npx cypress open

Testing production bundle locally

Let's restore the base property in file docs/.vuepress/config.js, build the site, copy the dist folder into output and try again.

Testing production site built correctly using base sub path

Great, the tests pass when when we correctly set the base property. Now we need to make sure we test the production site on CI. Let's add these commands to ci.yml file

.github/workflows/ci.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
name: ci
# only deploy site from master branch
on:
push:
branches:
- master
jobs:
test:
runs-on: ubuntu-latest
steps:
# https://github.com/actions/checkout
- name: Checkout 🛎️
uses: actions/[email protected]
with:
persist-credentials: false

# https://github.com/cypress-io/github-action
- name: Install and test 📦✅
uses: cypress-io/[email protected]
with:
start: 'npm run docs:dev'
wait-on: 'http://localhost:8080/triple-tested/'

- name: Build site 🏗
run: npm run docs:build

- name: Prepare built site for testing 🚛
run: |
mkdir output
cp -r ./docs/.vuepress/dist output/triple-tested

- name: Test production site
uses: cypress-io/[email protected]
with:
# we have already installed all dependencies above
install: false
start: 'npx serve output'
wait-on: 'http://localhost:5000/triple-tested/'
config: baseUrl=http://localhost:5000/triple-tested/

# https://github.com/marketplace/actions/github-pages-action
- name: Deploy 🚀
uses: peaceiris/[email protected]
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./docs/.vuepress/dist

The GitHub Actions run and pass.

GitHub Actions tested the site in dev mode and as built production bundle

The deploy went through to GitHub Pages and then was tested the 3rd time

Test jobs before and after deploy have passed

Tips

Before I reach the Discussion section, let me give a couple of tips.

GitHub Actions badges

For each GitHub workflow you can add a badge to README file. In our case we will have two separate workflows:

1
2
![ci](https://github.com/bahmutov/triple-tested/workflows/ci/badge.svg?branch=master)
![deployed](https://github.com/bahmutov/triple-tested/workflows/deployed/badge.svg?branch=master)

If you want each badge to lead immediately to matching workflow, add destination urls

1
2
[![ci](https://github.com/bahmutov/triple-tested/workflows/ci/badge.svg?branch=master)](https://github.com/bahmutov/triple-tested/actions?query=workflow%3Aci)
[![deployed](https://github.com/bahmutov/triple-tested/workflows/deployed/badge.svg?branch=master)](https://github.com/bahmutov/triple-tested/actions?query=workflow%3Adeployed)

CI badges

Environment variables

We can avoid hard-coding the production URL in several places in CI configuration files, and instead can use GitHub environment variables to specify the base url to test.

Instead of

deployed.yml
1
2
3
4
- uses: cypress-io/[email protected]
name: test deployed site
with:
config: baseUrl=https://glebbahmutov.com/triple-tested/

we will set the CYPRESS_baseUrl at the job level

deployed.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
name: deployed
on:
status:
branches:
- master
jobs:
test-deployed-page:
if: github.event.context == 'github/pages' && github.event.state == 'success'
runs-on: ubuntu-latest
env:
CYPRESS_baseUrl: https://glebbahmutov.com/triple-tested/
steps:
- run: echo "gh-pages 📑 built successfully ✅"
- name: show deployed page
run: curl -s $CYPRESS_baseUrl

# we need our source code that has package.json and Cypress specs
- name: Checkout 🛎️
uses: actions/[email protected]
with:
persist-credentials: false

# https://github.com/cypress-io/github-action
- uses: cypress-io/[email protected]
name: test deployed site

Record to Dashboard

GitHub Actions give you a lot of free resources for public repositories. But storing and viewing the Cypress test artifacts (screenshots on failure, videos of the test runs, test results) is inconvenient. Thus I recommend using Cypress Dashboard to record the tests.

I have set up project to record and stored the CYPRESS_RECORD_KEY as a secret at GitHub repo settings.

Cypress record key kept in secrets

We can set the secret as environment variable in the specific steps that run Cypress tests, and pass record: true parameter. You can find all parameters in examples inside the cypress-io/github-action repository.

ci.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
...
- name: Install and test 📦✅
uses: cypress-io/[email protected]
with:
start: 'npm run docs:dev'
wait-on: 'http://localhost:8080/triple-tested/'
record: true
group: 'development'
tag: 'ci'
env:
CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
...
- name: Test production site
uses: cypress-io/[email protected]
with:
# we have already installed all dependencies above
install: false
start: 'npx serve output'
wait-on: 'http://localhost:5000/triple-tested/'
config: baseUrl=http://localhost:5000/triple-tested/
record: true
group: 'production bundle'
env:
CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
...

In the deployed.yml workflow we similarly record the test results

1
2
3
4
5
6
7
8
...
- uses: cypress-io/[email protected]
name: test deployed site
with:
record: true
tag: 'production'
env:
CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}

You can add a badge to the README.md pointing at the project's dashboard, so users can find the test results. The project Id is stored in cypress.json file, in our case it is rroimy.

1
[![Cypress Dashboard](https://img.shields.io/badge/cypress-dashboard-brightgreen.svg)](https://dashboard.cypress.io/#/projects/rroimy/runs)

See results for yourself: Cypress Dashboard

For each deployment, we see two runs: one is tagged ci and and the second one is tagged production.

Two runs for each deployment

In the run tagged ci, the two sets of tests are in two groups:

CI run has two groups of tests

Each spec has its video captured, and if there were any failures, there would be screenshots too.

Discussion

I have shown how to build, test and deploy a static site using GitHub Actions and Pages working together. Because many things can go wrong, I have tested the site three times:

  • First using the development server that comes with VuePress.
  • Second time after building the production version by serving it locally to simulate the sub path hosting
  • Third time after deployment to check if the production site is working for my users

While triple testing might seem like an overkill, I hope you see why it was necessary. We ran end-to-end tests after each major code transformation: building the production bundle is such a big change, it was necessary to validate the produced bundle. Deploying the site to GitHub Pages is also a big change, so we want to run tests after that. GitHub Actions allowed us to run Cypress tests in all scenarios.

We did not have to run all tests in every scenario. For example, we could have executed just a few smoke tests after building the production bundles and after the deployment. Read the blog post Use meaningful smoke tests about running a small subset of tests.

Source code: you can find the code for this blog post in bahmutov/triple-tested. You can see the finished (well-tested) version of the site deployed at https://glebbahmutov.com/triple-tested/.