- The site
- Vercel deployment
- Preview deploys
- GitHub Checks
- Cypress Dashboard
- Cypress GH Integration
- More info
# My site
11ty NPM package
npm i -D @11ty/eleventy
And start the local site
npx eleventy --serve
The static page does not look like much - but the generator is fast and simple to use.
Tip: by default
eleventy --start serves the page at port 8080. If that port is busy, it automatically serves at the next available port, in this case the site was serves at port 8081.
We need the entire world to see our awesome site. Let's deploy it using Vercel platform. I have created a new project and picked "11ty" application's default settings for the build command and output folder.
I have linked the Vercel project with the GitHub repository bahmutov/eleventy-example
Every time we push a new commit to the
main branch, the static site is built and deployed globally under a subdomain of
Here is the production site deployed from the
We probably want to deploy the top level index page as well, and navigate to the
/README page. Let's throw a root
The navigation works: when the users clicks on the link, the browser goes to the
/README page. When the user clicks the browser's "back" button, it navigates back to the index page.
I want to deploy the new index page - but I am not sure if the top level navigation is going to work with an actual domain (because I seriously doubt my programming skills). It would be a nice idea to deploy the full site to a temporary preview domain first, test it out, and if it works correctly, then merge the pull request to the
main branch, which deploys to production.
I will create a new branch
add-index-page and commit the new code there:
~/git/eleventy-example on add-index-page
In my GitHub repository I have installed the Vercel GitHub App which automatically deploys every pull request.
By pushing the new branch to the repository and opening a pull request #2 I trigger the deployment. The Vercel bot comments on the PR with the deployed URL
We can click on the preview URL to visit the deployed site and confirm manually that the navigation works
The PR preview URL
https://eleventy-example-git-add-index-page.bahmutov.vercel.app/ is a concatenation of the project name "eleventy-example", the source type "git", the branch name "add-index-page", my user name "bahmutov", and the top level domain name "vercel.app". In addition, there is a unique preview URL for every commit. You can find these URLs at the Vercel deploy page.
If we push more commits to the branch
add-index-page, the new deploys will get their new unique
eleventy-example-<HASH>.vercel.app URLs, while the latest branch preview will still have the top level
We have tried the PR preview manually, but a better idea to prevent bugs in the deployed web applications is to write automated end-to-end tests using Cypress. Let's install Cypress and write a test in our pull request.
npm i -D cypress
When we ran the site locally, we tested it at the
localhost:8080. Let's put this setting into
Our spec file will perform what we have done manually - it will navigate by using the link to the
/README page and back.
/// <reference types="cypress" />
Start Cypress with
npx cypress open while the application is running and observe the test passing for the right reason. Hover over each command to observe the site's DOM snapshot and how it changed in response to the anchor
Tip: read how to write flake-free Cypress tests when navigating from page to page in the blog post When Can The Test Navigate?.
We ran the above test locally. Let's run the same test automatically against the deployed preview URL. Luckily for us, Vercel for GitHub dispatches deployment events to GitHub, and we can write a GitHub Action that would execute in response to this event. Let's first simply print the event object to see if it has the target preview URL to test.
Tip: if you have never used GitHub Actions, read my Trying GitHub Actions blog post.
pushevent happens for every commit. This would give us a chance to test the site separately from the deployment. For example, we could lint the site's source text and run the Cypress tests against the site running locally.
deploymentevent happens when Vercel starts the preview deploy - it does not have the deploy URL
deployment_statusevent is sent by Vercel twice. First, when the preview deployment starts, and second time when the preview deployment finishes.
Notice that the
deployment_status is not listed in the PR checks - they are "hidden" or overwritten by the "deployment" event, which to me seems like a bad user interface.
Instead, we need to look at the "Actions" tab to see the
ci workflows run, and by clicking inside figure out they ran triggered by the
deployment_status event has the following information inside the
github JSON event
"description": "Vercel is deploying your app",
deployment_status event inside the GitHub CI action has status
"description": "Deployment has completed",
We can limit the GitHub Action to only run a job when an expression is true. In our case we want to run end-to-end tests only after successful deploy - and we want to pass the "target_url" as
baseUrl parameter. We can pass the base url using the
CYPRESS_BASE_URL environment variable. Here is our updated workflow file
When we push the commit, we can see the CI jobs skipped - except for the last job.
I love requiring certain test jobs to pass before a pull request can be merged. Thus I set up protected branches with
e2e job required to allow merging.
The E2E job runs ... and fails!
Hmm, seems Vercel does not add a trailing slash when serving the production version of the code, while
11ty running locally does add one.
Testing against the deployed preview URLs just showed its usefulness. Let's update the test to be less strict.
- cy.location("pathname").should("match", /\/README\/$/);
Perfect. The test passes locally and against the preview URL.
Once the checks are green, I am confident and merge the pull request into the
main branch. Vercel then deploys the production site. I can also remove extra CI events and only run the GitHub workflow on successful deployment. To install, cache and run Cypress we can use Cypress GitHub Action:
There is one more detail that GitHub gets wrong when running tests on
deployment_status event. It seems to be confused about which commit is tested, so it never reports the status back to the Pull Request. For example in #3 we see a pending check.
To fix this, we can send the commit status ourselves using GitHub REST API call. Here is the added section of the GitHub workflow file. If the Cypress tests pass, we post success. If the Cypress tests fail, or any job step fails, we post the failure status.
# ci.yml file
You can see the example run in PR #4. The check "e2e" matches the required check name, thus the pull request is all green.
When we look at the Action steps, we can see the "Post success" step ran, while the "Post failure" step was skipped.
We can even provide a link that would take us from the status check straight to the workflow run by forming the full
target_url property when posting the status check.
This creates the little "Details" link on the right.
I have mentioned before that every Cypress run generates a video, and every failed Cypress test automatically saves the screenshot of the failure. You can store these test artifacts on GitHub, or send them to the Cypress Dashboard where they can be easily viewed. Let's set up our project for recording.
In the Cypress Desktop GUI select the "Runs" tab.
Click the "Set up project to record" button.
I will place the "eleventy-example" project under my personal "Gleb OSS" organization that has the Cypress Open Source Software billing plan. The test recordings will be public - everyone should be able to see them.
Once I click "Set up project" button, its project id will be added to the
cypress.json file and the recording key is shown. Please keep this key private.
I will set the recording key as a GitHub Action Secret in my repository.
Let's update the GitHub workflow file to record test results and artifacts. We are using Cypress GitHub Action, thus we need to add
record: true parameter and pass the recording secret as an environment variable.
# updated ci.yml file
The tests run on GitHub and show the recorded run URL
Cypress GH Integration
If we are using GitHub and Cypress Dashboard, we might as well use the Cypress GitHub Integration App. I can add it to all my repositories, or just install it for selected repository "eleventy-example".
Once installed, we can configure how the Cypress GH App comments on pull requests and sets status checks.
Let's try a new pull request #6 - we will see Vercel's comment and Cypress' GH comment.
Cypress GH App adds its own status check to the commit.
Because we have the Cypress GH status check, we could remove our custom test job check implemented using
curl commands. I will leave them in for completeness sake.
If you want to test a site deployed to GitHub Pages, read Triple Tested Static Site Deployed to GitHub Pages Using GitHub Actions.
You can test deployed previews using Netlify + CircleCI combination
For full confidence, we do recommend adding visual testing using open source or commercial service to your functional end-to-end tests. Visual testing will prevent any CSS or style regressions from creeping into your site.
We also recommend for complex web applications to measure the code coverage to ensure all implemented features are covered by end-to-end tests. E2E tests are extremely effective at covering a lot of code.