Run End-to-end Tests from Markdown Files

A way to test Cypress examples in Markdown files to guarantee their correctness.

Cypress documentation has a lot of examples. Pretty much every command, like cy.get or cy.last has several realistic examples.

`cy.last` example

There is a huge problem with documentation examples - they might be wrong. Since they are copied from tests, or written by hand without running them, there is no guarantee that they are correct with respect to the latest Cypress version.

We could write a real end-to-end test for each example and keep it in sync, but this requires a lot of work to stay up-to-date. What if we instead executed the examples directly from the documentation pages? After all, each example has the HTML to mount and test code to run, so it contains all the parts of a real test as code blocks.

This is what @cypress/fiddle does (now you should use my fork at @bahmutov/cypress-fiddle). It contains a test file preprocessor bahmutov/cypress-markdown-preprocessor that understand the Markdown file format, finds all specially marked example blocks and creates a Cypress test out of them. For example, see bahmutov/vuepress-cypress-test-example. In that repo, the README.md file contains the following block

Markdown section with a fiddle

Notice the special HTML comments <!-- fiddle DOM test --> and <!-- fiddle-end -->. They are ignored by Markdown render engines. Thus we can safely use them to mark the beginning and the end of each test. Inside the test we can have HTML and JavaScript code blocks. The HTML block will be mounted during the test, and the JavaScript code will be executed as the body of a test. The name of the test is encoded in the starting comment, in this case it will be "DOM test". The test runs and passes.

Dom test generated from README.md section

From the Markdown file we can generate both the static site (like https://docs.cypress.io) and run Cypress tests, find one such example in bahmutov/vuepress-cypress-test-example.

So, the cy.last example in the documentation becomes a fiddle like this:

Last fiddle

And it runs as a test in Cypress

Last fiddle test passing

Happy testing!

Update 1: cypress-examples

I have used bahmutov/cypress-markdown-preprocessor and @bahmutov/cypress-fiddle to implement my cypress-examples website in the repo bahmutov/cypress-examples. It now has more than 500 small examples and medium-size recipes, all tested. This is how I can record so many Cypress examples on my YouTube Channel with Cypress Tips & Tricks. Here is a typical way I answer a Cypress-related question by coding up a new example in Markdown: