Testing Pyramids

cypress-react-unit-test

I have a little project I am pretty fond of: cypress-react-unit-test. It is a library that allows you to write React component tests and run them inside Cypress test runner. Since there are many React features that one might wish to test, the library has a lot of unit tests. I link them from the README and I split them into basic and advanced examples.

I use the basic vs advanced separation because I do not want the users to be overwhelmed when seeing the library for the first time. If you are must trying to learn about cypress-react-unit-test you might simply look through the list of basic examples to get a taste of what this library can do for you (hint: it can test everything). Once you start using the library to test your React components, you might want to learn how to handle some particular difficult situation - and that's when you search the advanced examples. Testing a stateless React component is a basic example. Using path aliases in Webpack and in TypeScript is an advanced example.

So there is about 100 spec files in total between the basic and advanced examples. These are the library's unit tests in my opinion.

full examples

But you are probably not going to just test a React component in a vacuum. You probably want to write the component tests inside your actual React application - and how the application is structured, bundled, and served, is really important. Do you use react-scripts, or ejected react-scripts, or Next.js, or a custom Webpack config file? Since the component tests must hook into your app's bundler's settings to prepare the component and spec, each project type needs its own settings and the right preprocessor to work. These situations are not well tested using unit tests - they are better tested using the actual fully prepared example code.

Thus to be useful to the users the cypress-react-unit-test library needs to both show an example of every commonly used React application setup, and to be tested against this setup to release its new versions without accidentally breaking users.

This is the purpose of the full examples section in the library.

These examples are all in examples/* folders. Each has its own package.json with all dependencies listed. Thus a user can quickly scan the dependencies to understand if a particular example is matching their setup. For example, the examples/rollup has the full project that bundles files using Rollup preprocessor

package.json

{
  "name": "example-rollup",
  "description": "Component testing using Rollup bundler",
  "private": true,
  "scripts": {
    "build": "rollup -c",
    "test": "DEBUG=cypress-expect cypress-expect run --passing 1",
    "cy:open": "cypress open"
  },
  "devDependencies": {
    "@bahmutov/cy-rollup": "2.0.0",
    "@rollup/plugin-babel": "5.2.1",
    "@rollup/plugin-commonjs": "15.1.0",
    "@rollup/plugin-node-resolve": "9.0.0",
    "@rollup/plugin-replace": "2.3.3",
    "cypress-react-unit-test": "file:../..",
    "cypress-expect": "2.0.2",
    "cypress": "5.3.0",
    "@babel/preset-env": "7.4.5",
    "@babel/preset-react": "7.0.0",
    "@babel/preset-typescript": "7.10.4",
    "rollup": "2.28.1",
    "react": "^16.13.1",
    "react-dom": "^16.13.1"
  }
}

Now you might notice the "cypress-react-unit-test": "file:../.." dependency. This is linking the cypress-react-unit-test that normally would have some specific version back to the root folder of the repo. This is done so that we can work on the examples/rollup and use the development version of the library straight from the root folder.

The user can see what dependencies the project is actually using, and we can change the cypress-react-unit-test while testing it against this project.

Note the testing command cypress-expect run --passing 1 that uses cypress-expect. This verifies that the tests in the example actually execute and not just pass accidentally. Read Wrap Cypress Using NPM Module API blog post for details on Cypress wrapper scripts.

continuous integration

So what we have in the repository is not a monorepo. We have the library at the root (with its own node_modules and its own build step), and we have separate example projects in their subfolders, with their own package.json files. The dependencies are NOT shared. There is no hoisting or optimization of the dependencies, no Yarn workspaces, no lerna tricks.

You want to run tests? You run npm it. That's it. Almost poetic in Dr. Seuss' sense.

On continuous integration server (in my case I am using CircleCI, see the circle.yml file) we do the following two tricks to make our life quick and easy, and our users' lives safer:

1. We install the root level NPM dependencies and cache ~/.npm and ~/.cache folders. We also bring the workspace from the install job to the rest of the workflow's jobs. Every time examples/rollup install runs, it uses a lot of dependencies already present in the ~/.npm, so the total install time is not terrible at all. The examples/rollup npm install only takes 20 seconds!

2. We build the cypress-react-unit-test first and run npm pack to create cypress-react-unit-test-0.0.0-development.tgz in the workspace. This workspace is then passed to every examples/* job downstream. Every job then removes the root level node_modules folder and installs TGZ file AND its own dependencies:

ls -la ../..
echo ***Installing cypress-react-unit-test from root TGZ archive***
npm install -D ../../cypress-react-unit-test-0.0.0-development.tgz
echo ***Installing other dependencies***
npm install
echo ***rename root node_modules to avoid accidental dependencies***
mv ../../node_modules ../../no_modules

I perform the above steps to make sure every example project is actually using the dependencies it declares and does not accidentally find NPM packages from the root node_modules folder. This protects the end users from depending on my library that worked accidentally due to a stray undeclared but successfully loaded 3rd party NPM dependency.

The entire CI workflow is shown in the graph below. We install root dependencies, build the library, run basic and advanced component tests. If they pass we run all example jobs in parallel.

The circle.yml file makes good use of cypress-io/circleci-orb to set up all jobs without manually managing workspaces, caches, etc.

The large number of individual tests, and multiple full-scale application examples guarantee that every release of the library is thoroughly tested and will work inside most of the users' apps without a hitch.

Taken together, the unit tests are like a first line of defense, while full example apps are like big testing pyramids. They are separate, but work together to stop bugs from crawling into the library.

Bonus - external examples

Just to complete the picture, there is a number of external example repositories that use cypress-react-unit-test. I cloned a few popular projects and showed how one could use my library to write component tests. You can find these projects under GitHub topic cypress-react-unit-test-example.

The external examples are not tested immediately from the main repository, but they are upgraded automatically when a new version of cypress-react-unit-test comes out.

I have a trick up my sleeve with external repositories. Because a user looking at an example wants it to work and apply to their situation today, I have added version badges. These badges show the current version of cypress-react-unit-test listed in the package.json file in that repository.

Anyone looking at this list immediately can see if any example projects are behind the current released version.

Tip: use available-versions utility to see all published versions of an NPM package

npm i -g available-versions
vers cypress-react-unit-test
---------------------------------------------------------------------------
version                       age       dist-tag                    release
----------------------------  --------  --------------------------  --------
1.0.0                         3 years
...
4.0.0                         6 months
...
4.15.0                        15 days
4.16.0                        7 days    latest

Seems some of the external applications are behind and should be upgraded to the latest version of cypress-react-unit-test so we are sure they still works.

Tip: read Keep Examples Up To Date to learn how I keep hundreds of my GitHub repositories up-to-date automatically

These version badges in the Markdown table were created and can be updated using dependency-version-badge utility. You can insert such badge, or update an existing badge by running a command line this one:

npm i -D dependency-version-badge
npx update-badge cypress-react-unit-test --from bahmutov/emoji-search

These badges are updated nightly using a GitHub Workflow set to run on a schedule

.github/workflows/badges.yml

on:
  schedule:
    # update README badges every night
    # because we have a few badges that are linked
    # to the external repositories
    - cron: '0 3 * * *'
# run multiple "npx update-badge cypress-react-unit-test --from ..." commands

Related

• read Testing Trapezoid blog post
• browse slides for SDET meetup or watch the video to see me talk about testing crab