NPM Install with just GitHub

I have discussed how to Use GitHub instead of NPM to share and install NPM modules. But that blog post had a huge shortcoming - it did not deal with built or transpiled artifacts. More and more NPM packages are now transpiled from TypeScript for example, and thus usually have two folders: lib and dist. The GitHub repository only has the lib folder with the original source. The built dist folder is the one published to the NPM registry.

If we just install an NPM package from GitHub we will not get the dist folder - it is not on GitHub. We could try building it ourselves ... but that is tricky, since the project might be a pain to build.

Today I had just this problem - I needed to quickly patch @percy/cypress NPM module to get around Webpack bundling problem there. I have opened a pull request with my work around from my fork bahmutov/percy-cypress branch find-percy-58.

Meanwhile I wanted to use my patched version in bahmutov/calculator project. I could simply install my fork using github: reference

npm i -D github:bahmutov/percy-cypress

But this installation installs ... wrong files

$ ls -l node_modules/\@percy/cypress/
total 80
-rw-r--r--  1 gleb  wheel  1072 Oct 26  1985 LICENSE
-rw-r--r--  1 gleb  wheel   420 Oct 26  1985 README.md
-rw-r--r--  1 gleb  wheel   477 Oct 26  1985 appveyor.yml
-rw-r--r--  1 gleb  wheel   627 Oct 26  1985 dependencies.yml
-rw-r--r--  1 gleb  wheel  1878 Apr  1 22:59 package.json
-rwxr-xr-x  1 gleb  wheel   255 Oct 26  1985 release.sh
-rw-r--r--  1 gleb  wheel  1875 Oct 26  1985 run-tests.js
-rw-r--r--  1 gleb  wheel  5329 Oct 26  1985 tsconfig.json
-rw-r--r--  1 gleb  wheel   197 Oct 26  1985 tslint.json

I need to install the built module. So here is how to do this myself.

First, add the files: [] property to the package.json. It is more flexible than just .npmignore since when using files you can whitelist or blacklist files and folders. In our case, we want to only distribute the dist folder with the NPM package. Compare the files before and after.

before adding files property

$ npm pack --dry
npm notice
npm notice 📦  @percy/[email protected]
npm notice === Tarball Contents ===
npm notice 1.2kB package.json
npm notice 477B  appveyor.yml
npm notice 627B  dependencies.yml
npm notice 1.1kB LICENSE
npm notice 3.5kB percy-cypress-1.0.5.tgz
npm notice 420B  README.md
npm notice 255B  release.sh
npm notice 1.9kB run-tests.js
npm notice 5.3kB tsconfig.json
npm notice 197B  tslint.json
npm notice 507B  .circleci/config.yml
npm notice 97B   dist/environment.d.ts
npm notice 516B  dist/environment.js
npm notice 11B   dist/index.d.ts
npm notice 3.6kB dist/index.js
npm notice 775B  dist/percy-healthcheck
npm notice === Tarball Details ===
npm notice name:          @percy/cypress
npm notice version:       1.0.5
npm notice filename:      percy-cypress-1.0.5.tgz
npm notice package size:  11.2 kB
npm notice unpacked size: 20.4 kB
npm notice shasum:        3f195fc4d1e93032d14e88e4be885c0ceee4387e
npm notice integrity:     sha512-PoLpQJbOCd+2p[...]9rS2kGW/MQv0A==
npm notice total files:   16
npm notice
percy-cypress-1.0.5.tgz

Lots of stray files - no user of this NPM package needs tslint.json for example. After adding files: ["dist"] we get just the necessary file list.

$ npm pack --dry
npm notice
npm notice 📦  @percy/[email protected]
npm notice === Tarball Contents ===
npm notice 1.2kB package.json
npm notice 1.1kB LICENSE
npm notice 420B  README.md
npm notice 97B   dist/environment.d.ts
npm notice 516B  dist/environment.js
npm notice 11B   dist/index.d.ts
npm notice 3.6kB dist/index.js
npm notice 775B  dist/percy-healthcheck
npm notice === Tarball Details ===
npm notice name:          @percy/cypress
npm notice version:       1.0.5
npm notice filename:      percy-cypress-1.0.5.tgz
npm notice package size:  3.5 kB
npm notice unpacked size: 7.7 kB
npm notice shasum:        1afc7e5117dfda71fbb32d3326a7145faba052af
npm notice integrity:     sha512-NA75z/sUCpZLR[...]gmy9ZyI3rQo8Q==
npm notice total files:   8
npm notice
percy-cypress-1.0.5.tgz

Super. Now, push the code (using a separate branch for example) to GitHub. Start a new release there.

When you make a GitHub release you can drop binary files there.

To prepare a binary file, run npm pack command (without --dry option). This will give you a tgz file like percy-cypress-1.0.5.tgz. Drop this file into the GitHub release. Here is the fix-webpack I have made.

Right click on the percy-cypress-1.0.5.tgz file to grab the download url, in this case it will be https://github.com/bahmutov/percy-cypress/releases/download/fix-webpack/percy-cypress-1.0.5.tgz. You can npm install that URL!

So I went to my example project bahmutov/calculator and ran

$ npm i -D https://github.com/bahmutov/percy-cypress/releases/download/fix-webpack/percy-cypress-1.0.5.tgz

The installed NPM package is beautiful; it just has what is necessary.

$ ls -l node_modules/\@percy/cypress/
total 24
-rw-r--r--  1 gleb  staff  1072 Oct 26  1985 LICENSE
-rw-r--r--  1 gleb  staff   420 Oct 26  1985 README.md
drwxr-xr-x  7 gleb  staff   238 Apr  1 22:23 dist
-rw-r--r--  1 gleb  staff  2497 Apr  1 22:23 package.json

The package.json keeps the long original download url. Thus this installation works on CI server the same way.

Everything is peachy: I have my fork, and I can install it from other NPM projects.