Use GitHub instead of NPM

Let's say you have some JavaScript code you want to share among several projects, but you don't want to create a private NPM package yet. Maybe you are just experimenting, and setting things up takes effort. Why bother if it might not work out? Here is how you can quickly push code to a private GitHub repository yet make it available to other projects.

Private repo

I made a private repository bahmutov/private-module-example on GitHub. It contains a small JavaScript export just for show.

// src/index.js
console.log('this module will export stuff from "foo"')

module.exports = {
  foo: require('./foo')
}
// src/foo.js
module.exports = 'foo'

I can load this module locally from the project's root folder

$ node
> require('.')
this module will export stuff from "foo"
{ foo: 'foo' }
> .exit

The package.json sets the private: true to avoid accidentally publishing this package to the NPM registry.

package.json

{
  "name": "private-module-example",
  "version": "1.0.0",
  "description": "Private module installation example",
  "main": "src",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "private": true
}

I pushed the code to the remote origin

$ git push
Counting objects: 3, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 361 bytes | 361.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0)
To github.com:bahmutov/private-module-example.git
   b8613b3..a8a40d7  master -> master

Then I created a tag (same as version) and pushed it too

$ git tag 1.0.0
$ git push --tag
Total 0 (delta 0), reused 0 (delta 0)
To github.com:bahmutov/private-module-example.git
 * [new tag]         1.0.0 -> 1.0.0

Great, I have 1 release in my private GitHub repository.

Using GitHub repository

I have created another private GitHub repository bahmutov/private-module-example-user - this repo will install the code from the first repository without going to NPM.

package.json

{
  "name": "private-module-example-user",
  "version": "1.0.0",
  "description": "Private repo that uses another private repo as an NPM module",
  "main": "index.js",
  "private": true,
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  }
}

The install command looks almost the same as "standard" npm i <package name@version>. Only instead of the package name, I can specify GitHub username and repository name, instead of the version, I can specify a commit SHA or a tag. I prefer tags.

$ npm install -S bahmutov/private-module-example#1.0.0
+ [email protected]
added 1 package from 1 contributor and audited 1 package in 6.961s
found 0 vulnerabilities

My package.json reflects the installed dependency

package.json

{
  "name": "private-module-example-user",
  "version": "1.0.0",
  "description": "Private repo that uses another private repo as an NPM module",
  "main": "index.js",
  "private": true,
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "dependencies": {
    "private-module-example": "github:bahmutov/private-module-example#1.0.0"
  }
}

Great, but does it work? Let's open Node and load the dependency

$ node
> require('private-module-example')
this module will export stuff from "foo"
{ foo: 'foo' }

It is working.

Continuous Integration setup

I will set up continuous integration (CI) server to run "tests" on CircleCI. Here is my .circleci/config.yml file

.circleci/config.yml

version: 2
jobs:
  build:
    docker:
      - image: circleci/node:10

    working_directory: ~/repo

    steps:
      - checkout

      # Download and cache dependencies
      - restore_cache:
          keys:
          - v1-dependencies-{{ checksum "package.json" }}
          # fallback to using the latest cache if no exact match is found
          - v1-dependencies-

      - run: npm install

      - save_cache:
          paths:
            - node_modules
          key: v1-dependencies-{{ checksum "package.json" }}

      - run: npm test

And my test script will just load the private-module-example module. If the module has not been installed, the npm test would crash and burn.

package.json

{
  "scripts": {
    "test": "node -e \"console.log(require('private-module-example'))\""
  }
}

$ npm t

> [email protected] test /private-module-example-user
> node -e "console.log(require('private-module-example'))"

this module will export stuff from "foo"
{ foo: 'foo' }

Ok, push code to CircleCI and ... see if fail

#!/bin/bash -eo pipefail
npm install
npm ERR! Error while executing:
npm ERR! /usr/bin/git ls-remote -h -t git://github.com/bahmutov/private-module-example.git
npm ERR!
npm ERR! fatal: remote error:
npm ERR!   Repository not found.
npm ERR!
npm ERR! exited with error code: 128

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/circleci/.npm/_logs/2018-10-23T17_18_14_644Z-debug.log
Exited with code 1

When Circle connects the new project to the GitHub repository it created an SSH key restricted to that repository. Thus the same key cannot be used to clone another private repository. We need to change this. Go to the project's Settings / Checkout SSH keys and click the button twice.

Then

Now the build should be able to access clone NPM package from the private repository into this project.

Iterate

Now you can iterate on your first module, and when there are new features or fixes, increment package.json version (I suggest using next-ver to compute the next version based on commit messages), tag the commit and push the code and tag to GitHub. Then you can point the user project at the new tag, and you are good to go. This avoids private NPM registry, but of course this adds complexity to the CI with the user checkout key. On NPM you would need to use NPM_TOKEN to authenticate and install your own private modules (and of course pay for private scope).

Related

I use same approach if I need to fix 3rd party NPM module, read Fixing the Internet one NPM package at a time