Update Dependencies Using The Wisdom of The Crowds

Renovate dependency upgrades become even better by showing the upgrade stats collected across all projects.

Many many moons ago in 2013 I have released next-update CLI tool and described it in the Really painless modular development blog post. The next-update automates dependency updates by installing each dependency one by one, running the project's tests, and if they still pass, keeping it. While this was exciting, there was one more trick up my sleeve - I had a companion service running online that collected anonymous upgrade stats. It kept just a tally of update pass/fail counts per NPM module. Something like this table for every dependency:

Dependency table

When next-update wanted to check if dependency X could be updated safely from version X to version Y it queried this online service and showed the stats to the user:

Update statistics from next-update

Great - across all projects, updating check-more-types from version 1.1.1 to version 1.7.3 was 74% successful without any user input. This number was computed from 31 projects that used next-update. On the other hand, updating check-types from 1.4.0 to version 3.3.0 was pretty much a guaranteed failure - none of the 26 projects that tried it were successful.

I loved this information - I strongly believe the wisdom of the crowds across the NPM users finds all possible edge cases.

Now I mostly just use Renovate App service to keep all my dependencies up to date. I have more than one hundred GitHub repos updated this way - and I am not afraid to let the bot auto merge updates if the tests pass.

Last week Renovate became even better. It now collects the anonymous update stats and in every pull request shows the information about other projects across the internet that updated the same dependency. Read the official announcement here. The feature is called "Merge Confidence" and every pull request shows the following:

Merge confidence in Prettier v1 to v2 update

As you can see, updating Prettier from v1 to v2 will probably require manual work - the automattic updates mostly failed for this breaking change.

On the other hand, upgrading Cypress from v5 to v6 is pretty smooth - almost 80% of projects did the update without breaking tests.

Merge confidence in Cypress v5 to v6 update

Most projects could upgrade automatically but have not merged the Renovate pull requests yet - thus the confidence is still low. The confidence will increase once more projects that successfully pass tests actually merge the dependency update. Here is an example of a patch dependency update with lots of projects passing their tests, and thus the wisdom of the crowds marking "high confidence".

Merge confidence in Cypress Code Coverage update

By using these additional statistics you can update your dependencies with confidence. You also no longer have to rely on just your own tests, and you do not have to blindly trust the dependency authors to be diligent about semantic versioning - because you know how your own project and all other projects behave during upgrade.

See Merge Confidence documentation page for details.

👏 Rhys Atkins and WhiteSource Renovate Team.