Keeping the same load order of files and tests can lead to hidden bugs.
Imagine a typical project with bunch of unit tests. Usually there is a configuration file that lists all source files to be loaded, and then the second list of test files. It could be a Gruntfile.js, karma.conf.js or something else.
We often keep adding new source files at the end of the list of files to be loaded before tests, or we keep the list sorted alphabetical by using a wildcard
// alphabetical order
Some frameworks allow you to scramble the test execution order, but not the file load order. For example, QUnit allows to execute previously failed tests first Other frameworks, like Mocha allows you to sort test files before loading instead. None of the frameworks can detect the following situation (although global leak detection helps, but does not solve this).
In production we concatenate files alphabetically using a wildcard and everything works. We also list files to load for testing using wildcards
Problem: we will never discover that we forgot to initialize
foo object in file
something.js. We could turn on global leak detection, but this will give us false
positive - we need
This problem can partially be attributes to the Engish language and a programmer's
tendency to name the files to be included first starting with suffixes that
lead alphabetically, like
Take each group of the files to be loaded separately (source, test files). Instead of using wildcards directly, grab all matching filenames and scramble them! If using grunt you can call grunt.file.expand to get all filenames matching a pattern. Then randomize its order using shuffle:
This will lead to flaky tests of course. But chances are if you dig into each failing test and print the files before running the tests, you will get to the true reason: some files are meant to load first. And then you will use require.js or browserify.