Testing with es6 modules

[LarsDenBakker]

Most people author their code as es modules now, and many browsers can load them natively.

I'd like to write my tests as es modules as well, and just run them out of the box with karma without any code transformations and complex tooling involved. Unfortunately this isn't supported by karma by default.

If my configuration says:

files: [
  { pattern: './test/*.test.js', type: 'module' },
]

I expect it to run all my tests. It will pick up my test files, but any subsequent imports will 404 because 'karma doesn't know about them'. So instead I need to add wildcards:

files: [
  { pattern: './test/*.test.js', type: 'module' },
  { pattern: '**/*.js', included: false },
  { pattern: 'node_modules/**/*.js', included: false },
]

this works somewhat, but it takes much longer to startup. it seems like karma is indexing all the files for some reason?

Can we add an option where we can just rely on the native browser's module resolving, and serve any file requested relative to the module which requested it?

[johnjbarton]

(I doubt your assertion about ‘most people’, but wide support on modern browser is enough justification, https://caniuse.com/#feat=es6-module).

I can’t tell the root of your problem based on this information. Can you create a small github project to reproduce it? You may also learn more about your issue by setting —logLevel=DEBUG

[johnjbarton]

Here is the configuration used to test the module feature in karma:
https://github.com/karma-runner/karma/blob/e811adb9818f4046ea10e3bd4ca3d1388909bb56/test/e2e/module-types.feature
It looks to me like your configuration includes only tests and not code-under-test?

[LarsDenBakker]

Thanks for your answer. I've put together a project which demonstrates the problem:

https://github.com/LarsDenBakker/karma-modules-issue

If you clone the repository and run npm install. Then there are three commands:

npm run test runs the tests how I would expect to use karma, I just tell karma where my test files are. When you run them and enter the debug page you will see a 404 request for /base/src/a.js.

npm run test:include-src includes the project source files with include: false. now we get a 404 for /base/node_modules/lodash-es/lodash.js`

npm run test:include-all includes all node_modules under test

With the final command the test runs, but it's still not what I'm looking for. It seems like karma is now indexing all my node_modules and runs the preprocessor on any javascript file within my node_modules folder. I added a simple preprocessor which just logs the file being processed.

In an ideal world I'd just tell karma where my tests are, and you can just import other modules from there onwards. The preprocessors could then run on demand for each file served.

Plugins like karma-webpack or karma-browserify work around these problems by just resolving all imports beforehands and bundling the tests. This seems very uneccessary to me, and also leads to bugs as at least in the case of webpack it's loading the same dependencies multiple times.

I respect that karma has been around for a long time and that bundling was (is?) the primary way to ship code in the browser, but with module supports in all major browsers I'm looking forward to making this much simpler for developers to use.

[johnjbarton]

I'm looking forward to making this much simpler for developers to use.

And I am looking forward to your karma plugin to implement this great feature!

[devoto13]

The reason why Karma needs to index all files is because it supports plugins and preprocessors, which may need to modify some of these files. Besides this to be able to traverse imports in the source code Karma will need to parse this code and maintain up to date dependency graph, which is not a trivial task in a generic case. IMO the requested feature will be hard to implement without a significant rework to the Karma architecture. And may even not be very interesting given that karma-webpack and karma-browserify do most of this already and don't result in a much faster approach.

As a workaround I would suggest to specify only necessary dependencies in the files array, which should reduce the amount of processed files significantly:

• your include-all configuration loads ~7.5k files and takes ~1.5s to process
• { pattern: 'node_modules/lodash-es/**/*', type: 'module', included: false }, results in ~650 files and takes ~100ms to process (vs 30ms of the include-src configuration)

It may also be possible to do some optimisations in the Karma code to speed up the processing. I did some basic profiling and noticed one low-hanging fruit - short-circuiting preprocessor logic, when there are no preprocessors configured cuts processing time by ~50ms in the updated include-all, so with this change processing takes ~50ms. Here is the code used in my profiling experiments.

[LarsDenBakker]

I actually have a plugin to do this: https://www.npmjs.com/package/@open-wc/karma-esm but it's basically a workaround as I have to add a custom middleware to bypass karma's loading system and thus it doesn't work with preprocessors. My hope is that this can be properly supported by karma in some way, hence I'm posting here.

Manually maintaining a dependency graph in the karma configuration doesn't seem like a very scalable solution.

The preprocessors already run on a per-file basis, so they could also run when the modules are requested by the browser I think? If you do need to scan your entire codebase upfront you can still use existing plugins and preprocessors.

But even when bundling your code before running your tests there are issues with existing plugins when files are dynamically generated. For example when your code contains dynamic imports most bundlers split your application into multiple dynamically generated chunks. These chunks can't be requested at runtime, because karma didn't know about them beforehand. See for example: jlmakes/karma-rollup-preprocessor#48.

[johnjbarton]

How would karma’s middleware be different from the middleware in karma-esm?

[LarsDenBakker]

The difference is that people need to find the plugin, which is a shame if you just want to write a simple test using standard browser behavior. It also doesn't participate in the karma plugin ecosystem, like preprocessors.