Some of my webpack app assets (images, workers) are not being loaded which leads to broken tests

[eugenet8k]

• Operating System: macOS Big Sur v11.2.1
• Node Version: 15.10.0
• NPM Version: 7.5.3
• webpack Version: 5.24.2
• karma-webpack Version: 5.0.0

Expected Behavior

I have a number of SVG, PDF, JS workers, and other files in my webpack app. Obviously, it works all good while running the app normally, but when I get to run my mocha test with karma-webpack some of my tests fail due to failures with loading those mentioned asset-like files

Code

  // webpack.config.js

      {
        test: /\.(png|svg)$/,
        include: [resolve(__dirname, 'app', 'assets')],
        type: 'asset/resource',
        generator: {
          filename: 'images/[name][ext]',
        },
      },

So when I run via command line I see errors like:

404: /absolute/var/folders/qm/dx__qpt10rnd8rg2st400l200000gp/T/_karma_webpack_117318/f2462e1d0c87e9b2fd42.worker.js
404: /absolute/var/folders/qm/dx__qpt10rnd8rg2st400l200000gp/T/_karma_webpack_117318/images/logo-black-trademark.svg

I run it using browser to see what's going on in Dev Tools, and see the same results:

So when I got curious to see what's in the _karma_webpack_117318 folder, where I assumed it keeps my webpack assets. I found it in my macOS and indeed files are there:

So these files commons.js and runtime.js are loaded just fine, but not my files. This makes me think that when the browser does GET http://localhost:9876/absolute/var/folders/qm/dx__qpt10rnd8rg2st400l200000gp/T/_karma_webpack_117318/runtime.js it does not directly point to the disk folder. I made a test, by adding few lines of code into runtime.js to see if it will show up in the browser. It didn't. So I guess there is another memory cache or storage is used to serve the browser.

Maybe the state of _karma_webpack_117318 folder was cached before webpack finished the build, so my assets weren't cached in time and hence aren't served to browser?

The general question is... Is this a karma-webpack bug? Or did I miss some config?

[codymikol]

Can you post your karma-webpack config?

[eugenet8k]

@codymikol it is shared between the app itself and karma and it seems fairly generic despite a bunch of loaders:

const _ = require('lodash')
const webpack = require('webpack')
const {resolve} = require('path')
const MiniCssExtractPlugin = require('mini-css-extract-plugin')
const rupture = require('rupture')
const {getIfUtils} = require('webpack-config-utils')
const queryString = require('querystring')

const {ifProduction} = getIfUtils(process.env.NODE_ENV)
const cypressOptions = {
  CYPRESS: process.env.CYPRESS || false,
}
const cypressIfDefQuery = queryString.encode(cypressOptions)

module.exports = {
  devtool: 'inline-source-map',
  mode: 'development',
  module: {
    rules: [
      {
        test: /\.coffee|.js$/,
        exclude: /node_modules/,
        loader: 'babel-loader',
        options: {cacheDirectory: true},
      },
      {
        test: /\.coffee$/,
        exclude: /node_modules/,
        loader: 'coffee-loader',
      },
      {
        test: /\.hbs$/,
        loader: 'handlebars-loader',
        exclude: /node_modules/,
        options: {
          helperDirs: [
            resolve(__dirname, 'app', 'views', 'templates', 'helpers'),
          ],
          partialDirs: [
            resolve(__dirname, 'app', 'views', 'templates', 'partials'),
          ],
        },
      },
      {
        test: /\.yml$/,
        use: ['json-loader', 'yaml2json-loader'],
      },
      {
        test: require.resolve('jquery'),
        loader: 'expose-loader',
        options: {
          exposes: ['$', 'jQuery'],
        },
      },
      {
        test: /\.styl$/,
        exclude: /node_modules/,
        use: [
          MiniCssExtractPlugin.loader,
          'css-loader',
          {
            loader: 'stylus-loader',
            options: {stylusOptions: {use: rupture()}},
          },
        ],
      },
      {
        test: /\.less$/,
        exclude: /node_modules/,
        use: [MiniCssExtractPlugin.loader, 'css-loader', 'less-loader'],
      },
      {
        test: /\.css$/,
        use: [MiniCssExtractPlugin.loader, 'css-loader'],
      },
      {
        test: /\.(ico|png|svg|xml)$/,
        include: [resolve(__dirname, 'app', 'assets', 'favicons')],
        type: 'asset/resource',
        generator: {
          filename: 'favicons/[name][ext]',
        },
      },
      {
        test: /\.json$/,
        include: [resolve(__dirname, 'app', 'assets', 'favicons')],
        type: 'asset/resource',
        generator: {
          filename: 'favicons/[name][ext]',
        },
      },
      {
        test: /\.pdf$/,
        include: [resolve(__dirname, 'app', 'assets', 'documents')],
        type: 'asset/resource',
        generator: {
          filename: 'documents/[name][ext]',
        },
      },
      {
        test: /\.(png|svg)$/,
        include: [resolve(__dirname, 'app', 'assets')],
        type: 'asset/resource',
        generator: {
          filename: 'images/[name][ext]',
        },
      },
      {
        test: /\.(woff|woff2|ttf)$/,
        include: [resolve(__dirname, 'app', 'assets', 'fonts')],
        type: 'asset/resource',
        generator: {
          filename: ifProduction(
            'fonts/[name].[contenthash][ext]',
            'fonts/[name][ext]'
          ),
        },
      },
      {
        test: /index\.js$/,
        use: {
          loader: `ifdef-loader?${cypressIfDefQuery}`,
        },
      },
    ],
  },
  plugins: _.compact([
    new MiniCssExtractPlugin({
      filename: ifProduction(
        'stylesheets/[name].[contenthash].css',
        'stylesheets/[name].css'
      ),
    }),
    // Moment.js bundles large locale files by default due to how Webpack
    // interprets its code. This is a practical solution that requires the
    // user to opt into importing specific locales.
    // https://github.com/jmblog/how-to-optimize-momentjs-with-webpack
    new webpack.IgnorePlugin(/^\.\/locale$/, /moment$/),
    new webpack.ProvidePlugin({
      _: 'lodash',
      I18n: 'i18n-js',
      process: 'process/browser.js',
      Buffer: ['buffer', 'Buffer'],
    }),
  ]),
  resolve: {
    alias: {
      buffer: 'buffer',
    },
    extensions: ['.js', '.coffee', '.hbs', '.styl', '.less', '.css', '.svg'],
    modules: [resolve(__dirname, 'app'), 'node_modules'],
    symlinks: false,
    fallback: {
      stream: require.resolve('stream-browserify'),
    },
  },
  node: {__filename: true}, // is used in unit tests for full file name access
  target: 'web',
}

[codymikol]

Sorry, I mostly wanted to see your karma configuration. I'm wondering if the image files are not included in your files list, this would mean that they can't be associated to the webpack build.

[eugenet8k]

@codymikol oh no worries, at this point I am happy with any advice or direction to research as the issue is a blocker for our team. So here is the karma.conf.js:

const webpackDevConfig = require('./webpack.config.dev')

const isDebugMode = !!process.env.DEBUG || false
process.env.CHROME_BIN = require('puppeteer').executablePath()

module.exports = config =>
  config.set({
    browserDisconnectTimeout: 180000, // 3 min
    browserDisconnectTolerance: 1,
    browserNoActivityTimeout: 180000, // 3 min
    captureTimeout: 180000, // 3 min
    client: {mocha: {reporter: 'html', timeout: 10000}},
    files: ['app/index.spec.js'],
    frameworks: ['mocha', 'webpack'],
    plugins: [
      'karma-webpack',
      'karma-mocha',
      'karma-sourcemap-loader',
      'karma-chrome-launcher',
      'karma-coverage-istanbul-reporter',
    ],
    reporters: ['dots', 'coverage-istanbul'],
    preprocessors: {'app/index.spec.js': ['webpack', 'sourcemap']},
    singleRun: !isDebugMode,
    webpack: webpackDevConfig,
    browsers: isDebugMode ? ['Chrome'] : ['HeadlessChrome'],
    customLaunchers: {
      HeadlessChrome: {
        base: 'ChromeHeadless',
        flags: ['--no-sandbox', '--headless'],
      },
    },
  })

where ./webpack.config.dev is the file I posted earlier.

Our app/index.spec.js is

const testContext = require.context('../app/', true, /\.spec\.(coffee|js)$/)
testContext.keys().forEach(testContext)

I wonder if the problem is that we use this approach? The index.spec.js is shared between karma test runner (for command-line runs in CI) and the regular mocha runner using webpack mocha-loader (for running tests in browser during development).

[eugenet8k]

@codymikol I used http://karma-runner.github.io/latest/config/files.html as a reference and included my assets into files

        files: [
          'app/index.spec.js',
          {
            pattern: 'app/assets/**/*',
            watched: false,
            included: false,
            served: true,
          },
        ],

So now I can access my images with URL like

http://localhost:9876/base/app/assets/images/logo-black-trademark.svg

No more 404 for these URLs, which is great! But my problem is still not solved. When I debug the tests I see that URLs like

http://localhost:9876/absolute/var/folders/qm/dx__qpt10rnd8rg2st400l200000gp/T/_karma_webpack_481341/images/logo-black-trademark.svg

are being used to load my assets, but I need this URL

http://localhost:9876/base/app/assets/images/logo-black-trademark.svg

I am not sure if I can overcome this with karma proxies setting, as it seems like this repo instructs webpack to use the os temp dir URL (as per README.md):

output: {
    filename: '[name].js',
    path: path.join(os.tmpdir(), '_karma_webpack_') + Math.floor(Math.random() * 1000000),
  },

[eugenet8k]

Trying all the wild ideas like generating the path to tmpdir ahead of time and passing it to karma-webpack. So I can use the path to configure proxies

const output = {
  path: join(tmpdir(), '_karma_webpack_') + Math.floor(Math.random() * 1000000),
}

module.exports = config =>
  config.set({
    ....
    webpack: {... webpackDevConfig, output},
    proxies: {
      [`http://localhost:9876/absolute/${webpackOutput.path}/images/`]: 'http://localhost:9876/base/app/assets/images/',
    }
  })

but it still does not work. I don't think proxies should be used like this?...

[eugenet8k]

Oh man, I tried the wildest assumption and it worked out, which is great and also bad because it looks so dirty. So taking my latest approach by owning the webpack {output: {path}} parameter, essentially using the same expression as if karma-webpack would do. I just pass that path into karma files property:

const output = {
  path: join(tmpdir(), '_karma_webpack_') + Math.floor(Math.random() * 1000000),
}

module.exports = config =>
  config.set({
    ....
    webpack: {...webpackDevConfig, output},
    files: [
      {pattern: 'app/index.spec.js'},
      {
        pattern: `${output.path}/**/*`,
        watched: false,
        included: false,
      },
    ],
  })
        

it warns me in the console but it works!

03 03 2021 12:33:18.113:WARN [filelist]: All files matched by "/var/folders/qm/dx__qpt10rnd8rg2st400l200000gp/T/_karma_webpack_852561/**/*" were excluded or matched by prior matchers.

@codymikol this seems like something could be dealt with at the default behavior of this repo. So I guess my bug request is valid. Would it be possible to ensure that by default karma files config get the path to karma-webpack tmp dir where all webpack assets are being stored?

[codymikol]

The way karma-webpack currently works is that it maps the bundled files created by the webpack build process back to the files specified in the files property of the karma configuration.

At this time I don't believe there is a way to modify the original list of files (specified in files) that karma is aware of (passed in the karma configuration).

What I'd like to work on for a v6 of this plugin is to make the initial webpack step be the provider of the files to karma. This will require some kind of hook in karma that allows files to be defined by a plugin rather than explicitly. I saw this was in the works karma-runner/karma#3638 , but I haven't had much time to look it over and I'm not entirely sure that will make this possible.

Hopefully I'll have some time this week and I can see if there is some simple workaround for making assets work that won't require pointing back to the temporary build.

[jljorgenson18]

I'm running into this as well and it's pretty much completely derailing a general upgrade to Webpack v5. Is there anything I can do to assist? In our case, we have to fetch certain files from the Karma server and we are getting 404's back. I had thought that it was maybe related to removing webpack-dev-middleware but I wasn't sure.

[eugenet8k]

@jljorgenson18 have you tried this approach? #498 (comment)

[jljorgenson18]

@eugenet8k, yep it did not work.

[eugenet8k]

@jljorgenson18 well, you gotta share your config files (karma.conf.js, and probably webpack). That's the essential minimum to guess what's going on.

[jljorgenson18]

O scratch that @eugenet8k , I did just try your solution and it DID work. I had a misconfiguration where the publicPath was temporarily off. This is actually working.

[pimterry]

Quick note on this: this currently breaks all usage of wasm + webpack v5 + karma.

Wasm files get included bundled automatically by webpack nowadays, and get copied to the output directory (just like the images discussed here) but karma refuses to serve them with a 404, so the import reliably fails & no tests can run.

The workaround above (pass the webpack output path into to files) works for me too, but it throws an annoying warning before the build (Pattern "/.../*.wasm" does not match any file. and it's definitely messy.

My project is actually a testing library often used with karma & webpack downstream, so I'm going to have to pass this workaround onwards to all my users too. It'd be amazing if the karma-webpack output files could all be added to files automatically 👍👍👍

[danrice92]

Wow, thank you for your work on this! I ran into this same issue upgrading to Webpack 5 using Karma, because my web worker started 404'ing randomly.

For others reading this down the road, to do what @eugenet8k did in this post, I needed to import os and path. So my custom output looked like this:

const os = require('os');
const path = require('path');

const output = {
  path: path.join(os.tmpdir(), '_karma_webpack_') + Math.floor(Math.random() * 1000000),
}

I think os is always available, but path is an npm package? Not sure.

Also, I needed to put the output constant in my webpack config, like so:

module.exports = function(config) {
  config.set({
    // other config stuff
    webpack: Object.assign({}, webpackConfig, {
      output,
      optimization: {
        runtimeChunk: false, // if you have this to get source maps working, DELETE IT, or you will get 404's again
      }
      // other webpack config stuff
    })
  });
};

[ilgonmic]

@eugenet8k Thank you for sharing your solution. It helped me.
I went further in investigation and now I use Karma plugin to get webpack output and put it into files array.
So maybe it can be useful

function KarmaWebpackOutputFramework(config) {
    // This controller is instantiated and set during the preprocessor phase.
    const controller = config.__karmaWebpackController;

    // only if webpack has instantiated its controller
    if (!controller) {
        console.warn(
            "Webpack has not instantiated controller yet.\n" +
            "Check if you have enabled webpack preprocessor and framework before this framework"
        )
        return
    }

    config.files.push({
        pattern: `${controller.outputPath}/**/*`,
        included: false,
        served: true,
        watched: false
    })
}

const KarmaWebpackOutputPlugin = {
    'framework:webpack-output': ['factory', KarmaWebpackOutputFramework],
};

config.plugins.push(KarmaWebpackOutputPlugin);
config.frameworks.push("webpack-output");