[workbox-webpack-plugin] Apply manifestTransforms to Webpack assets

[ciampo]

Currently the dontCacheBustUrlsMatching, modifyUrlPrefix and manifestTransforms options do not seem to have any effect on the precache manifest generated off Webpack's compilation assets.

I'm generating some assets through Webpack that need to be served at the root of the website. Let's take favicon.ico as an example: the plugin precaches favicon.ico at the url /static/favicons/favicon.ico, but i'm serving this asset at \favicon.ico. The precached URL doesn't match with the request, and returns the fallback URL instead.

I would normally solve the problem with the modifyUrlPrefix or the manifestTransforms option, but they don't seem to apply in this case.

I believe this is quite a limitation for the Webpack plugin, is there any chance these transformations can be applied to all pracached assets, including the ones coming from Webpack?

Library Affected:
workbox-webpack-plugin, workbox-build

Browser & Platform:
N/A

Issue or Feature Request Description:
Issue

[jeffposnick]

You're correct that manifest transformations don't apply to webpack assets. (We've tried to be clearer about that in the docs so that this doesn't catch folks by surprise.)

While there are a lot of ways to use webpack, what we've found in general is that the URLs that webpack assigns to assets in manages tends to be the same URLs that can be used to access those assets once they're deployed to the web. Configuration settings that control that behavior, like publicPath, are already honored by workbox-webpack-plugin. If you need to completely transform all of the URLs that Workbox knows about, and you aren't already applying those transformations via the webpack configuration, I'd like to learn more about that scenario.

For instance, you mention that you're serving a file that lives at /static/favicons/favicon.ico as
/favicon.ico. Are you doing that by manually configuring your web server to respond to /favicon.ico by looking for a file that lives at another URL? Doesn't webpack need to know about that mapping so that its loaders could replace the reference to the original file with the final URL?

(favicon.ico isn't the best example, by the way, as the browser normally won't trigger a fetch event when requesting a web site's favicon, so there's no need to cache it.)

[ciampo]

I see your point about trusting Webpack's configuration, and I appreciate that the favicon example may not be the best one.

I spent some time in my project refactoring the values around publicPath and the handlers in my server configuration, but I can still find small problems that could be easily fixed with (a combination of)dontCacheBustUrlsMatching, modifyUrlPrefix and manifestTransforms.

For example, the Webpack plugin that I use to generate the favicon files and the Webapp Manifest needs a prefix option in its configuration. Because I want to store these assets in the root of my static files directory, I have to specify \ as the prefix to all filenames for this plugin.
My publicPath is also \, and therefore the precache manifest ends up storing URLs beginning with a double slash: \\[filename...]. These URLs get interpreted as a https:\\[filename] request by the browser and fail.

In short: I agree that 99% of the times Workbox can just rely on Webpack's compilation output, but there can be edge cases where little tweaks are necessary.
I believe that allowing the user to apply manifestTransforms also on the entries generated by Webpack would be of great help in order to solve certain issues and give more flexibility to its users.

[jeffposnick]

My publicPath is also , and therefore the precache manifest ends up storing URLs beginning with a double slash: \[filename...]. These URLs get interpreted as a https:\[filename] request by the browser and fail.

You say that those URLs end up in the precache manifest with a \\ prefix, but don't they also end up in the webpack asset pipeline with that prefix as well? Doesn't that already break your usage of those assets from within your HTML? Or are you not using a loader to translate those assets URLs, and instead hardcoding the "correct" values?

We can leave this open to track the general request to allow you to manipulate the precache manifest generated from your webpack assets in a future release of Workbox. I'd like to put some thought into what a properly API for that would look like, though, as I don't think you can just take whatever is currently being supplied as the manifestTransforms array and also apply it to the webpack precache manifest. There's a fundamental difference between the expectations a developer should have around the URLs in the webpack precache manifest (it's going to be a set of URLs that adhere to webpack's conventions) vs. the URLs in a glob-derived precache manifest (it's going to be a set of URLs that adhere to the filesystem's layout).

Here's a suggestion for now: don't include the "problematic" URLs in your webpack-generated precache manifest, and instead pick them up by setting something like globPatterns: ['**/favicon.ico'] in your Workbox config. You can then control how those URLs are modifying via modifyUrlPrefix, etc.

[jeffposnick]

This is also tracked in #1227

[carlosbensant]

I've been dealing with the same issue.

precache-manifest.[hash].js file:

self.__precacheManifest = [
  {
    "revision": "4faffe2c7059041f8cef",
    "url": "/css/front.min.css"
  },
  {
    "revision": "4faffe2c7059041f8cef",
    "url": "/css/app.min.css"
  },
  {
    "revision": "b4330fae6023d56a34ef",
    "url": "//js/front.min.js"
  },
  {
    "revision": "4faffe2c7059041f8cef",
    "url": "//js/app.min.js"
  }
];

URLs beginning with double slash only happen with JS files.

[carlosbensant]

You say that those URLs end up in the precache manifest with a \\ prefix, but don't they also end up in the webpack asset pipeline with that prefix as well? Doesn't that already break your usage of those assets from within your HTML? Or are you not using a loader to translate those assets URLs, and instead hardcoding the "correct" values?

Yes, after seeing the webpack asset pipeline I figure out they end up with that prefix as well. I don't know why this only happens with JS files, I'm looking for a way to remove them.

[clydegrey]

@carlosbensant did you end up resolving this? I recently ran into this and am looking for a solution.

[carlosbensant]

Yes, of course. I had to install a library to replace those prefixes in every autogenerated JS file.

const replace = require('replace-in-file')

({ WebpackConfig }).then( () => replace.sync( {
        // FIXME:   Workaround for laravel-mix placeing '//*.js' at the begining of JS filesystem
        files: path.normalize( `${__dirname}/public/*.js` ),
        // precache-manifest.d633c0ce0867dac679ce469c8f1f8fde.js
        from:  /"\/\//gu,
        to:    '"/'
    } ) )

[pmochine]

What worked for me is just to set the publicPath to nothing:

output: {
    publicPath: '', // important for service worker to remove the slash!
    chunkFilename: 'js/[name].[chunkhash].js',
},

[pmochine]

You can finde the problem here as well: laravel-mix/laravel-mix#1717