Note: specifying publicPath in prod can break django-whitenoise / ManifestStaticFilesStorage

[mik3y]

First, thanks for this project and your accompanying blog!

I am using django-whitenoise with this, which is a derivative of Django's ManifestStaticFilesStorage storage backend that adds forever-cacheable headers. All worked well except I noticed for some reason my bundle asset (let's call it $STATIC/js/main.js) was not being served correctly, unlike all other static assets.

In general, in this kind of setup, collectstatic transforms files by adding a content hash for the file name, for example js/main.js might be copied to $STATIC_ROOT/js/main.536ce6ddaf7b.js. The hashes are transparently added by the staticfiles backend, so a URL like {% static 'js/main.js' %} should automatically be translated to the hashed URL when served/rendered. For some reason, render_bundle did not do this.

It turns out there are two unrelated behaviors that are at play here:

1. collectstatic copies both the original resource and its hashed filename to the destination. So both js/main.js and js/main.536ce6ddaf7b.js can be served by the application — which in my case masked the fact that the hashed file was never being served (i.e. nothing in the app breaks, it just doesn't get the nice caching features).

2. When django-webpack-loader is called to render a bundle, it determines the URL in one of two ways, depending on the webpack-stats.json metadata for the chunk:

   • If the chunk has a publicPath, return that as the URL exactly.
   • Otherwise, call staticfiles_storage.url() for $BUNDLE_DIR_NAME/$CHUNK.

Because my original webpack.config.prod.js had a line like like this:

config.output.publicPath = '/static/js/';

... my calls to render_bundle would take that first branch & never call staticfiles_storage.url, returning the "plain" unhashed asset. The fix is simple: remove thepublicPath config.

In summary: Don't set output.publicPath in prod unless you know what you're doing; webpack loader will not consult your staticfiles backend if you do.

(PS: I'm guessing this isn't actionable, my apologies if this obvious to more webpack-savvy folks. I'm pretty unfamiliar with webpack/django guts, so I figured I'd splat some notes here for searchability if nothing else, in case another noob trips up like I did. cheers!)

[dangmai]

I'm running into this issue as well, only that I can't work around it by taking out publicPath. My Webpack set up use the code splitting feature, which points to the wrong directory without the publicPath. Is there anything else that we can do here?

Also, at this point, I'd be happy if I can get the resources to be loaded correctly. You mention that collectstatic copies both the original resource & the hashed one to the dest, but on mine it only copies the hashed ones. Do you know if there's a config somewhere to enable that? I've tried Googling but nothing has come up so far.

[dangmai]

I put up a possible solution on my own branch here. Basically I added a new config RESOLVE_PUBLIC_PATH that allows users to choose whether to let publicPath affects the URL determination or not. I'm pretty new to django-webpack-loader so I might have missed some cases where this approach might not be the best, if so let me know. If the maintainer thinks it's a good idea I'll open a PR for it.

[owais]

@mik3y Preferring publicPath over django's storage is a feature and the only way to give preference to django's storage right now is to omit publicPath from your webpack config like you correctly pointed out.

The reasoning behind this is that webpack is an independent system and we don't want to force users to couple it to django. Webpack already does everything that whitenoise or other django static collectors do and it does it quite well. So, there is no need for someone to use something like whitenoise in addition to webpack.

For such cases, users set the publicPath in webpack config to something like https://s3-us-west-2.amazonaws.com/my-bucket/static/ and then just upload all assets generated by webpack to my-bucket/static. Django will make it work without any integration with storages.

This is very powerful as your frontend team does not even have to deal with Django. They don't even have to run it if they want at any step in the build process. This way webpack and django can be completely decoupled.

However, in some cases, we want to use storages, and at that time it is recommended to not set publicPath in webpack config.

Does that make sense? If so, would you consider this as resolved if this was mentioned in the docs or an FAQ?

P.S. I've been planning a refactor that will let users override behavior by writing custom code and hook it up to the webpack loader instance. That change should make it possible to always prefer storage over publicURL in your project if you want.

[owais]

@mik3y Also, if you use a static file storage that is capable of doing hashing for long term caching, I would recommend to generate bundles from webpack always without content hashes in their names. Then process those assets/bundles using your static file storage (whitenoise, etc) and let them add the hashes. If you do this, you don't even need django-webpack-loader. Generating assets from webpack without hashes and then treating them as input to whitenoise should let you use django's and whitenoise's API without any need for webpack-loader.

[konoufo]

@owais I can understand why someone would still want to use django-webpack-loader even then. Say you want to build production bundles locally, you'd have a folder for production bundles and one for development bundles, you don't want to have to manually write something like :

    {% if dev %}
        <script src="{% static 'bundles/development/main.js' %}" ></script>
   {% else %}
       <script src="{% static 'bundles/production/main.js' %}" ></script>
   {% endif %}

where a single {% render_bundle 'main' js %} suffice when you appropriately set buildPath in both webpack.config.js and webpack.prod.config.js . Of course we could write a stripped down version of django-webpack-loader to do this and nothing more, but there's nothing to gain really versus using django-webpack-loader as is.

[owais]

This can still be easily solved in about 5 lines of python code. Just need to create a custom template tag that calls either render_bundle or static depending on the value of settings.DEBUG

[i-salameh95]

Hello,
I think I have the same problem with my integration of webpack & django & AWS s3
the tags that used webpack_static are still loaded from the public path and not from amazon s3 urls

for example: <img class="logo-img" alt="NNUH" src="{% webpack_static 'images/logo_final.png' %}"/> :
this still load the image from the static url, and not from amazon s3, while other static like ( admin css, js..) are loaded from amazon correctly

how to make those "webpack_static" load from s3?

[fjsj]

@i-salameh95 what publicPath config do you have in your webpack configuration file? Please share the whole config file if possible.
You may also try using webpack_asset which was introduced at #397 but not documented yet.

[i-salameh95]

@fjsj : this is my Webpack configuration:

const webpack = require('webpack');
const MiniCssExtractPlugin = require("mini-css-extract-plugin");
const BundleTracker = require('webpack-bundle-tracker');
const path = require('path');
const TerserPlugin = require("terser-webpack-plugin");
const CopyPlugin = require("copy-webpack-plugin");
const RtlCssPlugin = require('rtlcss-webpack-plugin');
const CssMinimizerPlugin = require("css-minimizer-webpack-plugin");

let config = {
    entry: {
        'bundle': [
            path.resolve(__dirname, 'js/main.js'),
        ],
    },
    output: {
        publicPath: '/static/',
        filename: 'js/[name]_[hash].js',
        chunkFilename: 'js/modules/[name]_[hash].js',
        path: path.resolve(path.dirname(__dirname), 'static'),
        clean: true,
    },
    module: {
        rules: [
            {
                test: /\.js$/,
                exclude: /(node_modules|bower_components)/,
                use: {
                    loader: 'babel-loader',
                }
            },
            {
                test: /\.css$/i,
                exclude: /\.rtl\.(css)$/,
                use: [MiniCssExtractPlugin.loader, "css-loader"],
            },
            {
                test: require.resolve('jquery'),
                use: {
                    loader: 'expose-loader',
                    options: {
                        exposes: ['$', 'jQuery']
                    }
                }
            },
            {
                test: /\.(woff2?|otf|ttf|eot|svg)$/,
                type: "asset/resource",
                generator: {
                    filename: 'fonts/[name][ext]',
                }
            },
            {
                test: /\.(png|jpg|jpeg|gif|ico)$/i,
                type: "asset/resource",
                generator: {
                    filename: 'images/[name]_[hash][ext][query]',
                },
            }
        ]
    },
    plugins: [
        new webpack.ProvidePlugin({
            $: 'jquery',
            jQuery: 'jquery'
        }),
        new MiniCssExtractPlugin({
            filename: 'css/[name]_[hash].ltr.css',
        }),
        new RtlCssPlugin({
            filename: 'css/[name]_[hash].rtl.css',
        }),
        new CopyPlugin({
            patterns: [
                {from: './images/main', to: 'images'}
            ]
        }),
        new BundleTracker({path: path.resolve(path.dirname(__dirname), 'static'), filename: 'webpack-stats.json'}),
    ],
    optimization: {
        minimizer: [
            new CssMinimizerPlugin(),
            new TerserPlugin({
                terserOptions: {
                    compress: {
                        drop_console: true,
                        unused: true,
                        dead_code: true,
                    },
                },
            }),
        ],
        splitChunks: {
            chunks: 'async',
            minSize: 30000,
            maxSize: 0,
            minChunks: 1,
            maxAsyncRequests: 10,
            maxInitialRequests: 10,
            automaticNameDelimiter: '-',
            cacheGroups: {
                vendors: {
                    test: /[\\/]node_modules[\\/]/,
                    priority: -10,
                },
                default: {
                    minChunks: 2,
                    priority: -20,
                    reuseExistingChunk: true
                }
            }
        }
    },
    devServer: {
        publicPath: "/static/",
        host: "0.0.0.0",
        port: "3000",
        hot: true,
        proxy: {
            "*": "http://django:8000"
        },
        watchOptions: {
            poll: true
        }
    }
};

module.exports = (env, argv) => {
    if (argv.mode === 'development') {
        config.devtool = 'source-map';
    }
    return config;
};

[fjsj]

@i-salameh95 OK, so you're using src="{% webpack_static 'images/logo_final.png' %}".

• If you set publicPath: auto,, the URL will be your settings.STATIC_URL + 'images/logo_final.png'.
• If you keep your publicPath: '/static/', the URL will be '/static/' + 'images/logo_final.png'.
• If you need to generate the URL from django-storages, use {% static 'images/logo_final.png' %}

As of now, webpack_static does not make use of STATICFILES_STORAGE. So you can either work with STATIC_URL or use static directly. Just remember to run webpack before collectstatic in production deployment.