Support webpack externals, modules listed as external should not be pruned during package

[dustinstein]

• I have read the contribution documentation for this project.
• I agree to follow the code of conduct that this project follows, as appropriate.
• I have searched the issue tracker for an issue that matches the one I want to file, without success.

I am using 'electron-edge-js' to run a simple C# script in my main.js file. To use the imported package I've set my webpack.main.config.js to:

module.exports = {

  entry: ['babel-polyfill','./src/main.js'],
  module: {
    rules: require('./webpack.rules'),
  },
  resolve: {
    extensions: ['*','.js', '.jsx'],
  },
 externals: {
   'electron-edge-js':  'electron-edge-js',
 }
};

This works as expected when using the start script (because the packages are in the node_modules folder). But when the app is packaged, I get the error: module 'electron-edge-js' not found.
When inspecting the built resources>app>node_modules folder I see that it's empty and the created package.json shows no dependencies. I feel like I'm missing something obvious but I've searched and tried possible solutions to no avail.

What does your config.forge data in package.json look like?

"config": {
    "forge": {
      "packagerConfig": {},
      "makers": [
        {
          "name": "@electron-forge/maker-squirrel",
          "config": {
            "name": "gps_app",
          }
        },
        {
          "name": "@electron-forge/maker-zip",
          "platforms": [
            "darwin"
          ]
        },
        {
          "name": "@electron-forge/maker-deb",
          "config": {}
        },
        {
          "name": "@electron-forge/maker-rpm",
          "config": {}
        }
      ],
      "plugins": [
        [
          "@electron-forge/plugin-webpack",
          {
            "mainConfig": "./webpack.main.config.js",
            "renderer": {
              "config": "./webpack.renderer.config.js",
              "entryPoints": [
                {
                  "html": "./src/index.html",
                  "js": "./src/renderer.js",
                  "name": "main_window"
                }
              ]
            }
          }
        ]
      ]
    }
  },

Minimum test case to reproduce can be found here
or clone: git@github.com:dustinstein/test-app.git

[andrewrt]

@dustinstein - I'm seeing the same issue here. Did you make any progress on this?

[dustinstein]

I'm almost embarrassed to say what I did but I was at the end of my expertise. I started a new project with electron-edge-js-quick-start and copied my main electron process code to that project. Since the electron-forge webpack was working well (before packaging or making) I built the project with electron-forge and used the compiled index.js file in .webpack/renderer/main_window folder in my electron-edge-js-quick-start project and it ran and packaged well.
Of course you'll have to manually move dependencies to your package.json and you may have to change Electron versions depending on your project.
When I have time I'm going to dive deeper and see if maybe I can do something a bit cleaner. It works but it's a messy solution.

[andrewrt]

@dustinstein -
The brute force method might not be elegant - but if it works, it works!
I may end up doing the same if I can't figure out the intended flow for forge + webpack, thanks for the quick response!

[lhr0909]

I have been digging through the packger code for hours and found a solution by adding a few hooks. Basically we need to find out which packages got marked as externals in webpack, and run npm install in the build path to fill up the node_modules folder in the package. Seems hacky but I think it works fine. The inspiration comes from the build steps of serverless-webpack package.

Sharing my forge config here:

const fs = require('fs-extra');
const path = require('path');
const { spawn } = require('child_process');

module.exports = {
  packagerConfig: {},
  makers: [
    {
      name: '@electron-forge/maker-squirrel',
      config: {
        name: 'puppeteer_playground',
      },
    },
    {
      name: '@electron-forge/maker-zip',
      platforms: [
        'darwin',
      ],
    },
    {
      name: '@electron-forge/maker-deb',
      config: {},
    },
    {
      name: '@electron-forge/maker-rpm',
      config: {},
    },
  ],
  plugins: [
    [
      '@electron-forge/plugin-webpack',
      {
        mainConfig: './webpack.main.config.js',
        renderer: {
          config: './webpack.renderer.config.js',
          entryPoints: [
            {
              html: './src/renderer/index.html',
              js: './src/renderer/index.tsx',
              name: 'main_window',
            },
          ],
        },
      },
    ],
  ],
  hooks: {
    readPackageJson: async (forgeConfig, packageJson) => {
      // only copy deps if there isn't any
      if (Object.keys(packageJson.dependencies).length === 0) {
        const originalPackageJson = await fs.readJson(path.resolve(__dirname, 'package.json'));
        const webpackConfigJs = require('./webpack.renderer.config.js');
        Object.keys(webpackConfigJs.externals).forEach(package => {
          packageJson.dependencies[package] = originalPackageJson.dependencies[package];
        });
      }
      return packageJson;
    },
    packageAfterPrune: async (forgeConfig, buildPath) => {
      console.log(buildPath);
      return new Promise((resolve, reject) => {
        const npmInstall = spawn('npm', ['install'], {
          cwd: buildPath,
          stdio: 'inherit',
        });

        npmInstall.on('close', (code) => {
          if (code === 0) {
            resolve();
          } else {
            reject(new Error('process finished with error code ' + code));
          }
        });

        npmInstall.on('error', (error) => {
          reject(error);
        });
      });
    }
  }
};

[flavioribeirojr]

@lhr0909 I can't thank you enough. I was stuck in this for two days

[ivancuric]

@lhr0909 I added your config and it works for the initial compilation step and HMR!

However, any page reload after subsequent webpack rebuilds results in an error:

Unable to load preload script: D:\dev\novocam-recording-software\.webpack\renderer\main_window\preload.js
Error: Cannot find module 'undefinedbuild/Release/video-module.node'
Require stack:
- D:\dev\novocam-recording-software\.webpack\renderer\main_window\preload.js

Even though preload.js is on that path.

Also, running yarn make or yarn package results in

C:\Users\ivanc\AppData\Local\Temp\electron-packager\win32-x64\Futudent-win32-x64\resources\app

An unhandled rejection has occurred inside Forge:
Error: spawn npm ENOENT
    at Process.ChildProcess._handle.onexit (node:internal/child_process:282:19)
    at onErrorNT (node:internal/child_process:480:16)
    at processTicksAndRejections (node:internal/process/task_queues:81:21) {
  errno: -4058,
  code: 'ENOENT',
  syscall: 'spawn npm',
  path: 'npm',
  spawnargs: [ 'install' ]
}

Electron Forge was terminated. Location:
{}
node:events:355
      throw er; // Unhandled 'error' event
      ^

Error: spawn electron-forge ENOENT
    at notFoundError (D:\dev\novocam-recording-software\node_modules\cross-spawn\lib\enoent.js:6:26)
    at verifyENOENT (D:\dev\novocam-recording-software\node_modules\cross-spawn\lib\enoent.js:40:16)
    at ChildProcess.cp.emit (D:\dev\novocam-recording-software\node_modules\cross-spawn\lib\enoent.js:27:25)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:290:12)
Emitted 'error' event on ChildProcess instance at:
    at ChildProcess.cp.emit (D:\dev\novocam-recording-software\node_modules\cross-spawn\lib\enoent.js:30:37)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:290:12) {
  code: 'ENOENT',
  errno: 'ENOENT',
  syscall: 'spawn electron-forge',
  path: 'electron-forge',
  spawnargs: [ 'make' ]
}
error Command failed with exit code 1.

[MarshallOfSound]

Good prior art from @timfish #1276 (comment)

https://www.npmjs.com/package/@timfish/forge-externals-plugin

[timfish]

There's also an unfinished PR for forge:
#2345

There were some reported issues around scoped packages which should probably be looked into/tested:
https://github.com/timfish/forge-externals-plugin/pulls

[augustnmonteiro]

I'm having the same issue, I'm using screeshot-desktop, sharp, active-win.

inside my webpack configuration I have set them up as external, they work fine when I run it locally, but when I pack it, it doesn't work

[augustnmonteiro]

I have been digging through the packger code for hours and found a solution by adding a few hooks. Basically we need to find out which packages got marked as externals in webpack, and run npm install in the build path to fill up the node_modules folder in the package. Seems hacky but I think it works fine. The inspiration comes from the build steps of serverless-webpack package.

Sharing my forge config here:

const fs = require('fs-extra');
const path = require('path');
const { spawn } = require('child_process');

module.exports = {
  packagerConfig: {},
  makers: [
    {
      name: '@electron-forge/maker-squirrel',
      config: {
        name: 'puppeteer_playground',
      },
    },
    {
      name: '@electron-forge/maker-zip',
      platforms: [
        'darwin',
      ],
    },
    {
      name: '@electron-forge/maker-deb',
      config: {},
    },
    {
      name: '@electron-forge/maker-rpm',
      config: {},
    },
  ],
  plugins: [
    [
      '@electron-forge/plugin-webpack',
      {
        mainConfig: './webpack.main.config.js',
        renderer: {
          config: './webpack.renderer.config.js',
          entryPoints: [
            {
              html: './src/renderer/index.html',
              js: './src/renderer/index.tsx',
              name: 'main_window',
            },
          ],
        },
      },
    ],
  ],
  hooks: {
    readPackageJson: async (forgeConfig, packageJson) => {
      // only copy deps if there isn't any
      if (Object.keys(packageJson.dependencies).length === 0) {
        const originalPackageJson = await fs.readJson(path.resolve(__dirname, 'package.json'));
        const webpackConfigJs = require('./webpack.renderer.config.js');
        Object.keys(webpackConfigJs.externals).forEach(package => {
          packageJson.dependencies[package] = originalPackageJson.dependencies[package];
        });
      }
      return packageJson;
    },
    packageAfterPrune: async (forgeConfig, buildPath) => {
      console.log(buildPath);
      return new Promise((resolve, reject) => {
        const npmInstall = spawn('npm', ['install'], {
          cwd: buildPath,
          stdio: 'inherit',
        });

        npmInstall.on('close', (code) => {
          if (code === 0) {
            resolve();
          } else {
            reject(new Error('process finished with error code ' + code));
          }
        });

        npmInstall.on('error', (error) => {
          reject(error);
        });
      });
    }
  }
};

thank you for taking your time to post it here.

I'm getting an error when I try to make my app with that, I'm adding the hooks to webpack.main.config
using electron-forge: 6.0.0-beta.63

the error:

An unhandled error has occurred inside Forge:
Invalid configuration object. Webpack has been initialized using a configuration object that does not match the API schema.

should I add the hooks somewhere else?

[lhr0909]

@augustnmonteiro the configuration should be added into forge.config.js

[mrdkprj]

I had the same issue using "sharp" module.
I solved by the use of "process.env.NODE_ENV" value.
(set the required module as externals only when "start" and remove externals when "package")

module.exports = {

  entry: ['babel-polyfill','./src/main.js'],
  module: {
    rules: require('./webpack.rules'),
  },
  resolve: {
    extensions: ['*','.js', '.jsx'],
  },
 externals: process.env.NODE_ENV === 'development' ? {
   'electron-edge-js':  'electron-edge-js',
 } : {}
};

[javierguzman]

Do packageAfterPrune run in development as well? I am getting errors with serialport and usb and the solutions people mention do not work on development mode.

Thank you in advance.

[bakabird]

Good prior art from @timfish #1276 (comment)

https://www.npmjs.com/package/@timfish/forge-externals-plugin

it work!

[branbarh]

I had to make a few modifications to @lhr0909's configuration to get it to work with ES6 modules and Windows. Here are my hooks:

  hooks: {
    // Handle externals properly:
    readPackageJson: async (forgeConfig, packageJson) => {
      // only copy deps if there isn't any
      if (Object.keys(packageJson.dependencies).length === 0) {
        const originalPackageJson = await fs.readJson(path.resolve(__dirname, "package.json"));
        Object.keys(rendererConfig.externals || {}).forEach((pkg) => {
          packageJson.dependencies[pkg] = originalPackageJson.dependencies[pkg];
        });
      }
      return packageJson;
    },
    packageAfterPrune: async (forgeConfig, buildPath) => {
      console.log(buildPath);
      try {
        const res = await execSync("npm install", { cwd: buildPath });
        console.log(res.toString());
      } catch (error) {
        console.error(error);
      }
    }
  }

You'll need these imports:

import { mainConfig } from "./webpack.main.config";
import { rendererConfig } from "./webpack.renderer.config";

import fs from "fs-extra";
import path from "path";
import { execSync } from "child_process";

It works for me on Windows and macOS.

Hopefully this helps someone!