Can't resolve nlapack, nblas and util in node_modules

[Roy-Kid]

Hi developers,

I try to import vectorious in my typescript project, but when using webpack 5.89 to start a dev server, it told me:

[1] ERROR in ../app/node_modules/nblas/dist/index.js 13:12-30
[1] Module not found: Error: Can't resolve './nblas' in '/home/jicli594/work/molcrafts/projects/molvis/app/node_modules/nblas/dist'
[1]  @ ../app/node_modules/vectorious/dist/index.js 1:5455-5471
[1]  @ ../app/lib/molvis/box.js 4:21-42
[1]  @ ../app/lib/molvis/system.js 4:14-30
[1]  @ ../app/lib/molvis/app.js 28:17-36
[1]  @ ../app/lib/index.js 7:30-53
[1]  @ ./src/index.js 1:0-39 19:12-25
[1] 
[1] ERROR in ../app/node_modules/nlapack/dist/index.js 12:14-34
[1] Module not found: Error: Can't resolve './nlapack' in '/home/jicli594/work/molcrafts/projects/molvis/app/node_modules/nlapack/dist'
[1]  @ ../app/node_modules/vectorious/dist/index.js 1:10863-10881 1:13876-13894
[1]  @ ../app/lib/molvis/box.js 4:21-42
[1]  @ ../app/lib/molvis/system.js 4:14-30
[1]  @ ../app/lib/molvis/app.js 28:17-36
[1]  @ ../app/lib/index.js 7:30-53
[1]  @ ./src/index.js 1:0-39 19:12-25
[1] 
[1] ERROR in ../app/node_modules/vectorious/dist/index.js 1:21972-21987
[1] Module not found: Error: Can't resolve 'util' in '/home/jicli594/work/molcrafts/projects/molvis/app/node_modules/vectorious/dist'
[1] 
[1] BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
[1] This is no longer the case. Verify if you need this module and configure a polyfill for it.
[1] 
[1] If you want to include a polyfill, you need to:
[1]     - add a fallback 'resolve.fallback: { "util": require.resolve("util/") }'
[1]     - install 'util'
[1] If you don't want to include a polyfill, you can use an empty module like this:
[1]     resolve.fallback: { "util": false }
[1]  @ ../app/lib/molvis/box.js 4:21-42
[1]  @ ../app/lib/molvis/system.js 4:14-30
[1]  @ ../app/lib/molvis/app.js 28:17-36
[1]  @ ../app/lib/index.js 7:30-53
[1]  @ ./src/index.js 1:0-39 19:12-25
[1] 
[1] 3 errors have detailed information that is not shown.
[1] Use 'stats.errorDetails: true' resp. '--stats-error-details' to show it.
[1] 
[1] webpack 5.89.0 compiled with 3 errors and 2 warnings in 68 ms

I am sure that I have installed liblapack and nlapack itself, but I still get the problem. How should I fix this problem? The reproduced project is https://github.com/MolCrafts/molvis, I just use the template and do not change any configs.

Thanks in advance for your help!!!!!

[mateogianolio]

Hey there, can you try installing it with npm install vectorious --no-optional? The C++ bindings to LAPACK/BLAS do not work in the browser.

[Roy-Kid]

Thanks for your rapid reply!!! I use --no-optional but it told me

(base) molvis/app [ npm install vectorious --no-optional                                                                                                                                                                                                  dev-bbl * ] 10:02 AM
npm WARN config optional Use `--omit=optional` to exclude optional dependencies, or
npm WARN config `--include=optional` to include them.
npm WARN config 
npm WARN config     Default value does install optional deps unless otherwise omitted.

Then I uninstall it and use (base) molvis/app [ npm install vectorious --omit=optional , the warning is gone. But dev the project, it still told me nblas and nlapack can't resolve. My node version is v21, which installed directly from nvm. Sorry to take up your time

[mateogianolio]

No problem!

I think this issue is related, you could try adding this to your package.json and see if it works:

"browser": {
  "nblas": false,
  "nlapack": false
}

[Roy-Kid]

Error still, sorry. Two more clues:

1. no matter use --omit=optional or just npm install vectorious, same "vectorious": "^6.1.12" shown in dependencies;
2. If I use a empty template, there is an extra error:

ERROR in ../app/node_modules/vectorious/dist/index.js 1:21972-21987
Module not found: Error: Can't resolve 'util' in '/home/jicli594/molcrafts/projects/molvis/app/node_modules/vectorious/dist'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "util": require.resolve("util/") }'
	- install 'util'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "util": false }

Actually, I don't understand, so I just followed up the instructions to install util and add a fallback for resolve. Is this related to this issue?

[mateogianolio]

1. Yes, this is expected. The difference is that adding --omit=optional will not install nblas and nlapack libraries as optional dependencies.
2. Not that familiar with new versions of webpack, but what if you also add "nblas": false and "nlapack": false to resolve.fallback instead of my suggestion above? I can't try it myself right now but maybe later!

[Roy-Kid]

I think it is related to webpack, maybe the browser field can not feed to vectorious?

[1] ERROR in ../app/node_modules/nlapack/dist/index.js 12:14-34
[1] Module not found: Error: Can't resolve './nlapack' in '/home/jicli594/molcrafts/projects/molvis/app/node_modules/nlapack/dist'
[1] resolve './nlapack' in '/home/jicli594/molcrafts/projects/molvis/app/node_modules/nlapack/dist'
[1]   using description file: /home/jicli594/molcrafts/projects/molvis/app/node_modules/nlapack/package.json (relative path: ./dist)
[1]     Field 'browser' doesn't contain a valid alias configuration
[1]     using description file: /home/jicli594/molcrafts/projects/molvis/app/node_modules/nlapack/package.json (relative path: ./dist/nlapack)
[1]       no extension
[1]         Field 'browser' doesn't contain a valid alias configuration
[1]         /home/jicli594/molcrafts/projects/molvis/app/node_modules/nlapack/dist/nlapack doesn't exist
[1]       .ts
[1]         Field 'browser' doesn't contain a valid alias configuration
[1]         /home/jicli594/molcrafts/projects/molvis/app/node_modules/nlapack/dist/nlapack.ts doesn't exist
[1]       .js
[1]         Field 'browser' doesn't contain a valid alias configuration
[1]         /home/jicli594/molcrafts/projects/molvis/app/node_modules/nlapack/dist/nlapack.js doesn't exist
[1]       as directory
[1]         /home/jicli594/molcrafts/projects/molvis/app/node_modules/nlapack/dist/nlapack doesn't exist
[1]  @ ../app/node_modules/vectorious/dist/index.js 1:10863-10881 1:13876-13894
[1]  @ ../app/lib/molvis/box.js 4:21-42
[1]  @ ../app/lib/molvis/system.js 4:14-30
[1]  @ ../app/lib/molvis/app.js 28:17-36
[1]  @ ../app/lib/index.js 7:30-53
[1]  @ ./src/index.js 1:0-39 19:12-25
[1] 
[1] webpack 5.89.0 compiled with 2 errors and 2 warnings in 113 ms

For util, I add a NodePolyfillPlugin according to stack overflow. I also try to add what you mentioned in both package.json and webpack configs. Maybe I should use another packing system such as turbopack?

[mateogianolio]

Ahh I realise I linked to the wrong issue in the previous comment haha. Here is what I meant: webpack/webpack#8826

[Roy-Kid]

I read the webpack link you provide carefully. As for my understanding, I should add browser in my lib's package.json. But when I did it( even try in each package.json), still the error raise. I switch webpack to nextjs but still:

3:34:34 PM - Starting compilation in watch mode...
[0] 
[1]    ▲ Next.js 14.0.1
[1]    - Local:        http://localhost:3000
[1] 
[0] 
[0] 3:34:35 PM - Found 0 errors. Watching for file changes.
[1]  ✓ Ready in 2.2s
[1]  ○ Compiling /page ...
[1]  ⨯ ../app/node_modules/nblas/dist/index.js:13:0
[1] Module not found: Can't resolve './nblas'
[1] 
[1] https://nextjs.org/docs/messages/module-not-found
[1] 
[1] Import trace for requested module:
[1] ../app/node_modules/vectorious/dist/index.js
[1] ../app/lib/molvis/box.js
[1] ../app/lib/molvis/system.js
[1] ../app/lib/molvis/app.js
[1] ../app/lib/index.js
[1] ./src/app/molvis.tsx
[1] ./src/app/page.tsx
[1]  ⨯ ../app/node_modules/nblas/dist/index.js:13:0

[mateogianolio]

Looked at your project!

Apparently webpack > 5 requires some additional config to work with conditional dependencies.

Use IgnorePlugin to ignore nblas and nlapack:

new IgnorePlugin({ resourceRegExp: /^nblas|nlapack$/ })

The util fallback can be set to false (util is only used in vectorious to prettyprint in the terminal):

fallback: { "util": false }

Finally, remove nblas and nlapack from /app and reinstall vectorious with --omit=optional!