automatically install peerDependencies [this was implemented in v7.1.3]

[paulpflug]

This was implemented! Update pnpm to v7.1.3 or newer and set auto-install-peers to true.

---

The original purpose of peerDependencies with npm@1 was, that a package can define packages to install alongside.
With the flattened dependencies tree with npm@3 this functionally was redundant, as ALL dependencies are getting installed alongside, as a result the automatic installation of peer dependencies was disabled and there is no real use-case for defining peer dependencies anymore..

With pnpm this isn't the case, as you choose to use a npm@1 like package dep tree, you should also use the npm@1 peerDep behaviour and install them automatically.

Say package A needs B,C,D as peerDep I would have to call:

pnpm install A B C D
# instead of
npm install A

if a peerDep conflicts with a normal dep, the normal dep should win and a warning should get printed..
(also peerDeps should get installed when linked)

What do you think?

[zkochan]

Peer dependencies are not installed because

The behavior in npms 1 & 2 was frequently confusing and could easily put you into dependency hell, a situation that npm is designed to avoid as much as possible.
src: https://docs.npmjs.com/files/package.json#peerdependencies

Peer dependencies are intended to be used by pluggable packages and are resolved from higher in the dependency tree. They are not supposed to be resolved from down the dependency tree. If they are resolved that way accidentally because of flattened node_modules we shouldn't try to emulate other package manager's bad design.

P.S. this article of mine - pnpm's strictness helps to avoid silly bugs, even started a discussion in a npm chat about making --global-style the default node_modules layout. The global-style layout reduces issues like that, issues when flat node_modules allow accessing not referenced dependencies

[paulpflug]

pnpm's strictness is a big advantage, I agree on that. But to be honest, peerDependencies in its current state are useless.. there is no point in using them neither in npm nor in pnpm. Real pluggable packages, don't exist (at least I have never seen one).

Now to the problem I want to solve: webpack.
Webpack doesn't resolve packages exactly as node.
I want a package which automatically provides a number of loaders for webpack.
With npm I can define them as normal deps and look them up either in node_modules of the package or its parent.
With pnpm it is not possible. The normal deps are not showing up in node_modules of the parent (strict - good design), but also not in node_modules of the package, so webpack has no way of finding them.

When working with peerDeps, I have to type out all peerDeps, then there is no point in using a dependency collection in the first place.

The current (untested) workaround is to manually resolve the package dir link with fs.realpathSync on runtime and add the parent directory to webpacks resolveLoader.

I think it would be totally sufficient to only auto install peerDeps on top level, after all other packages are installed, and only warn on failure, so they are treated like citizen second class - this would still allow the usage of pluggable packages, as normal dependencies would override them.

If you insist on the current way, this issue can be closed.

[zkochan]

I think the way people do it is they have their peerDependencies as devDependencies as well. That is kinda what you are asking for (installing it only on top level). Including them as devDependencies is not good?

Webpack doesn't resolve packages exactly as node.

Are you sure? We'll have to file an issue at webpack in that case. We had issues with CRA and with latest webpack they are gone, so I assumed they have fixed it. I know they have an option to turn off resolve symlink, but it is on by default. A package symlink is resolved to its real location and pnpm does create a node_modules folder one directory up the package's real location. Have you tried with webpack@2.6?

Pluggable packages don't exist (at least I have never seen one).

Maybe I named it incorrectly but babel, eslint use peer dependencies to work with their plugins. And they work with pnpm. (Except one issue with eslint #739)

[zkochan]

By the way, we have this issue at webpack webpack/webpack#5087

[vjpr]

@paulpflug

I want a package which automatically provides a number of loaders for webpack.
With npm I can define them as normal deps and look them up either in node_modules of the package or its parent.
With pnpm it is not possible. The normal deps are not showing up in node_modules of the parent (strict - good design), but also not in node_modules of the package, so webpack has no way of finding them.

I have tackled this issue extensively. Its very tricky in general - lots of edge cases, especially when npm linking during development.

You can just use webpack's resolve.modules: https://webpack.js.org/configuration/resolve/#resolve-modules to get most of the way there though.

[paulpflug]

I have tested it with webpack@3.
There is no way I can ship package A somehow connected to B so that webpack can resolve B

This worked with peerDep at npm@1-2 and with normal dep with npm@3-5 and yarn

[vjpr]

In package A you should refer to package B using require.resolve, or you can add package A's node modules dir to require.modules Do you have a sample project I can look at?

…

On Wed 28. Jun 2017 at 6:20 PM, Paul Pflugradt ***@***.***> wrote: I have tested it with ***@***.*** There is no way I can ship package A somehow connected to B so that webpack can resolve B This worked with peerDep at ***@***.*** and with normal dep with ***@***.*** and yarn — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#827 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AARLRa1XLns8OpxqYH4NdMTXhCESXs0Xks5sIn1jgaJpZM4OFG7M> .

[paulpflug]

In package A you should refer to package B using require.resolve
This won't work for other dependencies than js. Stylesheets for example.

or you can add package A's node modules dir to require.modules
This will work, but then the installation instructions of A get very ugly, I really want to prevent that.
I also could order the user to also install B, also ugly in my eyes.

[vjpr]

With webpack everything is possible! I'm not 100% on what your trying to do but if you can show me a sample project I'm pretty sure I can resolve it. Relying on flat node modules will be problematic if there are multiple versions of the dep in the tree too. But I'd have to see code.

…

On Wed 28. Jun 2017 at 6:59 PM, Paul Pflugradt ***@***.***> wrote: In package A you should refer to package B using require.resolve This won't work for other dependencies than js. Stylesheets for example. or you can add package A's node modules dir to require.modules This will work, but then the installation instructions of A get very ugly, I really want to prevent that. I also could order the user to also install B, also ugly in my eyes. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#827 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AARLRZ2k5-MwO6G-OSY8irkSAo0K4IGEks5sIoaGgaJpZM4OFG7M> .

[paulpflug]

I have ceri-materialize which is basically a wrapper around materialize-css with a few added stylesheets.
I want the user to only install ceri-materialize but be able to resolve materialize-css stylesheets in sass:

// sass
@import "~ceri-materialize/color"; //works
// works in npm due to flattened tree
// but not in pnpm
@import "~materialize-css/sass/components/variables";

// no way to resolve the dependency:
@import "~ceri-materialize/../materialize-css/sass/components/variables"; //doesn't work
@import "~ceri-materialize/node-modules/materialize-css/sass/components/variables"; //doesn't work

for example code you can clone ceri-tooltip
git clone https://github.com/ceri-comps/ceri-tooltip.git

and run cd ceri-tooltip && pnpm i && npm run dev

thanks for your time!

[vjpr]

Okay reproduced:

ERROR in ./node_modules/.registry.npmjs.org/css-loader/0.28.4/node_modules/css-loader!./node_modules/.registry.npmjs.org/sass-loader/6.0.6/node_modules/sass-loader/lib/loader.js!./dev/materialize.config.scss
Module build failed:
@import "~materialize-css/sass/components/mixins";
^
      File to import not found or unreadable: ~materialize-css/sass/components/mixins.
Parent style sheet: stdin
      in /Users/Vaughan/dev-live/ceri-tooltip/dev/materialize.config.scss (line 3, column 1)
 @ ./dev/materialize.config.scss 4:14-232 18:2-22:4 19:20-238
 @ ./dev/materialize.coffee
 @ ./node_modules/.registry.npmjs.org/ceri-dev-server/1.0.6/node_modules/ceri-dev-server/lib/ceri-dev-client.js
 @ multi ./node_modules/.registry.npmjs.org/koa-hot-dev-webpack/0.1.10/webpack@3.0.0/node_modules/koa-hot-dev-webpack/hot-reload.js ./node_modules/.registry.npmjs.org/ceri-dev-server/1.0.6/node_modules/ceri-dev-server/lib/ceri-dev-client.js

[vjpr]

So in ceri-tooltip/dev/materialize.coffe.scss you are referencing materialize-css. This is non-standard and should be avoided - there are many things that could go wrong and break.

You should depend directly on materialize-css or you should access materialize-css via the ceri-materialize module.

pnpm will never support accessing a module from a module that doesn't depend on it.

However, you can configure webpack with a custom loader resolver. Modify ceri-dev-server/lib/webpack.config.js to include node_modules/ceri-materialize/node_modules.

[vjpr]

If there is ever another version of materialize-css used in your code base by your library users, then this will cause issues. pnpm is much safer, and not relying on the flat module structure is always best.

@zkochan We should have a tag we use for all the issues relating to flat modules. And we should have a standard FAQ page to point people to explaining why relying on flat modules is bad.

[zkochan]

@vjpr lets create a pnpm-manifesto repo with listings of all the things pnpm is meant to be

And of course I am always happy to extend the FAQ section

[paulpflug]

I think it all breaks down to my impression, that if a package A has a peerDep B and I install A I should also have access to B. (same problem in #829)

C
-A
-B (peer of A)

Currently peerDep really means optionalDep, but with a warning if it isn't installed.. (making that optionality obsolete 😑)

[milahu]

Peer dependencies are intended to be used by pluggable packages
and are resolved from higher in the dependency tree.
They are not supposed to be resolved from down the dependency tree.

With npm@7 auto-installing peerDependencies now

No, we require that peer dependencies should be added as dependencies of the project.

so now pnpm has its own opinions, and is incompatible with npm? HMMM ... (fork, anyone?)

npm versions 1, 2, and 7 will automatically install peerDependencies
if they are not explicitly depended upon higher in the dependency tree.

For npm versions 3 through 6,
you will receive a warning that the peerDependency is not installed instead.

https://nodejs.org/en/blog/npm/peer-dependencies/

currently, pnpm does not even show a warning 0__o

related: a peerdep can be made optional with peerDependenciesMeta

  "peerDependencies": {
    "a": "*"
  },
  "peerDependenciesMeta": {
    "a": { "optional": true }
  }

[LumaKernel]

I noticed we can use .pnpmfile.cjs for this purpose.

function readPackage(pkg) {
  pkg.dependencies = {
    ...pkg.peerDependencies,
    ...pkg.dependencies,
  }
  pkg.peerDependencies = {};

  return pkg;
}

module.exports = {
  hooks: {
    readPackage,
  },
};

I'm using more opinionated version of this.

[fc]

Can we have add a parameter like pnpm install --include-peers that does what @LumaKernel posted?
What's the point of peerDependenciesMeta.*.optional if all peer dependencies are basically optional anyway?

[fc]

It looks like install-peerdeps (here) supports pnpm.

Example:

npx install-peerdeps --pnpm <your-package>

[milahu]

Can we add a parameter

this should be the default behavior. breaking with NPMv7 is just @zkochan's power trip ...
feel free to ban me from the pnpm org, only to prove my point : D
seems like all the political tensions transform maintainers into fanatic conservatives

team red is leaving the chat

edit: 🖕 to all the haters.
youre just lucky that your stupidity has no short-term consequences

[zkochan]

what is wrong with you?

[zkochan]

If you want this feature so bad, make a PR and make it opt-in. If most will vote to make it the default, then we'll make it the default. I don't understand what is your problem with me.

[milahu]

i tried some weeks ago, but i gave up on the complexity of pnpm

[zkochan]

So apparently not everyone is happy about how this changed in npm: npm/feedback#610

[prantlf]

Someone is happy (see install-peer-deps) and someone is unhappy about software features. It is nothing new. The important point here is that we cannot use PNPM as a drop-in replacement since NPM@7, because PNPM is not compatible with NPM any more. If everyone insisted on their own opinions in the software industry, developers would be only looking for bridges and workarounds. I wonder whom it helps. Developers are supposed to work on their projects.

Why not just following NPM's decision and avoiding unpleasant surprises that people switching to PNPM get because of this incompatibility?

[maicol07]

Personally, I'd prefer to keep the current behaviour of pnpm install and instead adding an option to this command to install all the peer dependencies and an option like the one I described in #4178

[OskarD]

That sounds good too, if it was clearly stated in the getting started page (Which I find kind of confusing btw, it could probably use a rewrite) that this is an option for people migrating from NPM@8 then I think it would be clear enough

[zkochan]

we need a page that makes a feature comparison between pnpm, npm, and Yarn.

[OskarD]

I thought the script had fixed everything but I think I ran into that thing you were talking about @zkochan. Something is loaded twice with different versions, causing errors. Sorry for OT but can you point me in the right direction to, like, a guide on how to handle this the right way?

[zkochan]

If you remove that workaround with .pnpmfile.cjs, you may just install the missing peer dependencies in the projects where they are missing. You even get the list of the missing peer deps in the output of pnpm install. So you can just peek the package from the list and add to pnpm add <peers..> --save-dev

For instance

[zkochan]

In v6.26 a new setting was released: auto-install-peers. If you set this option to true, the pnpm add <pkg> command will automatically install any missing peer dependencies as dev dependencies of the project.

[prantlf]

@zkochan, how is the setting auto-install-peers supposed be used? Is there a way to apply a global setting (user config?) to get the peer dependencies installed automatically without modifying every project or using long command lines? I tried some ways and found only one workaround with annoying side-effects. Let me know if you want to move this from here to an issue or another discussion, please.

My environment:

❯ node --version && npm --version && pnpm --version && cat ~/.npmrc
v16.13.1
8.1.2
6.26.1
package-lock=true

NPM

This is the scenario with npm that I want to achieve with pnpm:

❯ npm i requirejs-babel7

added 3 packages, and audited 4 packages in 4s

found 0 vulnerabilities

The package with its two peer dependencies was installed, package.json was not modified, package-lock.json was written. I am looking for settings, which can be applied globally in the development environment (user config, for example), without having to modify each project to make it working with pnpm.

PNPM - Forced Command Line

Although the forced command line was not what I was looking for, nothing else worked. Forcing usage of unknown config settings works, although the syntax is not what I expect for a documented setting and there are two other problems:

❯ pnpm i requirejs-babel7 --config.auto-install-peers=true
Already up-to-date
 WARN  Issues with peer dependencies found
.
└─┬ requirejs-babel7
  ├── ✕ missing peer @babel/standalone@7.x
  └── ✕ missing peer babel-plugin-transform-modules-requirejs-babel@0.x
Peer dependencies that should be installed:
  @babel/standalone@7.x
  babel-plugin-transform-modules-requirejs-babel@0.x

Installing missing peer dependencies
Progress: resolved 2, reused 1, downloaded 0, added 0, done

And although it works, it performs two unexpected actions:

1. It prints a warning "Issues with peer dependencies found" and then a message "Installing missing peer dependencies". Warnings should be printed for unexpected situations, which may mean a problem and the user should pay attention to them. This was wanted and configured situation. The warning is a false positive, makes the log output confusing and problem analysis less efficient. Could you remove the warning when auto-install-peers is set, please?
2. It adds the installed peerDependencies to the devDependencies object in package.json. But this is not a change that I want to commit to my project. I would have to constantly remove it before pushing my changes. How to suppress adding the extra devDependencies to package.json?

PNPM - Command Line

The argument auto-install-peers is rejected as unknown on the command line:

❯ pnpm i requirejs-babel7 --auto-install-peers=true
 ERROR   ERROR  Unknown option: 'auto-install-peers'
Did you mean 'unsafe-perm'? Use "--config.unknown=value" to force an unknown option.
For help, run: pnpm help add

PNPM - NPMRC Config

The config auto-install-peers appears not to be recognised, when set by pnpm config. The peer dependencies were not installed:

❯ pnpm config set auto-install-peers=true
❯ pnpm config get auto-install-peers
true
❯ pnpm i requirejs-babel7
Packages: +1
+
Packages are hard linked from the content-addressable store to the virtual store.
  Content-addressable store is at: /Users/ferdipr/.pnpm-store/v3
  Virtual store is at:             node_modules/.pnpm

dependencies:
+ requirejs-babel7 1.1.0

 WARN  Issues with peer dependencies found
.
└─┬ requirejs-babel7
  ├── ✕ missing peer @babel/standalone@7.x
  └── ✕ missing peer babel-plugin-transform-modules-requirejs-babel@0.x
Peer dependencies that should be installed:
  @babel/standalone@7.x
  babel-plugin-transform-modules-requirejs-babel@0.x

Progress: resolved 1, reused 1, downloaded 0, added 1, done

PNPM - package.json

Although I did not want to modify the local project, I was so desperate that I tried something crazy in package.json:

"pnpm": {
  "autoInstallPeers": true,
  "peerDependencyIssues": {
    "autoInstallPeers": true
  }
}

This had no effect on the peer dependency handling.

[zkochan]

It prints a warning "Issues with peer dependencies found" and then a message "Installing missing peer dependencies".

Yes, I know, this should be removed.

It adds the installed peerDependencies to the devDependencies object in package.json. But this is not a change that I want to commit to my project. I would have to constantly remove it before pushing my changes. How to suppress adding the extra devDependencies to package.json?

I don't understand this. Why don't you want to commit this change? It should be committed.

The argument auto-install-peers is rejected as unknown on the command line:

We can make --auto-install-peers work but I assumed that everyone will just set it as an option through .npmrc.

pnpm config set auto-install-peers true should work. If it doesn't work, then it is a bug. I tested this setting by mannually adding the setting to a .npmrc file in the root of the repository

[trathailoi]

it seems to work only with pnpm add <package> and not with pnpm i <package> or pnpm i.
@zkochan I hope that you could make it to work also with pnpm i, that would be really nice. It's very common the case when I clone an old repo and need to pnpm i and expect all the peerDependencies should be installed as well.

[jakubmazanec]

Peer dependencies should be installed automatically and package.json should not be changed, because I want to specify only those deps that I use directly. In the example @prantlf posted, I don't care that requirejs-babel7 (if I correctly understand the purpose of that package) uses Babel for something; for me as a user of requirejs-babel7, that babel requirement could have been regular dependency instead of peer.
Another view - if a package manager has all the information necessary to install all dependencies, why should I add another, redundant information to my package.json?
I think npm@7 made a good decision regarding autoinstall of peer deps - combine that with how pnpm structures node_modules, and we could have a perfect package manager.

[rafaelliu]

@zkochan Also looking for this feature.

I don't understand this. Why don't you want to commit this change? It should be committed.

Since I believe there hasn't been much talk about the use case but rather the mechanics, I'll give my intended use: I have config packages in my monorepo (eslint, mocha, tsconfig, etc) that could also act as meta-packages and install the relevant dependencies.

Right now, adding eslint would look like: 1) I have to pnpm add myeslint-config, 2) import it on my package's .eslint.cjs then 3) add all@typescript-eslint/eslint-plugin, @typescript-eslint/parser, etc. For each package. Would be nice to get that done dynamically, so changes on eslint-config (e.g. added prettier) could be handled elegantly with a simple pnpm i --recursive.

I understand that's sort of selectively doing hoisting back again (none of the hoist or peer-dep .npmrc configs worked for me tho), but I'd argue all such use cases happen in dev and the there are real benefits.

Maybe I'm doing myself a disservice bringing up Java, this is a feature of other package managers such as Maven.

[awkj]

I set auto-install-peers = true, but don't work

example is https://docusaurus.io/docs/typescript-support

pnpx create-docusaurus@latest my-website classic --typescript
pnpm install --save-dev typescript @docusaurus/module-type-aliases @tsconfig/docusaurus

[zkochan]

There was a bug (fixed in v6.32.7), update to the latest pnpm.

[unional]

I also can't get auto-install-peers to work. I'm using rush with pnpm

[mechadragon01]

• I have auto-install-peers=true in .npmrc and peer dependencies aren't installed.
• I am using the command pnpm add -D <package-name>

Does this not work with dev dependencies, the -D flag?

Running pnpm version 7.0.1

[zkochan]

auto-install-peers=true now makes pnpm work the same way as npm v7. From pnpm v7.1.3