Adopt pnpm as a package manager for this Repository

[ovflowd]

Hey, you all 👋 after doing some experimentation, I wanted to ask the team (cc @nodejs/website) what you think about adopting pnpm as the package manager for this repository.

Some of the key caveats/benefits I've noticed:

• Installing packages on CI reduced 60% from (varying 20-50seconds to blazingly 4-7 seconds)
• Running package binaries do not require npx
• Native monorepo support if we ever need
• Smaller node_modules and cache footprint (making the GitHub Actions cache storage have more space) (Around 70% less footprint)

Some of the disadvantages:

• Requires Extra Step on CI to install PNPM or use Corepack
• Not natively installed with Node.js, so newcomers need also to install pnpm

I also want the opinion of the @nodejs/build team and the TSC @nodejs/tsc as I want to ensure that this change is not controversial nor conflicting with any of our bylaws and/or creates an adverse affect on the "npm" package manager (as if this repository does any endorsement to pnpm over npm which is not the case).

[mhdawson]

My initial thought is that what is most important is how easy it is to contribute a change to the website. I'm less worried about things taking a bit longer in our CI.

Are there any significant benefits to people submitting a PR in terms of the overall time that will take them? If not I think avoiding people having to do an additional install may not be worth it.

[bmuenzenmeyer]

adding a note here, as the conversation came up again. see #5334 (comment)

I agree with the sentiment that website contribute should be easy. I suggest that this may add a step, but doesn't significantly change an already complex project

[ovflowd]

That is a great argument. And I'm in favour of it. If ultimately what matters is DX, adding another layer of dependency that requires another install on the userland can be discouraging and add extra debugging/issues to the contribution cycle towards this repository.

And a few seconds of CI time is not a big deal. Again the idea of adopting pnpm has its pros and cons.

And truth be said, npm is more than sufficient, and I see no issues with keeping it at the time of writing this. (NPM is darn great).

[AugustinMauroy]

With PNPM it's possible to keep package.json and package-lock.json and just use PNPM on CI CD ?
In this case PNPM will be great.

[ovflowd]

No? We should not allow users to arbitrarily decide which package manager they use. If we stay with npm they need to use npm.

But both yarn and pnpm "support" reading the package-lock.json, just for reference. So theoretically they can use and respect the versions. But anyhow that's off-topic.

[aduh95]

Imposing a specific package manager is a very common practice, the same way we impose e.g. using node, even though it might work to run the code with a different JS runtime; it’s easier for everyone if we’re all on the same page.

[JakobJingleheimer]

I dunno about very common. I've rarely seen it (perhaps just me).

Global tools are like furniture: I live here, and a project is a guest. A guest does not bring their own furniture.

As a potential contributor, in terms of global tools, I don't want to:

1. Bloat my local with a special tool for a particular project
2. Learn a new tool for a particular project

Unless the new global tool is awesome and/or solves a very tangible problem my "current" doesn't. So then it's something I'd want to adopt myself (replacing my current).

---

@AugustinMauroy npm and yarn are sometimes (maybe even often?) compatible, but npm and pnpm are very not compatible.

[ovflowd]

Based on latest discussions within Slack, we wouldn’t be imposing a package manager. Yes, our documentation will still only mention npm, and npm install. But the git committed lockfile and CI would use pnpm. This ensures that new packages added or updated use pnpm, but allows teams and individual contributors to still completely only rely on npm.

[bjohansebas]

btw, even though we’re not enforcing a package manager, we will need to ignore the package-lock.json generated by npm in git.

[ovflowd]

Hey, @nodejs/website, I want your opinions if we should adopt pnpm or not. During some tests, I've been doing lately:

• Cold (without cache) installs of Node.js NPM packages takes 1~ minute
• Cold (without cache) installs of Node.js NPM packages with PNPM takes ~10 seconds
• Warm installs of Node.js NPM packages takes 25 seconds to 50 seconds
• Warm installs of Node.js NPM packages with PNPM takes ~5 seconds

This is a considerable enough change to speed up the deployment process, PR reviews, and all the numerous different GitHub Actions we have.

I wanted to ask your opinions on adopting pnpm or not. It is a good enough change and it brings other benefits (possibly on the long term)

[ruyadorno]

it should be install-strategy=linked iirc, it's pb still experimental though

[ovflowd]

@ruyadorno being on the edge is my family name

[ovflowd]

Well, yeah it didn't work 😅

npm WARN reify The "linked" install strategy is EXPERIMENTAL and may contain bugs.
npm WARN deprecated querystring@0.2.0: The querystring API is considered Legacy. new code should use the URLSearchParams API instead.
npm ERR! code 1
npm ERR! path /Users/cwunder/GitHub/Personal/nodejs.org/node_modules/.store/esbuild@0.17.19-lPWS9eySCA4LWj0y2ZksTQ/node_modules/esbuild
npm ERR! command failed
npm ERR! command sh -c node install.js
npm ERR! node:internal/errors:868
npm ERR!   const err = new Error(message);
npm ERR!               ^
npm ERR! 
npm ERR! Error: Command failed: /Users/cwunder/.nvm/versions/node/v18.13.0/bin/node /Users/cwunder/GitHub/Personal/nodejs.org/node_modules/.store/esbuild@0.17.19-lPWS9eySCA4LWj0y2ZksTQ/node_modules/esbuild/bin/esbuild --version
npm ERR! /Users/cwunder/GitHub/Personal/nodejs.org/node_modules/.store/esbuild@0.17.19-lPWS9eySCA4LWj0y2ZksTQ/node_modules/esbuild/bin/esbuild:1
npm ERR! ����

[ruyadorno]

that's a bummer, it looks like for some reason esbuild doesn't like the location it has been assigned by the linked install strategy 😅

[ovflowd]

Important to mention that installing NPM packages is taking around 2-3 minutes on Windows (GitHub Actions)

[ovflowd]

We don't use dependabot right now.

[bmuenzenmeyer]

https://github.blog/changelog/2023-06-12-dependabot-version-updates-now-supports-pnpm/

[bmuenzenmeyer]

I've found value in pnpm for dedicated app teams using the software day in and day out, switching branches, installing dependencies, etc.... but it's benefits add up the more you use it. I think what @mhdawson said is probably the overriding concern here: ease of initial contributon. Maybe corepack helps enough here, but that feels too new IMO.

Imposing a specific package manager is a very common practice

https://github.com/pnpm/only-allow can help here if we choose to codify npm (or otherwise)

[bmuenzenmeyer]

As an aside, I've yet to find a great way to pin a version of npm or pnpm to a project... nvm sorta works by virtue of defining a Node.js version and then assuming users take the bundled npm version.

Perhaps something like volta could help - but here we are again talking more tools in an environment that needs to be simple.

Project setup is all about determinism and simplicity to me, and I'm not sure anything meets that mark aside from good docs (and ...perhaps... nvm)

[bmuenzenmeyer]

packageManager exists for that

[ljharb]

you mean devEngines :-)

[avivkeller]

packageManager is also valid

[bjohansebas]

npm does not respect packageManager, so it's better to use devEngines instead

[avivkeller]

Hey, I realize this is two years old, so I’m not sure if it’s still relevant, but I wanted to flag a major concern.

If we convert this repository to pnpm, it would introduce pnpm as a requirement for importing (right?). That’s a significant change because:
1. This repo would be pulled into api-docs-tooling.
2. api-docs-tooling is then imported by Node core.
3. Node core currently doesn’t use pnpm, and requiring it would likely incur pushback.

Loading

graph TD;
    A["Node Core"] --> B["api-docs-tooling"];
    B --> C["This Repository"];
    C -.->|Converted to pnpm| D["pnpm Required"];

[aduh95]

There's no reason that would be the case, the pacakge manager only affects the package, not its dependents nor its dependencies.

[bmuenzenmeyer]

Two years have passed since this was first proposed:

Concerns in this discussion, addressed

Concern	Comment
Harder for newcomers to contribute	This is the big one, addressed below
Hard to pin versions of npm / pnpm	packageManager field now exists
no dependabot support	now supported
does this require using pnpm elsewhere in the chain	no

Does pnpm make the project harder to contribute casually to?

tl;dr - the project (Node.js, the website, all of it) is already complex. Let's stop pretending it isn't. I say this having wrote a book on approachable open source software and helped twice now support newcomers to contribute to the project during Grace Hopper. I understand the irony here, but mention this hopefully with a perspective of objectivity and credibility.

If you count tools as integers, this adds one. Everyone can agree with that. But I posit that the codebase is already in a state that requires acumen around a modern web development project, installing things on your local computers, etc, such that adding pnpm is not a straw that "breaks the camel's back."

• the codebase is already a monorepo containing two packages and one app
• the sophisticated data model requires understanding of dynamic release data, blog content, OS-specific installation methods, community vs official pathways, download snippets, code examples in ESM and CJS
• internationalization requirements segregate casual translation efforts into another tool entirely
• a custom layout renderer requires careful acknowledgement and care working around
• our tailwind component library uses an off-the-beaten path mechanism to apply and share styles
• toolchains like Sentry, Storybook, Next.js, MDX, eslint, TypeScript, etc are ecosystems in and of themselves

😅 ha, this might read as a monologue complaining about how complicated the site is, from a maintainer themself. there are reasons we use the things we do, and reasons we look to evolve over time.

Tooling like corepack can be integral to a future state that replaces:

   npm ci # installs this project's dependencies
   npm run dev # starts a development environment

note: we don't tell people what version of Node.js and npm to use (despite having an .nvmrc), which can be buggy.

with (or some variant we can discuss later, i know corepack itself is in flux)

   corepack enable # make all package managers available
   corepack install # use the project's same version of the package manager
   pnpm install # installs this project's dependencies
   pnpm dev  # starts a development environment

this can be written better with some nicer technical writing around project setup vs running.... or even use a makefile to help people on systems that support it.

I didn't even touch all the benefits already cited around install speed, disk space, etc. Those stand on their own.
My point being, if the main concern is complexity, pnpm alone doesn't change our complexity. We're already there friends.

[ovflowd]

Let's vote for pnpm!

The Polls below are meant for @nodejs/nodejs-website and @nodejs/web-infra members to signal their preferences on the adoption of pnpm and the specifics of each implementation detail. Votes outside of these groups will be noted, but ignored.

[ovflowd]

pnpm on our README.md and Documentation

• 👍: pnpm should be advertised within our repository's README and Collaboration Guide + Docs on how to install it Globally
• 👎: We should keep npm in our docs and and only mention pnpm on collaborator-specific docs
• 🚀: I'm neutral about this (fine with 👍 or 👎)

Winner: 👍

[ovflowd]

Git Ignore package-lock.json

• 👍: We should add to our .gitignore npm's package-lock.json
• 👎: Keep the file out of .gitignore
• 🚀: I'm neutral about this (fine with 👍 or 👎)

Winner: 👍

[ovflowd]

GitHub Actions CI

• 👍: Our CI should use pnpm
• 👎: Our CI shoduld not use pnpm
• 🚀: I'm neutral about this (fine with 👍 or 👎)

Winner: 👍

[ovflowd]

GitHub Actions Lockfile Checker Security Action

• 👍: Our CI should generate the pnpm lockfile for each Pull Request and inform if the computed lockfile differs the commited one / one present in the branch/ref
• 👎: Do not add a new CI step for this
• 🚀: I'm neutral about this (fine with 👍 or 👎)

Winner: 👍

[ovflowd]

devEngines/packageManager on package.json

• 👍: We should use devEngines on package.json
• 👎: We should use packageManagers on package.json
• 😕: We should use neither on package.json
• ❤️: We should use both devEngines and packageManager on package.json
• 🚀: I'm neutral about this (fine with 👍 or 👎)

Winner: ❤️

[AugustinMauroy]

I chatted on slack with Brian, here's what came out:
I'm not opposed to pnpm but I really wish we could contribute to the project using npm without any strange manipulation. So that the average contributor who just has node can contribute without blocking.

[styfle]

Our CI should generate the pnpm lockfile for each Pull Request and inform if the computed lockfile differs the commited one / one present in the branch/ref

I strongly recommend doing this since nearly every major version of pnpm (now at 10) changes the lockfile format. So a contributor who isn't use corepack or similar might be on an older version of pnpm and submit a PR with accidental lockfile changes.

[MattIPv4]

I do wonder how this will interact with Dependabot PRs etc. where the lockfile is being intentionally edited from what main has. I guess we'll want it to be a non-blocking check.