[RRFC] Multi-app Monorepo Support

[mysterycommand]

Before Opening Please...

• Search for an existing/duplicate RRFC which might be relevant to your RRFC

Motivation ("The Why")

Please see detailed description in #462, but tl;dr I'd like npm to "natively" support installing a package workspace's npm pack'd artifact into an app workspace's local node_modules folder for production deployment without a bunch of extra prerelease/registry rigmarole. In a sense treat the monorepo/workspace root as a kind of ephemeral local registry

Example

A monorepo with three workspaces, two apps "a" and "b", one package "c". Apps "a" and "b" depend on package "c", and are deployed to different locations (Docker, AWS Lambda, Google Cloud Functions, etc). Following best practices, minimizing build tooling, and wanting to ship a lean, production-only, app artifact, I'd like to an app structure like this (the dist folder is just meant to represent some production-ready code, this is just to show a folder structure that expresses the relationship between packages):

$ tree apps/a --dirsfirst
apps/a
├── dist
├── node_modules
│   └── @my
│       └── c
│           ├── dist
│           └── package.json
└── package.json

With this structure I'd zip the apps/a folder and push it to AWS, or COPY it into a Docker image in my Dockerfile, etc.

How

Current Behaviour

This structure can be roughly achieved by running these commands:

npm pack -w @my/c
npm pack -w @my/c --pack-destination=apps/a
npm i my-c-1.0.0.tgz -w @my/a
rm my-c-1.0.0.tgz apps/a/my-c-1.0.0.tgz

Note:

I'm not sure why it's necessary to pack it twice, but something about running npm i my.tgz -w @my/a needs the tarball to exist in both the root and the workspace. Maybe a separate bug about resolving install paths when -w is set?

Also worth noting this changes apps/a/package.json's "@my/c" dependency from "^1.0.0" to "file:my-c-1.0.0.tgz" which I have to immediately undo in order to maintain the dev-time hoisted dependency functionality I want.

Desired Behaviour

Allow a way to specify that npm install and/or npm ci should look to the workspaces in the current monorepo before reaching out to a registry. If a match is found call npm pack on that workspace and "serve" the resultant tarball (or its contents). Some suggestions for what this'd look like in the cli:

1. npm i --only=prod --prefer-workspaces -w @my/a
2. npm i --only=prod --prefer-local -w @my/a
3. npm i --only=prod --registry=file:///. -w @my/a
   • this one's definitely the weirdest, it'd require npm to detect that the "registry" at that file URI is a monorepo/workspace-root, but has the advantage of not adding a new flag to the cli, and maybe being configurable via .npmrc?

It should work for npm ci as well.

References

• Initial discussion: Multi-app Monorepo? #462
• Why I think Lerna doesn't already solve this use case: Multi-app Monorepo? #462 (comment)
• Why I think file/folder references are also unsatisfactory: Multi-app Monorepo? #462 (reply in thread)

[mysterycommand]

Another note about file/folder references (in addition to my previous comments): running npm i -S ./packages/c -w @my/a given the above-described monorepo results in this error:

npm ERR! Cannot set property 'dev' of null

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/matt/.npm/_logs/2021-09-24T17_01_32_762Z-debug.log

Though it does seem to sort of work, it actually creates a symlink from apps/a/node_modules/@my/c to package/c, and I believe this process will also hoist any dependencies of package "c" to the root ... neither of which is hat I want for a deployable "app artifact"

[isaacs]

Action items:

• land npm-packlist in cli (@wraithgar)
• RFC about running prepare scripts for linked bundled deps
• RFC about workspace layout (@ljharb)

[everett1992]

I think this functionality extends beyond monorepos. To create a lambda archive of my module's files and it's production dependencies I have to

# run lifecycle hooks and create tarball of files respecting `files` and .npmignore
archive=$(npm pack | tail -n 1)
# extract files into scratch directory (in this case using the package/ wrapper)
tar xzvf "$archive" && cd package
# install prod dependencies.
npm install