build: bundle the npm CLI itself for release

[zkat]

This makes it so we can bundle the npm CLI into a handful of files, instead of hundreds of thousands, with some advantages:

• Smaller distribution and installed size
• Faster use through npx
• We'll be able to use scoped dependencies again (currently only needed for legacy upgrades)

So far, I've tried browserify, rollup, and even parcel, and webpack seems to be the closest thing to working, but it's not quite there yet.

Still TODO:

• There are a number of dynamic requires throughout npm and its dependencies. I'm not sure how to resolve this.
• On that topic, update-notifier uses require-lazy for its dependencies, and that doesn't get picked up by webpack at all.
• There are three workers that get loaded through child_process.spawn/exec. Those need to be standalone entry points. The workers are the extraction worker and the metrics worker on the npm installer, and the extraction worker in libcipm (a dependency).
• It's unclear whether this will seriously impact startup performance. Big slowdowns are no good.

[Daniel15]

It's unclear whether this will seriously impact startup performance.

In Yarn we actually had a improvement overall when we switched to a JS bundle + v8-compile-cache. Fewer files to read means fewer stat() calls which means less disk activity. It also significantly reduced the amount of code in our downloads, as modules tend to have a lot of junk (tests, docs, unused files) that are all excluded when you build a bundle. 😃

[mkg20001]

Maybe creating a fork of https://github.com/zeit/pkg that would create a js-bundle instead of an executable would be an idea. pkg already does things such as creating a virtual filesystem, mocking child_process.fork(), etc. so this would probably be the best solution that requires the least changes to npm itself (if deps are missing or not getting picked up by pkg one only has to add a glob to the pkg.scripts field in the package.json instead of creating a more complex build process). also it would be a lot more reusable.

[zkat]

As it stands, this feels like a fool's errand and a massive time sink. I think we should give it another whirl in the future when npm's gotten refactored a bit more and split into more modern/webpackable components (and maybe webpack has time to mature its node-building support?).

Until then, I'm gonna close this. I'll leave the branch up on the repo for future reference in case another adventurous soul decides this would be worth their time. In the meantime, I have way better things to do and this won't get any attention from me.