Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds documentation for PnP #907

Merged
merged 1 commit into from
Jan 24, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions _data/guides.yml
Original file line number Diff line number Diff line change
Expand Up @@ -236,6 +236,17 @@
- id: docs_workspaces
path: /docs/workspaces

- id: docs_plugnplay
title: docs_plugnplay_title
description: docs_plugnplay_description
pages:
- id: docs_plugnplay_overview
path: /docs/pnp
- id: docs_plugnplay_getting_started
path: /docs/pnp/getting-started
- id: docs_plugnplay_troubleshooting
path: /docs/pnp/troubleshooting

- id: yarn_organization
title: yarn_organization_title
description: yarn_organization_description
Expand Down
9 changes: 9 additions & 0 deletions _data/i18n/en.yml
Original file line number Diff line number Diff line change
Expand Up @@ -218,6 +218,15 @@ docs_workspaces_title: Workspaces
docs_workspaces_description: |
Link together your projects for easier maintenance.

docs_plugnplay: "Plug'n'Play"
docs_plugnplay_title: "Plug'n'Play"
docs_plugnplay_description: |
Install your projects in a safer and faster way.

docs_plugnplay_overview: Overview
docs_plugnplay_getting_started: Getting Started
docs_plugnplay_troubleshooting: Troubleshooting

yarn_organization_title: Yarn Organization
yarn_organization_description: |
The Yarn organization is a collaboration of many companies and
Expand Down
23 changes: 23 additions & 0 deletions lang/en/docs/pnp.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
---
id: docs_plugnplay_overview
guide: docs_plugnplay
layout: guide
---

{% include vars.html %}

Plug'n'Play is an alternative installation strategy unveiled in September 2018. It presents interesting characteristics that make suitable for a large panel of projects, and is designed for compatibility with the current ecosystem.

The way regular installs work is simple: Yarn generates a `node_modules` directory that Node is then able to consume. In this context, Node doesn't know the first thing about what a package is: it only reasons in terms of files. "Does this file exist here? No? Let's look in the parent `node_modules` then. Does it exist here? Still no? Too bad... parent folder it is!" - and it does this until it matches something that matches one of the possibilities. That's vastly inefficient.

When you think about it, Yarn knows everything about your dependency tree - it evens installs it! So why is Node tasked from locating your packages on the disk? Why don't we simply query Yarn, and let it tell us where to look for a package X required by a package Y? That's what Plug'n'Play (abbreviated PnP) is. Instead of generating a `node_modules` directory and leaving the resolution to Node, we now generate a single `.pnp.js` file and let Yarn tell us where to find our packages. Doing this provides a lot of benefits:

- The `node_modules` directory contains a gargantuan amount of files. Generating it makes up for more than 70% of the time needed to run `yarn install` with an hot cache. Because the copy is I/O bound, it's not like package managers can really optimize it either - we can use hardlinks or copy-on-write, but even then we still need to make a bunch of syscalls that slow us down dramatically.

- Because Node has no concept of "package", it doesn't know whether a file is _meant_ to be accessed, on top of being available. It's entirely possible that a code you write work in development but break in production because you forgot to list one of your dependencies in your `package.json` - and you won't know it until it becomes a problem and make you lose a day investigating the issue.

- Even at runtime, the Node resolution needs to make a bunch of `stat` and `readdir` calls in order to figure out where should a resolution end up. It's extremely wasteful, and is part of the reason why booting a Node application takes so much time - before even starting executing it, Node has to spend its time querying the filesystem for information that Yarn could have given him already.

- Finally, the very design of the `node_modules` folder is impractical in that it doesn't allow to dedupe packages as efficiently as one would hope. Because two packages with the same name but different versions cannot coexist in the same directory, we can't guarantee a perfect hoisting. Similarly, because the `node_modules` are deeply nested in a way that depend on the project dependencies, they cannot be shared from one project to the other.

All those problems and more are solved by Plug'n'Play.
29 changes: 29 additions & 0 deletions lang/en/docs/pnp/getting-started.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
---
id: docs_plugnplay_getting_started
guide: docs_plugnplay
layout: guide
---

{% include vars.html %}

Getting started with Plug'n'Play isn't difficult - at its basis it just involves enabling one tiny settings in your `package.json` file: `installConfig.pnp`.

```json
{
"installConfig": {
"pnp": true
}
}
```

From now on each time you'll run `yarn install` Yarn will create a single file named `.pnp.js` instead of the `node_modules` megafolder. You can try it right now by running `yarn --pnp` in your project, which will enable the settings and run the install in the same pass!

So enabling PnP isn't complicated at all - what might be an issue are third-party packages that reimplement the Node resolution themselves. Three major implementations exist, more might also hide from a project to another:

- [`resolve`](https://yarnpkg.com/en/package/resolve) is the main one, and is supported out-of-the-box thanks to the help of [Jordan Harband](https://github.com/ljharb). Every package using `resolve` (and that includes things you might have heard of like Babel or Gulp) now works with further configuration.

- [`enhanced-resolve`](https://yarnpkg.com/en/package/enhanced-resolve) is the second biggest one. You probably use it but might never have heard about it: it's the resolver used by Webpack. We do support this resolver through the [pnp-webpack-plugin](https://github.com/arcanis/pnp-webpack-plugin) addon.

- TypeScript uses its own resolver as well. In this case the situation is a bit more complex - the TS team has some concerns about [allowing third-party hooks](https://github.com/Microsoft/TypeScript/issues/18896) inside the `tsc` compiler, meaning that we can't work with it at the moment. That being said, TypeScript isn't only `tsc` and as such we've been able to add PnP support to the popular [`ts-loader`](https://yarnpkg.com/en/package/ts-loader) - meaning that as long as you compile your TypeScript through Webpack, everything works well! Consult the dedicated [section](https://github.com/arcanis/pnp-webpack-plugin#ts-loader-integration) about it for more information.

In case you find something not working, consult the Troubleshooting section for some tips, then open an issue if things still are problematic. We really want PnP to become the de-facto installation strategy for Javascript packages and finally put `node_modules` behind us once and for all.
31 changes: 31 additions & 0 deletions lang/en/docs/pnp/troubleshooting.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
---
id: docs_plugnplay_troubleshooting
guide: docs_plugnplay
layout: guide
---

{% include vars.html %}

While we do our best to make Plug'n'Play a delightful experience, sometimes things might go wrong. The following page describes some common scenario one should be aware of, and provides some guidance to fix them.

###### `<name> is trying to require <name> without it being listed in its dependencies` <a class="toc" id="toc-is-trying-to-require-without-it-being-listed-in-its-dependencies" href="#toc-is-trying-to-require-without-it-being-listed-in-its-dependencies"></a>

This error simply means that the specified package is requiring something without explicitly declaring it in its dependencies. Since this behavior is unsafe and relies on the hoisting being done a certain way, Plug'n'Play doesn't allow it.

That being said, it might happen that the use is pseudo-legitimate. For example, Jest does something similar to `` require(`jest-environment-${config.environment}`) `` - in this circumstances the intent is obviously to use the package as provided by the package setting up the package configuration. Still, this is an unsafe behavior that must be fixed in one of three ways:

- Either the code has to be modified so that the user becomes responsible for calling `` require.resolve(`jest-environment-jsdom`) `` (and then Jest can simply do `require(config.environment)`)

- Or the code has to be modified in order to use the `paths` option from `require.resolve`: `` require(require.resolve(`jest-environment-${config.environment}`, {paths:[config.projectPath]})) `` (note that this is the recommended option, being unintrusive for users and a good idea in general)

- Or, finally, the `jest-environment-jsdom` package can be specified as dependency of the top-level package - usually your own. In those circumstances, Yarn will allow Jest to access it even if it isn't declared in its dependencies (we do this because this is the only case where this is safe: it isn't possible for the top-level dependencies to be ambiguous).

The third option is typically meant as a way to unblock yourself, but please report it on the repository of the affected package and link them to this documentation (feel free to ping [@arcanis](https://github.com/arcanis) as well so that we can track those problems).

###### `Couldn't find a suitable Node resolution for unqualified path <path>` <a class="toc" id="toc-couldn-t-find-a-suitable-node-resolution-for-unqualified-path" href="#toc-couldn-t-find-a-suitable-node-resolution-for-unqualified-path"></a>

This error message means that Plug'n'Play was able to locate the _package_ part of the require (so for example it understood that `foo` should be resolved into `/usr/cache/yarn/foo-1.0.0`), but couldn't resolve the _file_ part of the resolution (to use the same example, it couldn't find an `index.js` file in `/usr/cache/yarn/foo-1.0.0/`). This problem typically isn't caused by Plug'n'Play itself, but rather by your application requiring something that doesn't exist (under non-PnP mode you'd likely have a generic "Cannot find module" error).

###### `Cannot find module <require-path>` <a class="toc" id="toc-cannot-find-module" href="#toc-cannot-find-module"></a>

_This error message is not generated by Plug'n'Play._ Ever. If you get it, it very likely means that your application is not running with the PnP resolver, meaning that your require calls won't be able to load files from your dependencies (since the `node_modules` needed for the regular Node resolution won't have been generated). Make sure to either run your code using `NODE_OPTIONS="--require /path/to/.pnp.js"` or to call it using `yarn node` which does the right thing transparently (whether you're using PnP or not).