"yarn knit" -- a command to test libraries while working on them

[ide]

This is a proposal for a new command that is like yarn link/npm link and addresses several of the problems with link.

(This is an edited version of yarnpkg/yarn#1213, written as an RFC as requested.)

[timdorr]

What if yarn knit assigned a nonce version string and also spit out something like To install in a local app, run: yarn knit dep@asdf1234? That could be referenced in the lockfile with a resolved of knit://dep@asdf1234.

[ide]

I think that could work though one issue is that other team members wouldn't be able to resolve knit://dep@asdf1234. We could address that in a few ways:

1. Add a second yarn.local.lock file that is .gitignored and contains knitted dependencies and is overlaid on top of yarn.lock.
2. Commit knitted tarballs to Git. Yarn already supports offline installations.
3. Ignore it

[joshduck]

Symlinks have issues with peer dependencies. I would love to see that accounted for here.

Node resolves the link fully before walking up the path looking for node_modules. So if 'dep' has a peer depenency on React, it will not know that 'react' is actually app/node_modules/react.

[ide]

@joshduck Node has a flag called --preserve-symlinks which turns off link resolution. And the React packager and Webpack could consider a different resolution policy.

I'm not sure it's possible without symlinks to let people make changes in their library's directory and have the same changes automatically applied to their app's node_modules. There is a program called wml that watches for changes and copies files over but it relies on Watchman and doesn't work in some scenarios where symlinks would.

[bestander]

Sounds like a good idea

[bestander]

Considering that it is a brand new API, we might want to get feedback after we have the first implementation.
Any idea how we can implement it behind a flag maybe for a couple of releases?

[ide]

We could name the command "knit-EXPERIMENTAL" which has a similar effect as a flag without adding much complexity. As for getting people to test it out, Exponent is already using a copy of Yarn from upstream directly so it's relatively straightforward for us to use an unreleased version of Yarn.

[bestander]

Agree, great idea!

[louisgv]

Is this out yet?

[KyleAMathews]

I'd love to start developing Gatsby with this. It'd lower complexity getting started a lot with contributing to our Lerna monorepo setup.

[kristojorg]

Agreed, would love to see this happen

[bestander]

I would hold on implementing knit as described here, I think Yarn workspaces could cover all the usecases described here.

[bestander]

Workspaces should become a good end to end solution for monorepos, in Workspaces phase 3 RFC we discuss linking between modules and it might be good enough to cover use cases raised here.

Thanks, @ide, for writing down this RFC, I got some ideas from it.

[sejoker]

After reading both RFCs on yarn knit and workspaces linking I tend to think they have different purpose and don't substitute one another. yarn knit is yarn link on steroids with a focus on the development experience. Workspaces on the other hand focus on the API while working in monorepo. I don't see how Workspaces address the main features of yarn knit:

• Isolating node_modules correctly
• Working on multiple versions

All in all, I find current RFC a good addition on top of yarn link even with rolled out Workspaces feature.

[saulshanabrook]

We had to write a custom linking mechanism in JupyterLab because if we used yarn link we ended up with duplicate packages in the webpack build. Basically we have a webpack build in one directory and then want to let users add their own folder (anywhere on the file system) and have that included in the webpack build and updated when the user updates their folder. We can't use workspaces for this since they are required to be subfolders of the root. And we can't use link, because then we end up with dependencies duplicated.