build(all): switch from yarn to pnpm

[eventualbuddha]

I tried to get all the apps and libraries under a single yarn workspace and found I could not do it. This message describes the difficulties I had. Closes #75.

"There might be a problem with the project dependency tree."

react-scripts does some preflight checks to ensure that there isn't another resolvable copy of some of the packages it needs that might conflict with the versions it wants. When it finds one, it bails with an error message.

Trying to put multiple react-scripts-using projects into the same yarn workspace means that yarn will hoist shared dependencies, such as babel-eslint, to the root. Thus, these packages become visible to the preflight check and it aborts. To work around this, you can use the nohoist option to tell yarn which packages not to hoist. This sort of worked, but there were subtle issues remaining.

Versions changing when adding to the workspace

I got BMD and ballot-encoder into the workspace together, but when I added election-manager it broke BMD. I still don't know exactly why, but I believe something changed the versions of some packages that BMD was using in a way that caused the tests to fail.

Incompatible type declarations

Sometimes multiple packages reference the same types, i.e. for a mutual dependency. However, sometimes they'd end up using different (and subtly incompatible) versions of the same @types/ package. When this happened, the best recourse was to use another yarn workspaces feature: resolutions. This allows you to configure what version packages should resolve to. This fixed most of the type incompatibility issues, but I suspect it caused another issue.

yarn resolved an older copy than requested

I noticed some crashes due to an API incompatibility. Specifically, one of the @babel/ packages required another one in the ^7.12.0 range, but yarn hooked it up with 7.8.6. Why? I don't know. Perhaps it was the resolutions stuff. I was only able to work around this by finding the entry in the yarn.lock file, deleting it, and re-running yarn.

Trying out alternatives

I tried both npm v7 and pnpm. npm v7 has support for workspaces similar to yarn, but does not yet have support for configuring hoisting behavior. Lacking hoisting control meant it hoisted everything it could, which made react-scripts angry. Thus I couldn't use npm v7.

Previous versions of pnpm did not hoist anything, and still was able to share dependencies by symlinking. Though it does hoist by default now, that behavior can be turned off. So I added hoist = false to .npmrc and then started trying to run everything to determine what would break without hoisting. It turns out that a good number of packages have implicit dependencies that hoisting allowed them to access, but that could not be accessed in a non-hoisted node_modules layout.

pnpm has a hook called readPackage that allows altering the package data for any npm package using any custom logic you want. This allows us to act as if these packages with implicit dependencies actually had those dependencies listed explicitly. In my testing, this has yielded a good balance of correctness and functionality.

So pnpm seems to work and I think is flexible enough to meet our needs going forward.

[eventualbuddha]

Here's a quick summary of differences I've discovered through usage in the pnpm and yarn CLIs:

Command	yarn	pnpm
Install dependencies	yarn	pnpm i
Add react to dependencies	yarn add react	pnpm install react
Add eslint to devDependencies	yarn add -D eslint	pnpm install -D eslint
Run script test	yarn test	pnpm test
Run script lint with args	yarn lint --fix	pnpm lint -- --fix
Run package binary tsc	yarn tsc --build	pnpx tsc --build

[benadida]

• we need a basic README at the top level now, I think
• should include how to install pnpm the right way.
• ignore .eslintcache files.
• Election Manager not working with make run

[benadida]

npm, not pnpm? When to use which?

[eventualbuddha]

This is for "external" consumers of the published library, so I think it makes sense to use the canonical package manager in the docs here.

[eventualbuddha]

• we need a basic README at the top level now, I think

Got a really basic one up (preview)

• should include how to install pnpm the right way.

I put this in the README, though it's a little buried.

• ignore .eslintcache files.

Done.

• Election Manager not working with make run

Should be fixed.

[carolinemodic]

Played around a bit and things mostly worked as expected except election-manager. When I run yarn start in apps/election-manager I get:

src/AppRoot.tsx
  Line 67:10:  Replace `⏎····electionDefinition,⏎····setElectionDefinition,⏎··]·=·useState<ElectionDefinition` with `electionDefinition,·setElectionDefinition]·=·useState<⏎····ElectionDefinition⏎··`  prettier/prettier

src/lib/votecounting.ts
  Line 458:38:  Replace `⏎····election.contests⏎··).reduce<ContestTallyMetaDictionary` with `election.contests).reduce<⏎····ContestTallyMetaDictionary⏎··`  prettier/prettier

When I make run instead it seems to run ok but when I launch kiosk-browser to localhost:3000 it doesn't load anything.

[eventualbuddha]

Thanks @carolinemodic, I think 3c11e45 fixes it. Try again?

[carolinemodic]

Hmm I was still getting the error but then did a rm -rf of node_modules and redid pnpm install and now election-manager is running fine so... not sure if the changes in 3c11e45 changed anything for me but it does seem to work ok now!

[beausmith]

Running Locally. LGTM.