Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Emit lockfile v2 and fix bin links with NPM v7+ #302

Merged
merged 1 commit into from
Sep 28, 2022

Conversation

lilyinstarlight
Copy link
Contributor

@lilyinstarlight lilyinstarlight commented Sep 13, 2022

Lockfile v2 mostly just has a bit of extra metadata and all dependencies are hoisted to the top-level with path-specific keys in a new lock value called "packages". This update emits enough of the format that NPM v7+ seem to be happy enough with it and does not try to rewrite it and cause ENOTCACHED errors with the sandbox.

As of NPM v7+, it no longer links bins for the top-level project automatically unless a global install is selected1 (and is not a problem specific to node2nix2). Given a global install would cause more problems than it would solve, I added a simple script to perform the linking ourselves and instructed npm install to never link them for consistency.

Additionally, this adds a postRebuild hook that can be used by packages to run extra build scripts as needed by the project before the npm install step prunes dev dependencies (also seems to be a new behavior of NPM v7+, but admittedly this behavior could be due to the fake lock file confusing it a little)

Closes #236, closes #293, closes #294

Edit: Marking as draft until I can get this to work completely with the nodePackages attrset in nixpkgs

@lilyinstarlight
Copy link
Contributor Author

Okay, I think I've got this rebuilding the nodePackages attrset without regressions with Node.js 18 (after fixing a few specific packages that hadn't been rebuilding properly before the update but still "built")

This should be ready for review/merge and I will be making a companion PR in nixpkgs in a few that brings the nodePackages attrset up to using the latest LTS rather than being pinned to nodejs-14_x

Lockfile v2 mostly just has a bit of extra metadata and all dependencies
are hoisted to the top-level with path-specific keys in a new lock value
called "packages". This update emits enough of the format that NPM v7+
seem to be happy enough with it and does not try to rewrite it and cause
ENOTCACHED errors with the sandbox.

As of NPM v7+, it no longer links bins for the top-level project
automatically unless a global install is selected[1][2]. Given a global
install would cause more problems than it would solve, I added a simple
script to perform the linking ourselves and instructed `npm install` to
never link them for consistency.

Closes svanderburg#236, svanderburg#293, svanderburg#294

[1]: npm/cli@e46400c#diff-24c01909dabbe2fc000fb5b43d14b511fb335b2f0c2e8e7a671f7d567a33d577R17-R18
[2]: npm/cli#4308
@lilyinstarlight
Copy link
Contributor Author

Opened PR in nixpkgs as NixOS/nixpkgs#193337

@svanderburg svanderburg merged commit a6041f6 into svanderburg:master Sep 28, 2022
@svanderburg
Copy link
Owner

Ok, nice contribution! I'll merge it

@lilyinstarlight lilyinstarlight deleted the fix/npmv7 branch September 28, 2022 20:40
@CMCDragonkai
Copy link

What version of node2nix is coming out as?

ryuheechul added a commit to ryuheechul/dotfiles that referenced this pull request Oct 1, 2022
lilyinstarlight added a commit to lilyinstarlight/nixpkgs that referenced this pull request Mar 7, 2023
This is minor fallout from svanderburg/node2nix#302 which made node2nix
handle installing script bins itself (since npm stopped doing that
correctly).

It seems npm was doing line-ending normalization when installing these
files itself, so in addition to making all bins executable we now also
do a crlf to lf conversion on them if they are a script with a shebang.
github-actions bot pushed a commit to NixOS/nixpkgs that referenced this pull request Mar 9, 2023
This is minor fallout from svanderburg/node2nix#302 which made node2nix
handle installing script bins itself (since npm stopped doing that
correctly).

It seems npm was doing line-ending normalization when installing these
files itself, so in addition to making all bins executable we now also
do a crlf to lf conversion on them if they are a script with a shebang.

(cherry picked from commit b85c1e3)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants