-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Upgrading to node 16.x LTS (and pkgs.nix upgrade to 21.11 or later) #37
Conversation
CodeSee Review Map:Review in an interactive map View more CodeSee Maps Legend |
Note that the current |
@tegefaulkes relevant for our soon to be release. |
Will need to test master of nixpkgs, not 21.11. |
Node 16 has Promise.any and AggregateError too. This should be ideally used in our ICE of direct connection and signalling. https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/any |
With the following changes I've got
|
Make sure that the version used by PKG fetch is the same version of nodejs used in nix-shell. Or at the very least a compatible version. The hashes would have to be updated. |
And test that AbortSignal exists and AggregateError exists. |
They're both using 16.14.2, and I've updated the hashes as well. AbortSignal and AggregateError also both exist, as well as Promise.any. |
With all of the changes from this comment #37 (comment) both building the releases
And installing into Nix user profile
Work without issues. However, I'm getting errors from attempting to build the package
And installing into Docker
|
After doing this:
Are you able to run the executable as Note that you should try just doing I read somewhere that node2nix has a problem producing the final executable right now, and that could be causing the docker image problem. |
Btw It now needs to be |
Can you update the README.md with the new command. |
If the bin executable is missing. See if this issue is the culprit: NixOS/nixpkgs#145432. Furthermore, remember to update the task list for this PR with your changes. |
Remember to push up your changes so I can run it too. |
And if the bin executable is truely missing, then you can try to debug why inside the # copy the dist
cp -r ${utils.node2nixDev}/lib/node_modules/${utils.node2nixDev.packageName}/dist $out/lib/node_module>
# copy over the production dependencies
if [ -d "${utils.node2nixProd}/lib/node_modules" ]; then
cp -r ${utils.node2nixProd}/lib/node_modules $out/lib/node_modules/${utils.node2nixDev.packageName}/
fi
# create symlink to the deployed executable folder, if applicable
if [ -d "${utils.node2nixDev}/lib/node_modules/.bin" ]; then
cp -r ${utils.node2nixDev}/lib/node_modules/.bin $out/lib/node_modules/
ln -s $out/lib/node_modules/.bin $out/bin # <--------- this is what should be producing the final bin link
fi |
Running |
Exactly, can you use So the bin executable isn't being produced. This is the thing that people have been mentioning for Can you debug |
For reference, this is what the current master shows:
Actually |
Then you post an issue upstream on node2nix (the issue I linked was on nixpkgs). |
Digging into this a bit more, it looks like running any of
But running either of the docker-related commands completely breaks the symlink:
So I don't think the problem is only with |
And what is in side |
Nothing, it doesn't exist |
Ok let's see what's inside the result tomorrow. But you may need to go into node2nix generated nix files and see what's going on with the generated package instructions. |
Looks like the problem might be that pkgs.nix hash I'm using isn't using node2nix 1.10.0 properly. Even though it says it's using 1.10.0, when running |
So unfortunately it looks like even master is still using 1.9.0 https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/node-packages/default.nix#L261 |
I've compared the old 1.9.0 to 1.11.0. I suspect that something changed with Reported this upstream: svanderburg/node2nix#293 While upstream figures out why. The workaround right now is to directly refer to these executables under our package and just create those symlinks in
To instead create symlinks for every file in: Or extract this information from Or continue figuring out how to make |
The closest thing I can get to is replacing
However this doesn't work, it has a weird error about not finding the Instead However this doesn't produce the links the place we want to have it. Partly because I think our Another alternative is to use:
In the
|
Given that there's no elegant solution atm, I've decided to go with some duct tape until upstream fixes the problem. In the This works well enough for now, as the previous system was already kind of weird, relying on Future solutions may involve redoing the node2nix derivations to be more direct to our infrastructure. |
Docker builds has a problem, but nothing to do with this repo, it appears nodejs itself doesn't work inside docker containers. However it's good idea for others to try @tegefaulkes to see if it's just my laptop right now. Reported upstream: NixOS/nixpkgs#170279 |
@emmacasolin 5.6.0 of
I wonder why this is the case. Btw if you want ot make sure to get the right hash, use |
Easy fix, just had to specify the node range in the |
80276de
to
ad3d86e
Compare
* Updated pkgs.nix to a5774e76bb8c3145eac524be62375c937143b80c * Updated node2nix to 1.11.0 and loading directly from GitHub * Updated pkg to 5.6.0 and using pkg-fetch 3.3 base binaries * Updated to TypeScript 4.5+ and eslint to ^5.0.0 * Updated @types/node to 16.11.7 * Updated node-gyp-build to 4.4.0 * Updated typedoc to 0.22.15, the .nojekyll generation is automatic * Added dist/**/*.json to pkg assets * Changed to target ES2021 as node 16 supports it * Bin executable duct-tape in default.nix because node2nix no longer generates bin executables
Everything seems to be working now, EXCEPT the docker builds... will wait and see. |
I've tested the linux executable on Ubuntu so it works. Going to merge this as CI/CD will be able to test on docker and macos builds and windows builds. The docker problem might be just my computer. Let's see. |
I can't seem to build or nix-shell this. I get the error
SO i'll have to check my AWS access. But I don't have this problem with the js-polykey repo. |
It could be that our NixOS has changed something with respect to aws access. That's also about a key id, not the AWS authentication key. Unfortunate since you'll need to update your NixOS to be the same as all of us, but there's a problem with the current NixOS version we are on and running nodejs inside a docker container. |
Trying to run the docker images give me
|
For now since it works in CI/CD and in ubuntu, we should conclude that our docker image is correctly built. But NixOS docker runtime has some misconfiguration/bug. So we will continue as is, until upstream figures out what's going on in our NixOS docker runtime. |
Description
Node 14 is already EOL. We should be upgrading to node 16.x LTS. This introduces some major features that we want in NodeJS:
This includes some other changes:
pkgs.nix
to 21.11 (or later/master) which hasnodejs-16_x
butnodejs
is still14.19.1
(we may need to go to master insteadnode2nix
at1.10.0
which supports16.x
, currently1.9.0
does not support16.x
pkg
to5.6.0
, which requires an update to ourpkgBuilds
inutils.nix
. As it would no longer be using3.1
and no longer using14.17.0
and upgrade to v16@types/node
to 16.x as wellIssues Fixed
Tasks
@types/node
from14.14.35
to16.11.7
pkg
from5.3.0
to5.6.0
typescript
from4.1.3
to4.5.2
pkgs.nix
to NixOS/nixpkgs@a5774e7ES2020
toES2021
pkgBuilds
from3.1
to3.3
with node version16.14.2
@typescript-eslint/eslint-plugin
and@typescript-eslint/parser
from4.12.0
to5.0.0
node-gyp-build
from4.2.3
to4.4.0
[ ] 10. Remove leveldown patch for- this did not work, and also the package.json patch did not work, this is kept the way it ispkg
, and replace it with--no-dict leveldown
option as specified here: Fix warnings when packaging with leveldown. vercel/pkg#1273Final checklist