Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ui dependencies in package-lock.json missing resolved/integrity fields #2299

Open
sarcasticadmin opened this issue Mar 25, 2024 · 2 comments · May be fixed by #2300
Open

ui dependencies in package-lock.json missing resolved/integrity fields #2299

sarcasticadmin opened this issue Mar 25, 2024 · 2 comments · May be fixed by #2300
Labels
security Related to security

Comments

@sarcasticadmin
Copy link

Wave SDK Version, OS

Im attempting to package up waved and components from v1.1.1 tag. My system is using the following versions:

$ node --version
v18.19.1
$ npm --version
10.2.4
$ nixos-version
23.11.20240117.8bf65f1 (Tapir)

Actual behavior

serveral ui dependencies are missing integrity and resolved fields. Some examples https://github.com/h2oai/wave/blob/5fd6aa5447859e9266ed42a6f9b0583281e62c63/ui/package-lock.json#L17944-L17946 and https://github.com/h2oai/wave/blob/5fd6aa5447859e9266ed42a6f9b0583281e62c63/ui/package-lock.json#L17994-L17997

These missing fields causes the nix build process to fail since it prefetchs and verifies each of the dependencies then performs a npm build offline in a sandbox to ensure reproducibility. This is the result of nix-build against v1.1.1 for the ui:

$ nix-build
this derivation will be built:
  /nix/store/0hj7a05h8kvj8h5zhmac393242nn4pf6-waved-ui.drv
building '/nix/store/0hj7a05h8kvj8h5zhmac393242nn4pf6-waved-ui.drv'...
Running phase: unpackPhase
unpacking source archive /nix/store/fm6brwgw7sq4dx1hafdl67ra9najqcq0-source
source root is source/ui
Running phase: patchPhase
Executing npmConfigHook
Configuring npm
Validating consistency between /build/source/ui/package-lock.json and /nix/store/2mq0dhwczf70pw9iz356ffdk6wmvi3xm-waved-ui-npm-deps/package-lock.json
Installing dependencies
npm ERR! code ENOTCACHED
npm ERR! request to https://registry.npmjs.org/wrappy failed: cache mode is 'only-if-cached' but no cached response is available.

npm ERR! Log files were not written due to an error writing to the directory: /nix/store/2mq0dhwczf70pw9iz356ffdk6wmvi3xm-waved-ui-npm-deps/_logs
npm ERR! You can rerun the command with `--loglevel=verbose` to see the logs in your terminal

ERROR: npm failed to install dependencies

Here are a few things you can try, depending on the error:
1. Set `makeCacheWritable = true`
  Note that this won't help if npm is complaining about not being able to write to the logs directory -- look above that for the actual error.
2. Set `npmFlags = [ "--legacy-peer-deps" ]`

error: builder for '/nix/store/0hj7a05h8kvj8h5zhmac393242nn4pf6-waved-ui.drv' failed with exit code 1;
       last 10 log lines:
       > npm ERR! Log files were not written due to an error writing to the directory: /nix/store/2mq0dhwczf70pw9iz356ffdk6wmvi3xm-waved-ui-npm-deps/_logs
       > npm ERR! You can rerun the command with `--loglevel=verbose` to see the logs in your terminal
       >
       > ERROR: npm failed to install dependencies
       >
       > Here are a few things you can try, depending on the error:
       > 1. Set `makeCacheWritable = true`
       >   Note that this won't help if npm is complaining about not being able to write to the logs directory -- look above that for the actual error.
       > 2. Set `npmFlags = [ "--legacy-peer-deps" ]`
       >
       For full logs, run 'nix log /nix/store/0hj7a05h8kvj8h5zhmac393242nn4pf6-waved-ui.drv'.

This seems to be an outstanding issue within npm if the packages are already cached locally and npm install is used to generate the package-lock.json : npm/cli#4460 and npm/cli#6301

Expected behavior

In order to assure reproducible installations, every package listed in a package-lock.json should contain a resolved and integrity field so it can be fetched from a registry.

Steps To Reproduce

Unsure how to reproduce this within npm, it seems to happily proceed without caring about the missing resolved and integrity fields. Here is a simple version to verify using nix

  1. Install nix: https://nixos.org/download/
  2. Copy the following into default.nix:
{ pkgs ? import
    (fetchTarball {
      url = "https://github.com/NixOS/nixpkgs/archive/d3f8923899a36a1985b58804de2b0cb0862015cd.tar.gz";
      sha256 = "sha256:0m2jf3chas1dls1gk6q2dg8q9ciq8mmbxbbmpfi5hjj8wd09a5kx";
    })
    { }
}:
let
  version = "1.1.1";
  src = pkgs.fetchFromGitHub {
    owner = "h2oai";
    repo = "wave";
    rev = "v${version}";
    sha256 = "sha256-fINuoJx7dPN613wLLzcC2aar5vz6L6qzAWm/bWgj9bo=";
  };
in
pkgs.buildNpmPackage {
  inherit version src;
  name = "waved-ui";

  sourceRoot = "${src.name}/ui";

  npmDepsHash = "sha256-eKBhQsqBzdC7znOiFeEAFNVKnbDXgVbBS23ZRPjpuuo=";

  nodejs = pkgs.nodejs_18;

  npmFlags = [ "--ignore-scripts" ];
}
  1. nix-build from the same directory where default.nix exists
  2. Result will be the aforementioned error

I'm happy to help troubleshoot or provide more info.

@sarcasticadmin sarcasticadmin added the bug Bug in code label Mar 25, 2024
@mturoci mturoci added security Related to security and removed bug Bug in code labels Mar 26, 2024
@mturoci
Copy link
Collaborator

mturoci commented Mar 26, 2024

Thanks for bringing this up @sarcasticadmin! I always assumed npm takes care of all the integrity stuff for us, but seems like it's not always the case.

Based on the npm issues you linked above, it seems like rm -rf node_modules package-lock.json should be enough to fill in the missing hashes. The unfortunate part is we will lose our lock file and dependencies will be upgraded. If you could find a way to add the hashes while keeping the existing package-lock, that would be awesome.

If not, once you confirm the issue on your side is resolved, feel free to make a PR (or just let me know here and I can make the PR myself).

@sarcasticadmin
Copy link
Author

Let me give that a go and Ill raise a PR shortly

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
security Related to security
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants