Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENOTCACHED / Dependency not found in cache #106

Open
demaisj opened this issue Aug 10, 2018 · 9 comments
Open

ENOTCACHED / Dependency not found in cache #106

demaisj opened this issue Aug 10, 2018 · 9 comments

Comments

@demaisj
Copy link

demaisj commented Aug 10, 2018

Hello,

I'm having trouble building multiple packages from github repositories. Upon the npm install step, npm reports that a dependency is missing from cache, despite it being built earlier.

To simplify steps to reproduce, here's the node-packages.json file narrowed down to the package that fails to build:

[
  { "extra-api": "git://github.com/extranet-oss/extra-api.git#rewrite" }
]

I use the node2nix --nodejs-8 --input node-packages.json command to generate the derivations.

Upon build, npm reports that the ms dependency is missing:

npm ERR! code ENOTCACHED
npm ERR! request to https://registry.npmjs.org/ms failed: cache mode is 'only-if-cached' but no cached response available.

npm ERR! A complete log of this run can be found in:
npm ERR!     /tmp/nix-build-node-extra-api-0.0.0.drv-0/.npm/_logs/2018-08-10T00_42_38_854Z-debug.log
builder for '/nix/store/bf9dszd1da9mzknqbav133gnfvsahhka-node-extra-api-0.0.0.drv' failed with exit code 1

I have no idea what could cause this, all required versions of ms are downloaded & built prior to executing npm install.
I can not bypass the npm install step, as I need it to create the executable files of the packages.

Note: this error doesn't occur when building for nodejs-6_x

@cryptix
Copy link

cryptix commented Sep 5, 2018

I also ran into this (using a git redirect in pkg json) and can confirm that downgrading to node@6 helps but it is unfortunate.

@demaisj: did you maybe come across another workaround since posting this? (Apart from publishing a package to a registry maybe)

cryptix added a commit to talenet/talenet that referenced this issue Sep 5, 2018
ps: this also uses scuttle-shell, which depends on systrayhelper
NixOS/nixpkgs#45997

hit a bug using node2nix -8 which is why i'm trying 6 right now:
svanderburg/node2nix#106 (comment)
cryptix added a commit to talenet/talenet that referenced this issue Oct 4, 2018
ps: this also uses scuttle-shell, which depends on systrayhelper
NixOS/nixpkgs#45997

hit a bug using node2nix -8 which is why i'm trying 6 right now:
svanderburg/node2nix#106 (comment)
cryptix added a commit to talenet/talenet that referenced this issue Oct 18, 2018
ps: this also uses scuttle-shell, which depends on systrayhelper
NixOS/nixpkgs#45997

hit a bug using node2nix -8 which is why i'm trying 6 right now:
svanderburg/node2nix#106 (comment)
@fuzzy-id
Copy link

Any progress here? I get a similar error having serverless in supplement.json. I tried all possible combinations of --lock and --include-peer-dependencies. The build throws ENOTCACHED on uri-js.

$ node2nix --version
node2nix 1.6.0

@svanderburg
Copy link
Owner

I think I have found the problem -- apparently some tarballs in the NPM registry bundle a pacakge-lock.json file. Normally, this is not supposed to happen -- package-lock.json files can only be used in development projects.

When you run npm publish on a project, then the package-lock.json file is blacklisted, but for some reason certain package authors have managed to bypass this restriction (maybe because they upload a custom crafted tarball)

I now modified node2nix in revision 603c19b to remove a package-lock.json file when no lock file has been provided. This seems to fix the problem.

You can try it by installing the development version, by cloning the Git repo and running:

$ nix-env -f default.nix -iA package

colehaus added a commit to colehaus/nix-extras that referenced this issue Dec 5, 2018
colehaus added a commit to colehaus/nix-extras that referenced this issue Dec 5, 2018
@cryptix
Copy link

cryptix commented Dec 20, 2018

I tried again with the new version but I still get ENOTCACHED for ansi-styles (which chalk depends on for instance). I even tried making all devDependencies normal dependencies and adding it to supplements but it still isn't fetched somehow...

Here is the npm ls from a working install form using nix-shell shell.nix <<< npm i. Maybe my node2nix env approach is just way off because I don't fully understand the intricacies at play here. Would love some feed back on the questions in PR NixOS/nixpkgs#46521 or the newer talenet/talenet#35.

@svanderburg
Copy link
Owner

node2ix 1.7.0 now contains a workaround for NPM dependencies in the registry that have package-lock.json files. This release should be able to deply this.

@IslandUsurper
Copy link

IslandUsurper commented Jun 4, 2020

I just started having this issue. node2nix 1.8.0, trying to build @bigcommerce/stencil-cli. When I checkout that repo and run node2nix with the lock file, it worked, but when I tried to just use a node-packages.json listing the one package, it stopped being able to build.

@dustinlacewell
Copy link

dustinlacewell commented Jan 15, 2022

I'm getting this error in 1.9.0

https://gist.github.com/dustinlacewell/b1b45edd7bec6abbaa4d833da0c298aa

sternenseemann added a commit to sternenseemann/likely-music that referenced this issue Feb 27, 2022
These days, support for npm's package-lock.json seems to besser than for
yarn, so let's drop yarn2nix which is not quite where it needs to be
yet (we pinned an experimental branch after all) and use napalm instead.
For this we have to migrate from yarn to npm completely.

Using npmlock2nix would be preferrable (as it doesn't require building a
Haskell registry server), but there we end up running into an unsolved
issue [1].

[1]: svanderburg/node2nix#106
sternenseemann added a commit to sternenseemann/likely-music that referenced this issue Feb 27, 2022
These days, support for npm's package-lock.json seems to besser than for
yarn, so let's drop yarn2nix which is not quite where it needs to be
yet (we pinned an experimental branch after all) and use napalm instead.
For this we have to migrate from yarn to npm completely.

Using npmlock2nix would be preferrable (as it doesn't require building a
Haskell registry server), but there we end up running into an unsolved
issue [1].

[1]: svanderburg/node2nix#106
@cumber
Copy link

cumber commented Mar 11, 2022

Hmm. I'm getting this as well with 1.9.0, trying to package cli-highlight to install in my Nixos system.

To generate the nix files I'm just running:

nix run 'nixpkgs#nodePackages.node2nix' -- -i node-packages.json

(where node-packages.json just contains [ "cli-highlight" ])

Interestingly the resulting nix files build fine if I nix build -f default.nix cli-highlight (and I can run it and it works). I only get the error if I package it in a flake, e.g.

{
  description = "Basic flake packaging of cli-highlight node app";

  outputs = { self, nixpkgs }:
    let system = "x86_64-linux";
        pkgs = nixpkgs.legacyPackages.x86_64-linux;
        called = pkgs.callPackage ./default.nix { inherit pkgs system; };
    in {
      packages.${system}.cli-highlight = called.cli-highlight;

      defaultPackage.${system} = self.packages.${system}.cli-highlight;
  };
}

(Or call it from my Nixos system flake, which is what I really want to do)

@yajo
Copy link

yajo commented May 12, 2022

I'm using node2nix 1.11.1, from commit 68f5735.

I added a simple node-packages.json and ran:

> node2nix -i node-packages.json -16 --include-peer-dependencies
info attempt registry request try #1 at 10:18:13
http request GET https://registry.npmjs.org/prettier
http 200 https://registry.npmjs.org/prettier
info attempt registry request try #1 at 10:18:13
http request GET https://registry.npmjs.org/@prettier%2Fplugin-xml
http 200 https://registry.npmjs.org/@prettier%2Fplugin-xml
info attempt registry request try #1 at 10:18:14
http request GET https://registry.npmjs.org/@xml-tools%2Fparser
http 200 https://registry.npmjs.org/@xml-tools%2Fparser
info attempt registry request try #1 at 10:18:15
http request GET https://registry.npmjs.org/prettier
http 200 https://registry.npmjs.org/prettier
info attempt registry request try #1 at 10:18:15
http request GET https://registry.npmjs.org/chevrotain
http 200 https://registry.npmjs.org/chevrotain
info attempt registry request try #1 at 10:18:16
http request GET https://registry.npmjs.org/regexp-to-ast
http 200 https://registry.npmjs.org/regexp-to-ast

The resulting directory is: prettier.zip (there you can see the node-packages.json too)

Just uncompress, cd into it, and build it. You should be able to reproduce the bug:

> nix-build -A '@prettier/plugin-xml'
this derivation will be built:
  /nix/store/0f2c1p3w11zrx0a3z7ccx4xagmjrslc9-_at_prettier_slash_plugin-xml-2.1.0.drv
building '/nix/store/0f2c1p3w11zrx0a3z7ccx4xagmjrslc9-_at_prettier_slash_plugin-xml-2.1.0.drv'...
unpacking sources
patching sources
configuring
no configure script, doing nothing
building
installing
unpacking source archive /nix/store/1i7dfk69632s59zi6y2qqdmi02cj41q4-plugin-xml-2.1.0.tgz
unpacking source archive /nix/store/2014jpya5nc0acvbpfg5hiw4ardsh6zs-parser-1.0.11.tgz
unpacking source archive /nix/store/akhj4ky8girggajvvhgd82nv1ddz2kx4-chevrotain-7.1.1.tgz
unpacking source archive /nix/store/s0gp8sy11df2b52vapn1p21s0iv6r3x8-prettier-2.6.2.tgz
unpacking source archive /nix/store/q9qx04dcajqvr5mf6n227lxp93dzizzs-regexp-to-ast-0.5.0.tgz
pinpointing versions of dependencies...
patching script interpreter paths in .
./@prettier/plugin-xml/node_modules/prettier/bin-prettier.js: interpreter directive changed from "#!/usr/bin/env node" to "/nix/store/3bvdynxk4q3jyjzd1b6h91xi7yp3bk70-nodejs-16.15.0/bin/node"
No package-lock.json file found, reconstructing...
Adding metadata fields to: node_modules/@xml-tools/parser/package.json
Adding metadata fields to: node_modules/chevrotain/package.json
Adding metadata fields to: node_modules/prettier/package.json
Adding metadata fields to: node_modules/regexp-to-ast/package.json
rebuilt dependencies successfully
npm WARN old lockfile marborist:ctor Completed in 0ms
npm WARN old lockfile The package-lock.json file was created with an old version of npm,
npm WARN old lockfile so supplemental metadata must be fetched from the registry.
npm WARN old lockfile 
npm WARN old lockfile This is a one-time fix-up, please be patient...
npm WARN old lockfile 
npm WARN old lockfile Error: request to https://registry.npmjs.org/@xml-tools%2fparser failed: cache mode is 'only-if-cached' but no cached response is available.
npm WARN old lockfile     at cacheFetch (/nix/store/3bvdynxk4q3jyjzd1b6h91xi7yp3bk70-nodejs-16.15.0/lib/node_modules/npm/node_modules/make-fetch-happen/lib/cache/index.js:12:13)
npm WARN old lockfile     at async fetch (/nix/store/3bvdynxk4q3jyjzd1b6h91xi7yp3bk70-nodejs-16.15.0/lib/node_modules/npm/node_modules/make-fetch-happen/lib/fetch.js:98:7)
npm WARN old lockfile     at async Array.<anonymous> (/nix/store/3bvdynxk4q3jyjzd1b6h91xi7yp3bk70-nodejs-16.15.0/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/build-ideal-tree.js:758:9)
npm WARN old lockfile  Could not fetch metadata for @xml-tools/parser@1.0.11 NotCachedError: request to https://registry.npmjs.org/@xml-tools%2fparser failed: cache mode is 'only-if-cached' but no cached response is available.
npm WARN old lockfile     at cacheFetch (/nix/store/3bvdynxk4q3jyjzd1b6h91xi7yp3bk70-nodejs-16.15.0/lib/node_modules/npm/node_modules/make-fetch-happen/lib/cache/index.js:12:13)
npm WARN old lockfile     at async fetch (/nix/store/3bvdynxk4q3jyjzd1b6h91xi7yp3bk70-nodejs-16.15.0/lib/node_modules/npm/node_modules/make-fetch-happen/lib/fetch.js:98:7)
npm WARN old lockfile     at async Array.<anonymous> (/nix/store/3bvdynxk4q3jyjzd1b6h91xi7yp3bk70-nodejs-16.15.0/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/build-ideal-tree.js:758:9) {
npm WARN old lockfile   code: 'ENOTCACHED'
npm WARN old lockfile }
npm WARN old lockfile Error: request to https://registry.npmjs.org/chevrotain failed: cache mode is 'only-if-cached' but no cached response is available.
npm WARN old lockfile     at cacheFetch (/nix/store/3bvdynxk4q3jyjzd1b6h91xi7yp3bk70-nodejs-16.15.0/lib/node_modules/npm/node_modules/make-fetch-happen/lib/cache/index.js:12:13)
npm WARN old lockfile     at async fetch (/nix/store/3bvdynxk4q3jyjzd1b6h91xi7yp3bk70-nodejs-16.15.0/lib/node_modules/npm/node_modules/make-fetch-happen/lib/fetch.js:98:7)
npm WARN old lockfile     at async Array.<anonymous> (/nix/store/3bvdynxk4q3jyjzd1b6h91xi7yp3bk70-nodejs-16.15.0/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/build-ideal-tree.js:758:9)
npm WARN old lockfile  Could not fetch metadata for chevrotain@7.1.1 NotCachedError: request to https://registry.npmjs.org/chevrotain failed: cache mode is 'only-if-cached' but no cached response is available.
npm WARN old lockfile     at cacheFetch (/nix/store/3bvdynxk4q3jyjzd1b6h91xi7yp3bk70-nodejs-16.15.0/lib/node_modules/npm/node_modules/make-fetch-happen/lib/cache/index.js:12:13)
npm WARN old lockfile     at async fetch (/nix/store/3bvdynxk4q3jyjzd1b6h91xi7yp3bk70-nodejs-16.15.0/lib/node_modules/npm/node_modules/make-fetch-happen/lib/fetch.js:98:7)
npm WARN old lockfile     at async Array.<anonymous> (/nix/store/3bvdynxk4q3jyjzd1b6h91xi7yp3bk70-nodejs-16.15.0/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/build-ideal-tree.js:758:9) {
npm WARN old lockfile   code: 'ENOTCACHED'
npm WARN old lockfile }
npm WARN old lockfile Error: request to https://registry.npmjs.org/prettier failed: cache mode is 'only-if-cached' but no cached response is available.
npm WARN old lockfile     at cacheFetch (/nix/store/3bvdynxk4q3jyjzd1b6h91xi7yp3bk70-nodejs-16.15.0/lib/node_modules/npm/node_modules/make-fetch-happen/lib/cache/index.js:12:13)
npm WARN old lockfile     at async fetch (/nix/store/3bvdynxk4q3jyjzd1b6h91xi7yp3bk70-nodejs-16.15.0/lib/node_modules/npm/node_modules/make-fetch-happen/lib/fetch.js:98:7)
npm WARN old lockfile     at async Array.<anonymous> (/nix/store/3bvdynxk4q3jyjzd1b6h91xi7yp3bk70-nodejs-16.15.0/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/build-ideal-tree.js:758:9)
npm WARN old lockfile  Could not fetch metadata for prettier@2.6.2 NotCachedError: request to https://registry.npmjs.org/prettier failed: cache mode is 'only-if-cached' but no cached response is available.
npm WARN old lockfile     at cacheFetch (/nix/store/3bvdynxk4q3jyjzd1b6h91xi7yp3bk70-nodejs-16.15.0/lib/node_modules/npm/node_modules/make-fetch-happen/lib/cache/index.js:12:13)
npm WARN old lockfile     at async fetch (/nix/store/3bvdynxk4q3jyjzd1b6h91xi7yp3bk70-nodejs-16.15.0/lib/node_modules/npm/node_modules/make-fetch-happen/lib/fetch.js:98:7)
npm WARN old lockfile     at async Array.<anonymous> (/nix/store/3bvdynxk4q3jyjzd1b6h91xi7yp3bk70-nodejs-16.15.0/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/build-ideal-tree.js:758:9) {
npm WARN old lockfile   code: 'ENOTCACHED'
npm WARN old lockfile }
npm WARN old lockfile Error: request to https://registry.npmjs.org/regexp-to-ast failed: cache mode is 'only-if-cached' but no cached response is available.
npm WARN old lockfile     at cacheFetch (/nix/store/3bvdynxk4q3jyjzd1b6h91xi7yp3bk70-nodejs-16.15.0/lib/node_modules/npm/node_modules/make-fetch-happen/lib/cache/index.js:12:13)
npm WARN old lockfile     at async fetch (/nix/store/3bvdynxk4q3jyjzd1b6h91xi7yp3bk70-nodejs-16.15.0/lib/node_modules/npm/node_modules/make-fetch-happen/lib/fetch.js:98:7)
npm WARN old lockfile     at async Array.<anonymous> (/nix/store/3bvdynxk4q3jyjzd1b6h91xi7yp3bk70-nodejs-16.15.0/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/build-ideal-tree.js:758:9)
npm WARN old lockfile  Could not fetch metadata for regexp-to-ast@0.5.0 NotCachedError: request to https://registry.npmjs.org/regexp-to-ast failed: cache mode is 'only-if-cached' but no cached response is available.
npm WARN old lockfile     at cacheFetch (/nix/store/3bvdynxk4q3jyjzd1b6h91xi7yp3bk70-nodejs-16.15.0/lib/node_modules/npm/node_modules/make-fetch-happen/lib/cache/index.js:12:13)
npm WARN old lockfile     at async fetch (/nix/store/3bvdynxk4q3jyjzd1b6h91xi7yp3bk70-nodejs-16.15.0/lib/node_modules/npm/node_modules/make-fetch-happen/lib/fetch.js:98:7)
npm WARN old lockfile     at async Array.<anonymous> (/nix/store/3bvdynxk4q3jyjzd1b6h91xi7yp3bk70-nodejs-16.15.0/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/build-ideal-tree.js:758:9) {
npm WARN old lockfile   code: 'ENOTCACHED'
npm WARN old lockfile }
npm ERR! code ENOTCACHEDtier/plugin-xml: sill idealTree buildDeps
npm ERR! request to https://registry.npmjs.org/@types%2fjest failed: cache mode is 'only-if-cached' but no cached response is available.

npm ERR! A complete log of this run can be found in:
npm ERR!     /build/.npm/_logs/2022-05-12T09_19_08_038Z-debug-0.log

error: builder for '/nix/store/0f2c1p3w11zrx0a3z7ccx4xagmjrslc9-_at_prettier_slash_plugin-xml-2.1.0.drv' failed with exit code 1;
       last 10 log lines:
       > npm WARN old lockfile     at async fetch (/nix/store/3bvdynxk4q3jyjzd1b6h91xi7yp3bk70-nodejs-16.15.0/lib/node_modules/npm/node_modules/make-fetch-happen/lib/fetch.js:98:7)
       > npm WARN old lockfile     at async Array.<anonymous> (/nix/store/3bvdynxk4q3jyjzd1b6h91xi7yp3bk70-nodejs-16.15.0/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/build-ideal-tree.js:758:9) {
       > npm WARN old lockfile   code: 'ENOTCACHED'
       > npm WARN old lockfile }
       > npm ERR! code ENOTCACHEDtier/plugin-xml: sill idealTree buildDeps
       > npm ERR! request to https://registry.npmjs.org/@types%2fjest failed: cache mode is 'only-if-cached' but no cached response is available.
       > 
       > npm ERR! A complete log of this run can be found in:
       > npm ERR!     /build/.npm/_logs/2022-05-12T09_19_08_038Z-debug-0.log
       > 
       For full logs, run 'nix log /nix/store/0f2c1p3w11zrx0a3z7ccx4xagmjrslc9-_at_prettier_slash_plugin-xml-2.1.0.drv'.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants