Unable to gather build info for Node app w/ npm

[eric-gonzalez-tfs]

Describe the bug

The JFrog CLI is unable to publish build info for my application. I see errors regarding failed checksum calculations.

Current behavior

This shows the initial setup that I have been using to test:

Moments later the app runs it's npm preinstall and prepare scripts, and we start seeing some additional details populate the logs...

• A huge JSON object produced by npm ls
• A long list of failed checksum warnings:

17:28:35 [Debug] couldn't calculate checksum for @babel/helper-hoist-variables:7.22.5. Error: '01d391be5bf1545a42bdab9ab92d5cf43c6b5a2f8aef684b7797126cdccdd345 is not found in /Users/myusername/.npm/_cacache/index-v5/01/d3/91be5bf1545a42bdab9ab92d5cf43c6b5a2f8aef684b7797126cdccdd345'.
17:28:35 [Debug] couldn't calculate checksum for reusify:1.0.4. Error: '6b3faa2b1e9a3bbe6138bd8db38e1e3e02da17f4c1888217e5a6568f85b1f630 is not found in /Users/myusername/.npm/_cacache/index-v5/6b/3f/aa2b1e9a3bbe6138bd8db38e1e3e02da17f4c1888217e5a6568f85b1f630'.
17:28:35 [Debug] couldn't calculate checksum for esquery:1.5.0. Error: 'b194d9ee383b240b5aa870c2765e41b1e6a125930434a3d53e3d2778e50a8e05 is not found in /Users/myusername/.npm/_cacache/index-v5/b1/94/d9ee383b240b5aa870c2765e41b1e6a125930434a3d53e3d2778e50a8e05'.
17:28:35 [Debug] couldn't calculate checksum for jsonparse:1.3.1. Error: '456740d3ab7aeb959caa7c28930999399d183c4be535a41854342a008e3a0ad7 is not found in /Users/myusername/.npm/_cacache/index-v5/45/67/40d3ab7aeb959caa7c28930999399d183c4be535a41854342a008e3a0ad7'.
17:28:35 [Debug] couldn't calculate checksum for destroy:1.2.0. Error: '0e3ed571cf72377d8cb2121c552e5a9e5dff14ccb980e2b324e0885e45cb55a1 is not found in /Users/myusername/.npm/_cacache/index-v5/0e/3e/d571cf72377d8cb2121c552e5a9e5dff14ccb980e2b324e0885e45cb55a1'.
17:28:35 [Debug] couldn't calculate checksum for micromark-util-subtokenize:2.0.1. Error: 'adfd2246c3dd8c5d2b425b3f493e5819d1f655665bfaa94dd5d649071e551526 is not found in /Users/myusername/.npm/_cacache/index-v5/ad/fd/2246c3dd8c5d2b425b3f493e5819d1f655665bfaa94dd5d649071e551526'.
17:28:35 [Debug] couldn't calculate checksum for file-type:19.0.0. Error: '9ac853c7e959e809f3495e9cee05b194fd7a51b1cc7d4098d46c7d66484911ac is not found in /Users/myusername/.npm/_cacache/index-v5/9a/c8/53c7e959e809f3495e9cee05b194fd7a51b1cc7d4098d46c7d66484911ac'.
17:28:35 [Debug] couldn't calculate checksum for is-plain-obj:2.1.0. Error: '3ed3b8d4d4f32264cb974b0feed3ad748ef591df313d1aaf9b17fa5b6c00b8ca is not found in /Users/myusername/.npm/_cacache/index-v5/3e/d3/b8d4d4f32264cb974b0feed3ad748ef591df313d1aaf9b17fa5b6c00b8ca'.
17:28:35 [Debug] couldn't calculate checksum for clone:1.0.4. Error: 'c5680beeb9063df9690790cb9f3417a06ea69e76a44031909dc7e390918f1f57 is not found in /Users/myusername/.npm/_cacache/index-v5/c5/68/0beeb9063df9690790cb9f3417a06ea69e76a44031909dc7e390918f1f57'.
17:28:35 [Debug] couldn't calculate checksum for argparse:1.0.10. Error: '7fba4e128822ff93d1ee185a07e33587f76faa07ed64f0c7d52467273165726b is not found in /Users/myusername/.npm/_cacache/index-v5/7f/ba/4e128822ff93d1ee185a07e33587f76faa07ed64f0c7d52467273165726b'.
17:28:35 [Debug] couldn't calculate checksum for basic-ftp:5.0.5. Error: 'c63f835d0c7330c5d8817a45243d1611ecb7a13edab5baef8b75e853c87e0f84 is not found in /Users/myusername/.npm/_cacache/index-v5/c6/3f/835d0c7330c5d8817a45243d1611ecb7a13edab5baef8b75e853c87e0f84'.
17:28:35 [Debug] couldn't calculate checksum for moment:2.30.1. Error: '6ea308a85ff980d8bcc1a5aa9e26a53919c29110b64364aa88a695fc7f563455 is not found in /Users/myusername/.npm/_cacache/index-v5/6e/a3/08a85ff980d8bcc1a5aa9e26a53919c29110b64364aa88a695fc7f563455'.
17:28:35 [Debug] couldn't calculate checksum for parse-path:7.0.0. Error: '482a706354bb7f416ff5b9860613409aca1f3ac28e1b22df6634116a40b58e35 is not found in /Users/myusername/.npm/_cacache/index-v5/48/2a/706354bb7f416ff5b9860613409aca1f3ac28e1b22df6634116a40b58e35'.
17:28:35 [Debug] couldn't calculate checksum for normalize-url:8.0.1. Error: 'b8a8a97351674afa45a45c697263810dbedf87ca08e0db2c7fd9fe19d76d5a48 is not found in /Users/myusername/.npm/_cacache/index-v5/b8/a8/a97351674afa45a45c697263810dbedf87ca08e0db2c7fd9fe19d76d5a48'.

# and it goes on forever......

• Finally this warning regarding not dependencies not being captured in build-info:

17:28:35 [🟠Warn] The following dependencies will not be included in the build-info, because they are missing in the npm cache: '@babel/helper-hoist-variables:7.22.5,reusify:1.0.4,esquery:1.5.0,jsonparse:1.3.1,destroy:1.2.0,micromark-util-subtokenize:2.0.1,file-type:19.0.0,is-plain-obj:2.1.0,clone:1.0.4,argparse:1.0.10,basic-ftp:5.0.5,moment:2.30.1,parse-path:7.0.0,normalize-url:8.0.1,oas-validator:5.0.8,require-directory:2.1.1,yoctocolors-cjs:2.1.2,minipass:5.0.0,agent-base:6.0.2,run-async:2.4.1,micromark-factory-label:2.0.0,cosmiconfig:8.2.0,open-editor:4.1.1,iconv-lite:0.4.24,bl:4.1.0,to-regex-range:5.0.1,send:0.18.0,boxen:7.1.1,@webassemblyjs/wast-printer:1.12.1,@babel/helper-module-transforms:7.24.5,emoji-regex:10.3.0,micromark-factory-whitespace:1.1.0,yargs:17.7.2,wcwidth:1.0.1,yaml:2.3.4,signal-exit:3.0.7,ansi-regex:6.0.1,is-glob:2.0.1,semver-diff:4.0.0,remark-lint-strong-marker:3.1.2,@webassemblyjs/wasm-opt:1.12.1,eslint-plugin-n:16.6.2,json-stable-stringify:1.1.1,@types/mocha:10.0.6,content-type:1.0.5,unist-util-remove-position:5.0.0,longest-streak:3.1.0,eslint-visitor-keys:2.1.0,openapi-validator-middleware:2.0.1,@webassemblyjs/helper-api-error:1.11.6,minimatch:9.0.5,@types/swagger-ui-express:4.1.6,async-listener:0.6.10,workerpool:6.2.1,kind-of:6.0.3,qrcode:1.5.3,@sindresorhus/is:5.6.0,ansi-align:3.0.1,remark-lint-definition-case:3.1.2,remark-lint-no-table-indentation:4.1.2,meow:13.2.0,@types/wait-on:5.3.4,parseurl:1.3.3,unified-lint-rule:2.1.2,glob-to-regexp:0.4.1,is-docker:2.2.1,mkdirp:3.0.1,memorystream:0.3.1,spdx-exceptions:2.5.0,mute-stream:0.0.8,log-symbols:4.1.0,graceful-fs:4.2.11,sort-package-json:2.10.0,wildcard-match:5.1.3,@babel/helper-string-parser:7.24.1,deep-is:0.1.4,cacache:18.0.4,path-is-absolute:1.0.1,is-stream:2.0.1,lint-staged:15.2.2,available-typed-arrays:1.0.7,remark-lint-final-newline:2.1.2,@commitlint/is-ignored:19.2.2,http-status-codes:2.3.0,http2-wrapper:2.2.1,oas-linter:3.2.2,jsesc:3.0.2,url-or-path:2.3.0,type-fest:0.21.3,append-field:1.0.0,chalk:5.3.0,tar-fs:2.1.1,brace-expansion:1.1.11,indent-string:4.0.0,remark-lint-code-block-style:3.1.2,@eslint/js:8.57.0,@babel/parser:7.23.0,progress:2.0.3,http-errors:2.0.0,color-name:1.1.4,cli-spinners:2.9.2,dir-glob:3.0.1,array.prototype.flat:1.3.2,find-cache-dir:5.0.0,mkdirp-classic:0.5.3,tar-stream:3.1.7,call-bind:1.0.7,json-schema-ref-parser:6.1.0,joi:17.13.1,p-limit:3.1.0,optionator:0.9.4,lodash.upperfirst:4.3.1,@octokit/auth-token:4.0.0,@typescript-eslint/typescript-estree:7.10.0,is-get-set-prop:1.0.0,esm-utils:4.3.0,@nicolo-ribaudo/eslint-scope-5-internals:5.1.1-v1,type-fest:4.21.0,bare-events:2.4.2,core-util-is:1.0.3,continuation-local-storage:3.2.1,typed-array-byte-offset:1.0.2,mdast-util-mdx-expression:1.3.2,lodash.startcase:4.4.0,bare-path:2.1.3,unique-slug:4.0.0,minipass:3.3.6,postgres-range:1.1.4,@npmcli/config:8.3.2,@types/acorn:4.0.6,valid-url:1.0.9,mimic-response:3.1.0,camelcase:7.0.1,mdast-util-gfm-autolink-literal:2.0.0,mdast-util-phrasing:3.0.1,@commitlint/top-level:19.0.0,bare-fs:2.3.1,globby:11.1.0,cpe-fs:1.0.1,read-pkg:3.0.0,is-typedarray:1.0.0,micromark-extension-gfm-autolink-literal:2.0.0,array.prototype.findlastindex:1.2.5,@types/json5:0.0.29,@babel/compat-data:7.24.4,ini:4.1.1,readable-stream:3.6.2,conventional-changelog-codemirror:4.0.0,tar-fs:3.0.6,p-is-promise:3.0.0,accepts:1.3.8,globby:14.0.2,external-editor:3.1.0,properties-reader:2.3.0,inquirer:8.2.5,shebang-command:2.0.0,etag:1.8.1,is-regex:1.1.4,@octokit/rest:20.1.1,socks:2.8.3,unicode-emoji-modifier-base:1.0.0,deep-extend:0.6.0,process-nextick-args:2.0.1,conventional-commit-types:3.0.0,is-ci:3.0.1,path-scurry:1.11.1,jsonpath-plus:7.2.0,typed-array-length:1.0.6,lru-queue:0.1.0,proto-list:1.2.4,eslint-compat-utils:0.5.0,is-js-type:2.0.0,@types/ssh2-streams:0.1.12,isexe:2.0.0,copyfiles:2.4.1,which-boxed-primitive:1.0.2,rechoir:0.6.2,remark-lint-emphasis-marker:3.1.2,lodash.snakecase:4.1.1,webidl-conversions:3.0.1,data-view-buffer:1.0.1,postgres-array:2.0.0,require-main-filename:2.0.0,events:3.3.0,common-path-prefix:3.0.0,@babel/helpers:7.24.5,delay:6.0.0,diagnostic-channel:0.3.1,p-try:2.2.0,unist-util-position:4.0.4,to-absolute-glob:3.0.0,callsites:3.1.0,cacheable-request:10.2.14,micromark-factory-whitespace:2.0.0,estraverse:4.3.0,flat-cache:3.2.0,split2:4.2.0,debug:4.3.5,which:4.0.0,shebang-regex:3.0.0,eventemitter3:5.0.1,wrap-ansi:7.0.0,abbrev:2.0.0,@types/ms:0.7.34,untildify:4.0.0,unist-util-stringify-position:4.0.0,is-arrayish:0.2.1,cpu-features:0.0.10,@nodelib/fs.scandir:2.1.5,lazystream:1.0.1,http-cache-semantics:4.1.1,@mycompany/my-package:6.1.3,@npmcli/name-from-folder:2.0.0,@types/mdast:3.0.15,mdast-util-mdx-jsx:3.1.2,peek-readable:5.0.0,fast-diff:1.3.0,@eslint-community/regexpp:4.10.0,socks-proxy-agent:8.0.4,is-ssh:1.4.0,didyoumean:1.2.2,get-intrinsic:1.2.4,eslint-mdx:3.1.5,sade:1.8.1,encodeurl:1.0.2,forwarded:0.2.0,concat-stream:1.6.2,openapi-schema-validation:0.4.2,@xtuc/ieee754:1.2.0,import-modules:2.1.0,debug:4.3.4,bindings:1.5.0,ajv-formats:2.1.1,listr2:8.0.1,object-assign:4.1.1,pg-types:4.0.2,has-flag:3.0.0,p-cancelable:3.0.0,json5:2.2.3,dargs:8.1.0,buffer-crc32:0.2.13,tr46:0.0.3,es-errors:1.3.0,resolve-dir:1.0.1,log-update:6.0.0,pidtree:0.6.0,mocha:10.4.0,node-pty:1.0.0,memoizee:0.4.15,flatted:3.3.1,@commitlint/config-validator:19.0.3,zwitch:2.0.4,ansi-colors:4.1.1,@types/http-cache-semantics:4.0.4,micromark-extension-gfm-task-list-item:2.0.1,@babel/helper-function-name:7.23.0,fast-deep-equal:3.1.3,locate-path:7.2.0,dot-prop:5.3.0,json-schema-ref-parser:7.1.4,get-stream:5.2.0,y18n:4.0.3,json5:1.0.2,minimatch:3.1.2,wrap-ansi:6.2.0,form-data:4.0.0,type-is:1.6.18,is-empty:1.2.0,mdast-util-mdxjs-esm:2.0.1,toidentifier:1.0.1,pgpass:1.0.5,es-shim-unscopables:1.0.2,@commitlint/types:19.0.3,micromark-extension-mdxjs-esm:3.0.0,is-interactive:2.0.0,@eslint/eslintrc:3.1.0,path-exists:5.0.0,cli-width:4.1.0,typedarray:0.0.6,jackspeak:3.4.3,is-valid-path:0.1.1,esniff:2.0.1,@octokit/request:8.4.0,lowercase-keys:3.0.0,eslint-config-xo:0.44.0,human-signals:2.1.0,pupa:3.1.0,is-proto-prop:2.0.0,cz-conventional-changelog:3.3.0

# also goes on forever....

Hint: Try deleting 'node_modules' and/or 'package-lock.json'.
17:28:35 [Debug] Creating temp build file at: /var/folders/yt/06w44b0n2b52yl7bq__z5sym0000gq/T/jfrog/builds/e7a3c2e18273a65c51013e8460d2a29b84d6b43727c5e52a7e3e673c89122e07
17:28:35 [Debug] Creating temp build file at: /var/folders/yt/06w44b0n2b52yl7bq__z5sym0000gq/T/jfrog/builds/e7a3c2e18273a65c51013e8460d2a29b84d6b43727c5e52a7e3e673c89122e07
17:28:35 [Debug] Restored the file /Users/myusername/Git/--/.npmrc successfully

The temp build file looks like:

When I try a dry-run of the publish build info, I don't see any depedencies being captured.

Reproduction steps

rm -rf node_modules 
npm cache clean --force
jf npm --version # should equal 10.2.4
jq '.lockfileVersion' package-lock.json # should equal 3
jf rt bc "build-prefix/my-app" "v1"
export JFROG_CLI_LOG_LEVEL=DEBUG
jf npm ci --build-name="build-prefix/my-app" --build-number="v1" --threads=1
jf rt bp "build-prefix/my-app" "v1" --env-exclude="*password*;*psw*;*secret*;*key*;*token*;*auth*;" --dry-run

Expected behavior

The JFrog CLI captures the dependencies correctly and the the rt bp dry run displays the correct dependencies.

JFrog CLI version

jf version 2.68.0

Operating system type and version

MacOS Sonoma 14.4.1

JFrog Artifactory version

Cloud SaaS

JFrog Xray version

Cloud SaaS

[eric-gonzalez-tfs]

BTW - I have a copy of my entire terminal transcript that I am willing to share over email.

[eric-gonzalez-tfs]

Hi JFrog team, do you have any recommendations here?

@omerzi @eyalbe4 @yahavi

[Or-Geva]

@eric-gonzalez-tfs, please remove any post or pull scripts from the package.json as they may interfere with the checksum calculations.s.

[eric-gonzalez-tfs]

@eric-gonzalez-tfs, please remove any post or pull scripts from the package.json as they may interfere with the checksum calculations.s.

Hi @Or-Geva - thank you for the details. I can definitely test that and follow up.

Can you confirm if removing preinstall, postinstall, or other lifecycle scripts from package.json is the official JF recommendation? This approach isn't practical as these scripts are crucial for our workflows.

Please advise on a more sustainable solution.

[udhay13190]

@eric-gonzalez-tfs Did you get a workaround for this ?

[eric-gonzalez-tfs]

@eric-gonzalez-tfs Did you get a workaround for this ?

Hi @udhay13190 - I added omit-lockfile-registry-resolved=true to my .npmrc, deleted node_modules, and re-generated the package-lock.json and it has been working consistently since.

Unfortunately, I do not know the repercussions of adding this, and the JF team has not been responsive to my question (above.)