[BUG] npm@10 refuses to install packages from GitHub package registry #7024

grahamb · 2023-11-25T00:55:09Z

Is there an existing issue for this?

I have searched the existing issues

This issue exists in the latest npm version

I am using the latest npm

Current Behavior

npm@10 is not installing packages from a private GitHub Enterprise Server registry. It fails with a 403 forbidden error when trying to fetch the package. Previous versions of npm do not encounter this issue.

Expected Behavior

I should be able to install packages from a private registry when my .npmrc is configured properly.

Steps To Reproduce

Remove ~/.npm to ensure we're not pulling from cache

rm -rf ~/.npm

Configure .npmrc to use a private registry (in this case, I am using GitHub Enterprise Server 3.9.3)

//npm.github.sfu.ca/:_authToken=${GITHUB_NPM_TOKEN}
@its:registry=https://npm.github.sfu.ca/

Install a package from the private registry:

$ npm i @its/grouper-node       
npm ERR! code E403
npm ERR! 403 403 Forbidden - GET https://xxxxxxx.s3.ca-central-1.amazonaws.com/1631/npm_***?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=xxxxxxxx&X-Amz-Date=20231125T002943Z&X-Amz-Expires=300&X-Amz-SignedHeaders=host&X-Amz-Signature=xxxxxxx
npm ERR! 403 In most cases, you or one of your dependencies are requesting
npm ERR! 403 a package version that is forbidden by your security policy, or
npm ERR! 403 on a server you do not have access to.

npm ERR! A complete log of this run can be found in:  A complete log of this run can be found in: /home/grahamb/.npm/_logs/2023-11-25T00_29_42_285Z-debug-0.log

Debug log: 2023-11-25T00_29_42_285Z-debug-0.log

When using npm v9, this error does not occur. For example, with Node 18/npm 9:

$ node --version && npm --version
v18.17.1
9.6.7
$ rm -rf node_modules ~/.npm
$ npm i @its/grouper-node        

added 9 packages in 1s

3 packages are looking for funding
  run `npm fund` for details
npm notice 
npm notice New major version of npm available! 9.6.7 -> 10.2.4
npm notice Changelog: https://github.com/npm/cli/releases/tag/v10.2.4
npm notice Run npm install -g npm@10.2.4 to update!
npm notice

Using npm v10 with node 18 fails with the same 403.

Use npm v9 with node 21 works.

$ rm -rf node_modules ~/.npm
$ node --version && npm --version
v21.2.0
9.9.2
$ npm i @its/grouper-node        

added 9 packages in 2s

3 packages are looking for funding
  run `npm fund` for details
npm notice 
npm notice New major version of npm available! 9.9.2 -> 10.2.4
npm notice Changelog: https://github.com/npm/cli/releases/tag/v10.2.4
npm notice Run npm install -g npm@10.2.4 to update!
npm notice

This seems to indicate that something has changed with npm 10 with how it handles private registries (or at least with GitHub's registry).

Environment

npm: 10.2.4
Node.js: 20, 21
OS Name: Linux (RHEL 8)
npm config:

; "user" config from /home/grahamb/.npmrc

@grahamb:registry = "https://npm.github.sfu.ca/" 
@its:registry = "https://npm.github.sfu.ca/" 
//npm.github.sfu.ca/:_authToken = (protected) 

; node bin location = /home/grahamb/.nvm/versions/node/v21.2.0/bin/node
; node version = v21.2.0
; npm local prefix = /tmp/test
; npm version = 10.2.4
; cwd = /tmp/test
; HOME = /home/grahamb
; Run `npm config ls -l` to show all defaults.

The text was updated successfully, but these errors were encountered:

marshalys · 2023-12-01T00:57:08Z

I have the same problem, npm v9 is OK, but npm v10 got error.

bacterik · 2023-12-01T09:49:01Z

Ive also got issues with Node 18.19 and npm 10.

npm ci
shell: /usr/bin/bash -e {0}
env:
NPM_TOKEN: ***
GITHUB_PACKAGES_TOKEN: ***
HTTP_PROXY: ***
HTTPS_PROXY: ***
npm ERR! code E400
npm ERR! 400 Bad request - GET https://npm.pkg.github.com/download/@xxx

...

Error: Process completed with exit code 1.

madmanxxx · 2023-12-04T22:54:10Z

OK, this took HOURS to debug.
Environment:

Docker containers downloaded today (all are affected, because all have npm@10): node:18-buster, node:20-buster, node:20-bullseye
package-lock.json (interesting part):

    "node_modules/node-csv": {
      "version": "0.5.0",
      "resolved": "https://github.com/voodootikigod/node-csv/tarball/master",
      "integrity": "sha512-dzSBhb1hyo0vijIe4ST7G0RFzqgjuIsTMDnw80+ygidl0q8ca+wBpnZH3FFWjqWHwzNJD2N7uJgfdmXks46G/A==",
      "engines": {
        "node": ">= 0.8"
      }
    },

I use corporate proxy to download dependencies.

All other dependencies (downloaded from registry.npmjs.org) are OK, but this specific one causes cryptic HTTP 400, much like @bacterik describes.
If I use a direct connection (without proxy) the dependency downloads correctly.
Proxy does not modify the request in any way, because HTTPS.

After tricking npm to download this module from another server under my control I noticed that HTTP request line looks odd:

GET http://mytestdomain.dev:443/voodootikigod/node-csv/tarball/master HTTP/1.1

and in fact when I send a request to github.com including this atrocious line, it responds with HTTP 400. For clarity, that request should be

GET /voodootikigod/node-csv/tarball/master HTTP/1.1
Host: mytestdomain.dev

I can also confirm that downgrading npm to v9 (npm i -g npm@9) resolves this issue.

bacterik · 2023-12-05T09:13:28Z

Nice @madmanxxx. I didnt have the time myself to do the extensive work you did. But yes, I can confirm this also. I'm using a corporate proxy, and trying to fetch a package from https://npm.pkg.github.com. We have to specifiy 18.18.2 Node version from docker hub, to make it work.

lukasmahr · 2023-12-06T19:45:35Z

I have the same problem, but it does not seem to be bound to GitHub package registries. I am using Azure DevOps artifact streams and get a 403 error when using npm 10 in combination with an HTTP proxy.

Downgrading to npm 9 or removing the proxy "solves" the issue.

Edit: I could narrow it down to version 10.2.x. The previous version 10.1.0 is not affected by this problem.

madmanxxx · 2023-12-06T20:28:34Z

@lukasmahr The problem is not bound to any particular server, because the problem is within NPM. In fact I would even say that some servers (namely https://registry.npmjs.org/) are broken, because they tolerate malformed requests in the first place. By doing so the problem was very hard to debug.

Reading your post again though I have second thoughts, because the problem reported by the server is HTTP 403, which in normal circumstances means access denied. To confirm the issue is caused by NPM you may try to send a request with malformed path and see what happens.

lukasmahr · 2023-12-07T09:09:05Z

@lukasmahr The problem is not bound to any particular server, because the problem is within NPM. In fact I would even say that some servers (namely https://registry.npmjs.org/) are broken, because they tolerate malformed requests in the first place. By doing so the problem was very hard to debug.

Reading your post again though I have second thoughts, because the problem reported by the server is HTTP 403, which in normal circumstances means access denied. To confirm the issue is caused by NPM you may try to send a request with malformed path and see what happens.

@madmanxxx Indeed, it seems to be caused by the malformed path. I tested by running NPM with --verbose flag to get the URLs that return a 403. Copying and requesting exactly those URLs with curl works perfectly fine and returns a 200.
So apparently at least the URL that NPM logs are okay. However, when modifying the URL by adding a character somewhere to the query params causes the 403.

shaunxu · 2023-12-19T03:59:24Z

I have exactly same issue when install packages from github private packages via proxy. I have npm 10.2.5 installde.

bodograumann · 2024-01-03T08:31:17Z

Also seeing the 400 Bad Request behind proxy with github package registry after updating npm.
Did anybody find a workaround, or do I need to downgrade?

Btw: nodejs 20.9.0 seems to be the last version that bundles the working npm 10.1.0 according to https://nodejs.org/dist/index.json

raihle · 2024-01-12T09:06:12Z

I am seeing similar behavior (404 Not Found) using npm 10.2+ with private NPM registries (JFrog Artifactory) and a corporate proxy. The issue does not appear with npm 9 or 10.1, when bypassing the proxy, or when using the "normal" NPM registry - all three factors are necessary.

Request parameters in npm 10.1.0 (results in 304 Not Modified):

GET /artifactory/api/npm/xxx-virtual/react HTTP/1.1
accept: application/json
authorization: Basic xxxx
npm-auth-type: web
npm-command: view
pacote-pkg-id: registry:react
pacote-req-type: packument
pacote-version: 17.0.4
user-agent: npm/10.1.0 node/v20.9.0 darwin arm64 workspaces/false
if-none-match: 66ce436a0d8e65ca1eb557c710e6c355e6543ec3
connection: keep-alive
Accept-Encoding: gzip,deflate
Host: jfrog.xxx.io
content-length: 0

Request parameters in npm 10.2.3 (results in 404 Not Found):

GET https://jfrog.xxx.io:443/artifactory/api/npm/xxx-virtual/react HTTP/1.1
accept: application/json
authorization: Basic xxxx
npm-auth-type: web
npm-command: view
pacote-pkg-id: registry:react
pacote-req-type: packument
pacote-version: 17.0.4
user-agent: npm/10.2.3 node/v20.10.0 darwin arm64 workspaces/false
if-none-match: 66ce436a0d8e65ca1eb557c710e6c355e6543ec3
connection: keep-alive
Accept-Encoding: gzip,deflate
Host: jfrog.xxx.io
Proxy-Connection: Keep-Alive
content-length: 0

The differences are:

The inclusion of scheme, host, and port in what would normally be the path
Updated "user-agent" header
Addition of "Proxy-Connection: Keep-Alive" (this may not be due to npm version, it just seems to sometimes be there and sometimes not...)

Of these, only the first seems like a potential explanation, though it may be a red herring.

LaughDonor · 2024-01-16T22:51:48Z

I've also run into this issue with a private repository (using //repo.url/:_auth instead of :_authToken), and setting Node to the latest LTS (20.11.0) as of now, and forcing npm to use 10.1.0 (since it's bundled with 10.2.4, and 10.3.0 is the latest and still doesn't work) works for us.

hax · 2024-01-18T03:30:02Z

It seems #6901 is the same bug.

My current workaround:

Using node v20.0.0 (which use npm 9.x)
Using cnpm client or yarn to publish/install private packages
Do not use proxy 😜

candrews · 2024-01-18T15:40:04Z

npm (I'm using version 10.2.3) is also using the wrong protocol to make the request.

When using a proxy with the http protocol, the GET request for the .tgz file is always over the http protocol, even went package-lock.json or npm config set registry specifies an https protocol registry.

It appears that npm may be incorrectly applying the proxy's protocol to the registry URL.

gitnubster · 2024-01-25T13:28:41Z

Same issue. I have a package-lock.json with lockfile version set to 3. Giving the same headaches with npm 10.2.3 as others here, but with azure devops packages. Changing the lockfile version to 1 does something different though...
First it errors with 403, but a second npm ci it succeeds... with the packages all installed ❓ ❓ ❓ Magic happens here.

Adding another package never downloaded before on the machine, and having package-lock.json set back to lockfileVersion 3 still works (but then no cache cleared or node_modules removed).

gitnubster · 2024-01-29T15:03:14Z

Behind a proxy, refreshed token, npm cache clear -force, removed project's node_modules folder, tried npm ci twice and then it seems to have packages downloaded. Subsequent installing random packages does seem to work also.

So without changing lockfileVersion. Thing I can think of is some internal variable or lower level http caching is playing a role here. Perhaps something with npm cli agent, as suggested by @rosen-dimitrov here?

options.secureEndpoint is not always defined. There is however, a function to properly identify the type of connection in the base class, so intead of rolling the own check, relly on the base class implementation. Fixes npm#7024 Contributed by STMicroelectronics Signed-off-by: Torbjörn SVENSSON <torbjorn.svensson@foss.st.com>

options.secureEndpoint is not always defined. There is however, a function to properly identify the type of connection in the base class, so intead of rolling the own check, relly on the base class implementation. Fixes npm/cli#7024 Contributed by STMicroelectronics Signed-off-by: Torbjörn SVENSSON <torbjorn.svensson@foss.st.com>

options.secureEndpoint is not always defined. There is however, a function to properly identify the type of connection in the base class, so intead of rolling the own check, relly on the base class implementation. Fixes npm#87 and npm/cli#7024 Contributed by STMicroelectronics Signed-off-by: Torbjörn SVENSSON <torbjorn.svensson@foss.st.com>

`options.secureEndpoint` is not always defined. There is however, a function to properly identify the type of connection in the base class, so instead of rolling the own check, rely on the base class implementation. Fixes npm/cli#7024 Contributed by STMicroelectronics

grahamb · 2024-02-01T19:49:35Z

This isn't fixed yet; there's no shipped version of npm that resolves this issue. Please reopen.

ljharb · 2024-02-01T19:52:12Z

@grahamb issues on github are generally closed when the fix is merged, not when it's released.

grahamb · 2024-02-01T19:53:28Z

Yup, I'm aware of how auto-closing issues work. As the opener of this issue, however, I haven't confirmed it's fixed the issue, so it isn't "fixed" yet.

johnjardinemd · 2024-03-01T16:08:06Z

I spent a few days looking into this issue and here are my findings:

Using npm@10.5.0 I received 403s against my AzureDevops private NPM feed:

npm ERR! 403 403 Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.

Using a MITM I was able to see that the following requests take place:

GET https://myorg.pkgs.visualstudio.com/_packaging/MyRepo/npm/registry/somepackage/-/somepackage-8.12.0.tgz
Results in a 303 redirect to
https://zrtvsblobprodeus2154.blob.core.windows.net/b-x.blob[...]

Which causes in a second GET request to the blob. The blob request includes an Authorization header and results in a 403 error:

<?xml version="1.0" encoding="utf-8"?>
<Error>
    <Code>AuthenticationFailed</Code>
    <Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:0000000-000-0000-0000-000000000000
Time:2024-03-01T16:01:47.1290272Z</Message>
</Error>

Note that is above error was seen by our developers who authenticated using vsts-auth where a "Basic" auth token was passed as the auth header. Our build agent seems to pass a "Bearer" token and resulted in a slightly different error (this time it's a 400):

<?xml version="1.0" encoding="utf-8"?>
<Error>
    <Code>InvalidAuthenticationInfo</Code>
    <Message>Authentication information is not given in the correct format. Check the value of Authorization header.
RequestId:0000000-000-0000-0000-000000000000
Time:2024-03-01T16:14:56.3864768Z</Message>
</Error>

However, using npm@9.8.1 the same request to the blob does not include an Authorization header and it succeeds with a 200 response.

I even replayed my request that resulted in the 403 in postman, disabled just the Authorization header, and it succeeded.

My team has seen very intermittent behaviour. Sometimes things work, sometimes they don't. But I guess we'll stop asking questions and simply use npm@9.x until things stabilize.

heath-freenome · 2024-03-01T17:49:57Z

I gave up on this issue getting fixed and switched over all of my repos and CI/CD system to use pnpm.

raihle · 2024-03-12T10:04:05Z

This fix was included in v10.5.0 (bundled with Node 21.7.1).

gitnubster · 2024-03-13T09:33:52Z

This fix was included in v10.5.0 (bundled with Node 21.7.1).

Thanks for mentioning! I hope the fix will land very soon in node v20 and v18 as well.

grahamb · 2024-03-15T17:47:12Z

I can confirm that npm@10.5.0 fixes the issue for me as well. I second the hope for it landing in Node LTS as well.

heath-freenome · 2024-03-19T19:03:50Z

I third that 10.5.0 lands in Node 20

engineerakki · 2024-03-21T13:06:06Z

Works well with npm 10.5.0

heath-freenome · 2024-04-05T16:45:07Z

In my Dockerfiles running in a CI/CD system on GitLab I'm still seeing intermittent timeout errors:

#15 470.1 npm ERR! code ETIMEDOUT
#15 470.1 npm ERR! syscall connect
#15 470.1 npm ERR! errno ETIMEDOUT
#15 470.1 npm ERR! network request to https://registry.npmjs.org/xtend/-/xtend-2.1.2.tgz failed, reason: connect ETIMEDOUT 104.16.2.35:443
#15 470.1 npm ERR! network This is a problem related to network connectivity.
#15 470.1 npm ERR! network In most cases you are behind a proxy or have bad network settings.
#15 470.1 npm ERR! network 
#15 470.1 npm ERR! network If you are behind a proxy, please make sure that the
#15 470.1 npm ERR! network 'proxy' config is set properly.  See: 'npm help config'
#15 470.1 
#15 470.1 npm ERR! A complete log of this run can be found in: /root/.npm/_logs/2024-04-05T09_00_13_641Z-debug-0.log
#15 ERROR: process "/bin/sh -c npm run artifactregistry-login\nnpm ci\n" did not complete successfully: exit code: 1

grahamb added Bug thing that needs fixing Needs Triage needs review for next steps Release 10.x labels Nov 25, 2023

Torbjorn-Svensson mentioned this issue Jan 31, 2024

Properly differintiate http and https proxies #7187

Closed

Torbjorn-Svensson mentioned this issue Jan 31, 2024

fix: properly differentiate http and https proxies npm/agent#92

Merged

wraithgar closed this as completed in npm/agent#92 Feb 1, 2024

gitnubster mentioned this issue Feb 15, 2024

[BUG] npm http fetch GET 403 durign download from azure feed #7210

Closed

2 tasks

benq-tyreke-t-yang mentioned this issue May 14, 2024

[BUG] npm 10.2.0 npm publish to github get 400 Bad request response #6901

Closed

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG] npm@10 refuses to install packages from GitHub package registry #7024

[BUG] npm@10 refuses to install packages from GitHub package registry #7024

grahamb commented Nov 25, 2023

marshalys commented Dec 1, 2023

bacterik commented Dec 1, 2023 •

edited

Loading

madmanxxx commented Dec 4, 2023

bacterik commented Dec 5, 2023

lukasmahr commented Dec 6, 2023 •

edited

Loading

madmanxxx commented Dec 6, 2023 •

edited

Loading

lukasmahr commented Dec 7, 2023 •

edited

Loading

shaunxu commented Dec 19, 2023

bodograumann commented Jan 3, 2024 •

edited

Loading

raihle commented Jan 12, 2024 •

edited

Loading

LaughDonor commented Jan 16, 2024

hax commented Jan 18, 2024 •

edited

Loading

candrews commented Jan 18, 2024 •

edited

Loading

gitnubster commented Jan 25, 2024 •

edited

Loading

gitnubster commented Jan 29, 2024

grahamb commented Feb 1, 2024

ljharb commented Feb 1, 2024

grahamb commented Feb 1, 2024

johnjardinemd commented Mar 1, 2024 •

edited

Loading

heath-freenome commented Mar 1, 2024

raihle commented Mar 12, 2024

gitnubster commented Mar 13, 2024

grahamb commented Mar 15, 2024

heath-freenome commented Mar 19, 2024

engineerakki commented Mar 21, 2024

heath-freenome commented Apr 5, 2024

[BUG] npm@10 refuses to install packages from GitHub package registry #7024

[BUG] npm@10 refuses to install packages from GitHub package registry #7024

Comments

grahamb commented Nov 25, 2023

Is there an existing issue for this?

This issue exists in the latest npm version

Current Behavior

Expected Behavior

Steps To Reproduce

Environment

marshalys commented Dec 1, 2023

bacterik commented Dec 1, 2023 • edited Loading

madmanxxx commented Dec 4, 2023

bacterik commented Dec 5, 2023

lukasmahr commented Dec 6, 2023 • edited Loading

madmanxxx commented Dec 6, 2023 • edited Loading

lukasmahr commented Dec 7, 2023 • edited Loading

shaunxu commented Dec 19, 2023

bodograumann commented Jan 3, 2024 • edited Loading

raihle commented Jan 12, 2024 • edited Loading

LaughDonor commented Jan 16, 2024

hax commented Jan 18, 2024 • edited Loading

candrews commented Jan 18, 2024 • edited Loading

gitnubster commented Jan 25, 2024 • edited Loading

gitnubster commented Jan 29, 2024

grahamb commented Feb 1, 2024

ljharb commented Feb 1, 2024

grahamb commented Feb 1, 2024

johnjardinemd commented Mar 1, 2024 • edited Loading

heath-freenome commented Mar 1, 2024

raihle commented Mar 12, 2024

gitnubster commented Mar 13, 2024

grahamb commented Mar 15, 2024

heath-freenome commented Mar 19, 2024

engineerakki commented Mar 21, 2024

heath-freenome commented Apr 5, 2024

bacterik commented Dec 1, 2023 •

edited

Loading

lukasmahr commented Dec 6, 2023 •

edited

Loading

madmanxxx commented Dec 6, 2023 •

edited

Loading

lukasmahr commented Dec 7, 2023 •

edited

Loading

bodograumann commented Jan 3, 2024 •

edited

Loading

raihle commented Jan 12, 2024 •

edited

Loading

hax commented Jan 18, 2024 •

edited

Loading

candrews commented Jan 18, 2024 •

edited

Loading

gitnubster commented Jan 25, 2024 •

edited

Loading

johnjardinemd commented Mar 1, 2024 •

edited

Loading