docker push images to harbor occured unauthorized to access

[KevinWang2015]

version: harbor 2.0

docker push images to harbor occured unauthorized to access repository:library/redis action:push with admin user
but If the docker client is in another datacenter, it is ok to push the image

is there someone give me some advices? thanks a lot

[KevinWang2015]

docker login successfully。 Pls refer to the docker push logs from /var/log/harbor/core.log

Nov 13 11:35:11 172.22.0.1 core[30822]: 2020/11/13 03:35:11.688 #33[1;44m[D]#33[0m [transaction.go:62] | 127.0.0.1|#33[97;42m 200 #33[0m| 355.486¦Ìs| match|#33[97;44m GET #33[0m /api/v2.0/ping r:/api/v2.0/ping
Nov 13 11:35:12 172.22.0.1 core[30822]: 2020/11/13 03:35:12.799 #33[1;44m[D]#33[0m [middleware.go:52] | 10.59.99.181|#33[97;42m 200 #33[0m| 25.858451ms| match|#33[97;44m GET #33[0m /service/token r:/service/token
Nov 13 11:35:19 172.22.0.1 core[30822]: 2020/11/13 03:35:19.443 #33[1;44m[D]#33[0m [middleware.go:52] | 10.59.99.181|#33[97;43m 401 #33[0m| 895.406¦Ìs| match|#33[97;44m GET #33[0m /v2/ r:/v2/*
Nov 13 11:35:19 172.22.0.1 core[30822]: 2020/11/13 03:35:19.585 #33[1;44m[D]#33[0m [middleware.go:52] | 10.59.99.181|#33[97;42m 200 #33[0m| 28.765171ms| match|#33[97;44m GET #33[0m /service/token r:/service/token
Nov 13 11:35:19 172.22.0.1 core[30822]: 2020/11/13 03:35:19.915 #33[1;44m[D]#33[0m [transaction.go:62] | 10.59.99.181|#33[97;42m 202 #33[0m| 133.615419ms| match|#33[97;46m POST #33[0m /v2/library/redis/blobs/uploads/ r:/v2//blobs/uploads
Nov 13 11:35:19 172.22.0.1 core[30822]: 2020/11/13 03:35:19.921 #33[1;44m[D]#33[0m [transaction.go:62] | 10.59.99.181|#33[97;42m 202 #33[0m| 138.680945ms| match|#33[97;46m POST #33[0m /v2/library/redis/blobs/uploads/ r:/v2//blobs/uploads
Nov 13 11:35:19 172.22.0.1 core[30822]: 2020/11/13 03:35:19.921 #33[1;44m[D]#33[0m [transaction.go:62] | 10.59.99.181|#33[97;42m 202 #33[0m| 141.106469ms| match|#33[97;46m POST #33[0m /v2/library/redis/blobs/uploads/ r:/v2//blobs/uploads
Nov 13 11:35:19 172.22.0.1 core[30822]: 2020/11/13 03:35:19.921 #33[1;44m[D]#33[0m [transaction.go:62] | 10.59.99.181|#33[97;42m 202 #33[0m| 140.516994ms| match|#33[97;46m POST #33[0m /v2/library/redis/blobs/uploads/ r:/v2//blobs/uploads
Nov 13 11:35:19 172.22.0.1 core[30822]: 2020/11/13 03:35:19.922 #33[1;44m[D]#33[0m [transaction.go:62] | 10.59.99.181|#33[97;42m 202 #33[0m| 120.43912ms| match|#33[97;46m POST #33[0m /v2/library/redis/blobs/uploads/ r:/v2//blobs/uploads
Nov 13 11:35:20 172.22.0.1 core[30822]: 2020/11/13 03:35:20.025 #33[1;44m[D]#33[0m [transaction.go:62] | 10.59.99.181|#33[97;43m 401 #33[0m| 4.321904ms| match|#33[97;42m PATCH #33[0m /v2/library/redis/blobs/uploads/726bcef5-9c9b-4a53-9de5-d8e331856162 r:/v2//blobs/uploads/:session_id
Nov 13 11:35:20 172.22.0.1 core[30822]: 2020/11/13 03:35:20.025 #33[1;44m[D]#33[0m [transaction.go:62] | 10.59.99.181|#33[97;43m 401 #33[0m| 4.233904ms| match|#33[97;42m PATCH #33[0m /v2/library/redis/blobs/uploads/5c8ff720-9462-4107-b1f6-25799df7b55a r:/v2//blobs/uploads/:session_id
Nov 13 11:35:20 172.22.0.1 core[30822]: 2020/11/13 03:35:20.027 #33[1;44m[D]#33[0m [transaction.go:62] | 10.59.99.181|#33[97;43m 401 #33[0m| 5.953989ms| match|#33[97;42m PATCH #33[0m /v2/library/redis/blobs/uploads/5046293d-8656-4e1e-8c2b-deef27e47df6 r:/v2//blobs/uploads/:session_id
Nov 13 11:35:20 172.22.0.1 core[30822]: 2020/11/13 03:35:20.097 #33[1;44m[D]#33[0m [transaction.go:62] | 10.59.99.181|#33[97;43m 401 #33[0m| 4.114612ms| match|#33[97;42m PATCH #33[0m /v2/library/redis/blobs/uploads/e82db9ed-f0da-46fa-8ca5-03db17d5e3a8 r:/v2//blobs/uploads/:session_id
Nov 13 11:35:20 172.22.0.1 core[30822]: 2020/11/13 03:35:20.098 #33[1;44m[D]#33[0m [transaction.go:62] | 10.59.99.181|#33[97;43m 401 #33[0m| 4.984567ms| match|#33[97;42m PATCH #33[0m /v2/library/redis/blobs/uploads/e2c70dbc-28f9-4698-b933-a94fdb84b802 r:/v2//blobs/uploads/:session_id
Nov 13 11:35:20 172.22.0.1 core[30822]: 2020/11/13 03:35:20.245 #33[1;44m[D]#33[0m [transaction.go:62] | 10.59.99.181|#33[97;42m 202 #33[0m| 24.228976ms| match|#33[97;46m POST #33[0m /v2/library/redis/blobs/uploads/ r:/v2/*/blobs/uploads

[reasonerjt]

@KevinWang2015
The log messages you pasted do not have relevant errors.

Under the hood docker will get a token via service token and use that token to push image.
Does the docker push fail immediately with unauthorized or failed in the middle of data transfer?

[KevinWang2015]

@KevinWang2015
The log messages you pasted do not have relevant errors.

Under the hood docker will get a token via service token and use that token to push image.
Does the docker push fail immediately with unauthorized or failed in the middle of data transfer?

in the middle of data transfer: unauthorized: unauthorized to access repository: evolve/redis, action: push: unauthorized to access repository: evolve/redis, action: push

[heww]

Hi, @KevinWang2015 it's really hard to debug the problem in this issue. Can you install an HTTP proxy tool and configure the Docker daemon to use the HTTP proxy and capture the HTTP request/response during the failed pushing?

whistle (https://github.com/avwo/whistle) a good tool to capture the HTTP request/response. Note that after installation, enable the HTTPS feature.

Then trust the rootca of the whistle in the system and enable HTTP proxy in docker daemon. See here https://docs.docker.com/config/daemon/systemd/#httphttps-proxy to configure the HTTPS proxy for the docker daemon.

In the network panel of the whistle, find the failed request during pushing, and show it to me, thanks!

[KevinWang2015]

@heww hi, i can't install whistle by software installation restrictions。

1. has any other way？

2. Under what circumstances will the backend application return 401 while docker push image ？

[heww]

This issue may be caused by docker-credential-helpers, see #13553 and docker/docker-credential-helpers#154 for more information.

[zhaogaolong]

do you have https gateway server proxy harbor(http)?

| https nginx gateway |
            |
           \|/
      -------------
      | http harbor |
      -------------

if this Architecture ， can change proxy mode nginx.conf

location /v2/{
  proxy_set_header X-Forwarded-Proto https;  // default $schema
}

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[DaNussi]

I solved the issue by changeing my reverse proxy settings from http to https. (TLS is disabled in harbor) I alsow enabled http2 support in.

I hope this helps someone. (: