Response "Unauthorized"

[ScubaDrew]

Issue details

I followed the steps to install and setup the proxy as detailed in the README.md

When I point my webbrowser at the proxy address I just get back an "unauthorized" response.

Environment

Dashboard version: latest as of 5/28/16
Kubernetes version: 1.2.2
Operating system: core

Steps to reproduce

1. setup cluster on AWS per https://coreos.com/kubernetes/docs/latest/kubernetes-on-aws.html
2. install dashboard as described in README.md
3. I have an ubuntu server that I run kubectl proxy --address="172.20.32.54" --port=9090
4. point webbrowser on osx system at the ubuntu server address (...32.54 address above), get successful http connection but response is simply "unauthorized"

All other kubectl commands work correctly.

kubectl cluster-info
Kubernetes master is running at https://kub2.drewoconnor.com
Heapster is running at https://kub2.drewoconnor.com/api/v1/proxy/namespaces/kube-system/services/heapster
KubeDNS is running at https://kub2.drewoconnor.com/api/v1/proxy/namespaces/kube-system/services/kube-dns
kubernetes-dashboard is running at https://kub2.drewoconnor.com/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard

Observed result

Unauthorized

Expected result

Dashboard

[bryk]

Can you kubectl get pods --namespace=kube-system and kubectl logs <pod-of-dashboard> --namespace=kube-system?

[ScubaDrew]

Hi bryk, I found the issue here.

I was using the kubectl proxy command as noted above on an ubuntu server, specifying the IP address in the command. I was when pointing a browser on an OSX machine at the ubuntu server's IP. The connection worked, and I got the unauthorized response. I found that if i used "kubectl --port=9090" and then did a wget to localhost, the request worked as it should.

This appears to be an issue with the proxy command in kubectl. Or perhaps they intended to limit connections to localhost.

I don't believe this is an issue with the web ui.

Thanks,
Drew

[bryk]

@ScubaDrew Yeah, that's what I expected. I'm closing this issue. Please reopen if needed.

[EamonZhang]

I have the same issue .

$kubectl proxy --port=9090
Starting to serve on 127.0.0.1:9090

curl 127.0.0.1:9090
<h3>Unauthorized</h3>

curl 127.0.0.1:9090/ui
<a href="/ui/">Moved Permanently</a>.

What can i do, thanks

[bryk]

Yeah, that's correct that it is moved permanently. Can you open the URL with a browser?

[EamonZhang]

@bryk

$kubectl proxy --address="10.2.0.10"  --port=9090
Starting to serve on 10.2.0.10:9090

or

$kubectl proxy --address="0.0.0.0"  --port=9090
Starting to serve on 0.0.0.0:9090

in browser window

http://10.2.0.10:9090/ui

Unauthorized

[ScubaDrew]

@EamonZhang it only works when the url is localhost. Accessing it via IP address is... Unauthorized by design.

[EamonZhang]

@ScubaDrew

Server computer have no browser to be used.
Does nginx proxy support . or other measures ?

Thanks

[bryk]

If your master is publicly accessible you can access it to see the UI: https://master/ui or your clients can use kubectl proxy on their machines. Finally, you can expose the UI as external service and access it from outside world.

[hwinkel]

I have the master running on a public IP but get unauthorized un http://ip/ui

the dashboard pod self is not running on the master node. But another one.
kubectl cluster-info does not show the kubernetes-dashboard. But the service is listed and the pod as well

How do I expose the UI? any hints?

[natejoebott]

I have the same issue as hwinkel above. Just installed K8 and the dashboard per https://github.com/kubernetes/dashboard#kubernetes-dashboard

I get the 'unauthorized" message when accessing https:///ui

I am using a mozilla browser from a windows client, so the kubectl proxy approach doesn't seem appropriate. What I am missing here?

[remonlam]

Same here running 1.4 installed it following the guide at http://kubernetes.io/docs/getting-started-guides/kubeadm/.

However when installing the dashboard/UI it seems to be running but get an "Unauthorized"

Followed the guide at http://kubernetes.io/docs/user-guide/ui/ to install the UI basically just run "kubectl create -f https://rawgit.com/kubernetes/dashboard/master/src/deploy/kubernetes-dashboard.yaml"

kubectl get pods --all-namespaces

NAMESPACE     NAME                                         READY     STATUS    RESTARTS   AGE
kube-system   etcd-kube-master.net.loc                      1/1       Running   1          19m
kube-system   kube-apiserver-kube-master.net.loc            1/1       Running   1          20m
kube-system   kube-controller-manager-kube-master.net.loc   1/1       Running   1          20m
kube-system   kube-discovery-982812725-s79yq                1/1       Running   1          20m
kube-system   kube-dns-2247936740-b9a2o                     3/3       Running   3          20m
kube-system   kube-proxy-amd64-4p9vg                        1/1       Running   1          20m
kube-system   kube-proxy-amd64-cbbrm                        1/1       Running   0          20m
kube-system   kube-proxy-amd64-ela05                        1/1       Running   0          20m
kube-system   kube-scheduler-kube-master.net.loc            1/1       Running   1          19m
kube-system   kubernetes-dashboard-1655269645-arfpw         1/1       Running   0          15m
kube-system   weave-net-cvcgd                               2/2       Running   0          19m
kube-system   weave-net-hxkwf                               2/2       Running   2          19m
kube-system   weave-net-pwuto                               2/2       Running   0          19m

@natejoebott are you running 1.4?

[natejoebott]

Right, I too installed it via the beta version of Kubeadm with 1.4. Upon further investigation this may be expected behavior based upon the limitation number 4 - There is not yet an easy way to generate a kubeconfig file which can be used to authenticate to the cluster remotely with kubectl.

[colemickens]

Are people expecting the dashboard to be publicly available without auth? If this were really the case, everyone would be exposing write access for their cluster to the anonymous world. Unless I'm missing a detail here?

[natejoebott]

No, but there doesn't seem to be simple way to auth. In the documentation for the dashboard, it provides the follow guidance after installation:
And then navigate to https:///ui
If it asks password, use $ kubectl config view to find it.

I was not prompted for a password, nor does the kubectl config view provide any indication of a password. Next up is using the alternative proxy method; however, only localhost can be used - http://localhost:8001/ui - thus unless you are using a mac or linux host where the kubectl proxy script was invoked will this work.

[colemickens]

Presumably you're using client-cert auth then if there's no password/token in kubeconfig. You can configure your browser to send the relevant client certificate, but it's usually not-straightforward in my experience.

kubectl proxy is certainly easier. There is a build of kubectl for Windows and if you want, you can run it with flags that enable it to listen on all interfaces and for connections from all hosts so that you can run it on a server machine and allow anyone to hit it (this is obviously insecure).

(I'm working on an example of how you can run a reverse-proxy with auth in front of the dashboard that should make things like this easier.)

[bryk]

(I'm working on an example of how you can run a reverse-proxy with auth in front of the dashboard that should make things like this easier.)

Can you share any details here? I'd love us to incorporate something like this to default install, so that folks can expose the UI to external world.

[colemickens]

This is the idea: kubernetes-retired/contrib#1492. I still haven't had time to put the oauth2_proxy configuration together. Doesn't really solve for out-of-the-box though, as oauth2_proxy will require a configmap/secret with oauth2 secrets to work the way I'm imagining.

[bryk]

All right. Share anything you make work :) We need to explore all possible solutions, because, eventually, we need to bake a solution to this into Dashboard.

[dannyman]

Heya, so we are all actively commenting on an issue that is closed since May .. begs the question as to whether this issue is Closed? (Maybe a scope change...)

Here's where we are at, total newbs:

1. Install Kubernetes with fancy new kubeadm tool: http://kubernetes.io/docs/getting-started-guides/kubeadm/
2. Per above "Explore other add-ons" ... http://kubernetes.io/docs/admin/addons/
3. "Dashboard is a dashboard web interface for Kubernetes." -- wow this sounds useful for us newbs!
4. https://github.com/kubernetes/dashboard#kubernetes-dashboard
5. kubectl create -f https://rawgit.com/kubernetes/dashboard/master/src/deploy/kubernetes-dashboard.yaml
6. https:///ui --> Unauthorized

root@kub-test0:~# kubectl config view
apiVersion: v1
clusters: []
contexts: []
current-context: ""
kind: Config
preferences: {}
users: []

8. Find this issue and:

root@kub-test0:~# wget --no-check-certificate https://localhost/ui
--2016-10-04 15:28:58--  https://localhost/ui
Resolving localhost (localhost)... ::1, 127.0.0.1
Connecting to localhost (localhost)|::1|:443... connected.
WARNING: cannot verify localhost's certificate, issued by ‘CN=kubernetes’:
  Unable to locally verify the issuer's authority.
WARNING: no certificate subject alternative name matches
    requested host name ‘localhost’.
HTTP request sent, awaiting response... 401 Unauthorized

Username/Password Authentication Failed.

So since this is my first time on a test cluster I can do this:
kubectl proxy --address 0.0.0.0
Nope: <h3>Unauthorized</h3>

Maybe helpful would be a pointer on how to add a username/password in the Usage section at https://github.com/kubernetes/dashboard#kubernetes-dashboard

[colemickens]

What URL did you try to hit after running kubectl proxy?

[dannyman]

From my workstation, to external IP of kubernetes cluster Master:
http://10.10.1.188:8001/ --> <h3>Unauthorized</h3>

From localhost on kubernetes cluster Master:
https://localhost/ui --> 401 Unauthorized Username/Password Authentication Failed.

[colemickens]

And what happens if you try, after running kubectl proxy of course:

• http://10.10.1.188:8001/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard/#/workload?namespace=_all
• http://localhost:8001/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard/#/workload?namespace=_all

[dannyman]

Workstation -> http://10.10.1.188:8001/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard/#/workload?namespace=_all -> Unauthorized

Kubenetes Master wget http://localhost:8001/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard/#/workload?namespace=_all I get an index.html that informs me that I am using an outdated browser.

[anutech2001]

@dannyman I got the same problem. So, i try using NodePort and it's work. I can use dashboard right now on the browser.
My env.

• windows 7, vagrant + centos 7, kubernetes 1.4

[traceypooh]

something seems weird here...
like others, I want to use (mac) laptop and (chrome) browser to hit the (great) dashboard gui.

If I port forward from mac over ssh
ssh -A [MASTER-HOSTNAME] -L 8001:127.0.0.1:8001

the browser loads gui but 403s all data.
however, I'm 100% free to "raid" secrets from cmd-line/browser:
curl http://localhost:8001/api/v1/secrets

It would seem, if we (the clients/users/admins of k8 cluster) firewall off the proxy port from outside world, but port forward over ssh, that allowing fully GUI dashboard access would be pretty reasonable, security thinking-wise?

[praseodym]

That only makes sense with RBAC disabled. I'm not sure why the dashboard 403s when not authenticating even though is not required.

If RBAC is enabled (which it is with e.g. kubeadm 1.6), you'll need a way to authenticate as a client, and that is what kubectl proxy does nicely. In that case an SSH tunnel doesn't really make sense.

[codTango]

I have go through this very long thread and yet haven't found a solution for my case. In my scenario, I have a kube cluster with 3 nodes, and they're deployed on VMs that have no monitors attached, neither I want to use X11 to forward it. This is what I need:

1. set up multi users with credentials
2. people can use the credentials to login the dashboard remotely

But now I did this kubectl config set-credentials userA --username=userA --password=pwd, and I got Unauthorized when hitting https://{IP}/ui. Any solution? I'm not that familiar with the other ways like openssl, but tried by following the guide and still no luck.

[traceypooh]

thanks @praseodym for the info.
do you happen to have any pointers or places too look for more info/help on that?

i just spent 3 days in a black hole trying to sort out kubectl v1.6.1 "client" issue compared to working v1.6.0 on sun/mon, bleah! (my ops team has an over-aggressive firewall so made sorting out issues a super challenge..)

(i can get my admin.conf on my laptop and kubectl proxy with it and seems like is connecting -- but same kind of "instantly hides access to the details" (same denied stuff as with trying to port forward a proxy from the server). so i'm probably close!)

[dotw]

The solution #692 (comment) posted above worked for me. Thanks @waynebrantley!

I installed k8s using kubeadm in https://kubernetes.io/docs/getting-started-guides/kubeadm/. I wanted to run kubectl proxy on the master, and access the k8s API from a different host (not the master) using the IP address (not localhost) of the master.

172.18.7.245 is the IP address of my master, and after doing kubectl proxy --port=8181 --address=172.18.7.245 --accept-hosts='^*$' & on the master, I was able to successfully access the k8s API from a different host (with IP address 172.18.7.246 and that pings the master 172.18.7.245) using

curl http://172.18.7.245:8181/api/v1
curl http://172.18.7.245:8181/api/v1/services
curl http://172.18.7.245:8181/api/v1/pods
curl http://172.18.7.245:8181/api/v1/secrets
I was also able to access the k8s API from another machine that can SSH into 172.18.7.246 after setting up SSH port-forwaring ssh -L 8080:172.18.7.245:8181 172.18.7.245 on the machine and using the following commands.

curl http://localhost:8080/api/v1
curl http://localhost:8080/api/v1/services
curl http://localhost:8080/api/v1/pods
curl http://localhost:8080/api/v1/secrets

this works for my case

[zhuroy]

Finally both below works for me cause you need to ensure master can ping pod from worker node, I found that somehow --iface not to be loaded to start the kube-flannel pod as I am using vagrant. use kubectl replace -f kube-flannel.yml --force thus to delete and recreate the source.

My issues, installed a few times and spent two days but no clue.
I want to access cluster from outside host.
1) with kubectl proxy --address PubIP --port=9090 --accept-hosts='^*$'
Almost can access any from outside browser, but not for /ui/, all others like /api/ no problem

2) with NodePort not working for me.
I did see the service and NodePort for dashboard, but PubIP:NodePort not working for me, just show ERR_EMPTY_RESPONSE after a long waiting

[adpit]

@zhuroy Thank you so much for the free time you gave me, I have tried it and it works:

kubectl proxy --address xxx.xx.xx.xx --port=2087 --accept-hosts='^*$'

I access with the browser http://xxx.xx.xx.xx:2087/ui automatically leads to

http://xxx.xx.xx.xx:2087/api/v1/namespaces/kube-system/services/kubernetes-dashboard/proxy/#!/workload?namespace=default

[floreks]

Closing as stale.

[dbreise6583]

EASY ANSWER
You can get around this by not forwarding the host header... in Apache Virtualhost Config:
ProxyPreserveHost Off

[j3ffyang]

I've not tried to configure Apache (or Nginx) web server. Instead, the following works for me

kubectl proxy --address 0.0.0.0 --accept-hosts '.*'

[pgnaleen]

kubectl proxy
running above command from windows worked for me. you have to give admin.conf file using --kubeconfig=="admin.conf"
kubectl kubeconfig=="admin.conf" proxy

[miguelcastilho]

The "Unauthorized" error can be fixed using the disable-filter flag:

$ kubectl proxy --disable-filter=true --address=0.0.0.0

[naisanza]

@colemickens the documentation doesn't say to run kubectl proxy from your machine you're physically on. I kept thinking it meant to run it from master. So that would mean you would need to install all of kubernetes packages on your client machine to use Dashboard via kubectl proxy?

And that would also mean you need a copy of /etc/kubernetes/admin.conf on to your local client as well?

[floreks]

From docs:

kubectl proxy creates proxy server between your machine and Kubernetes API server. By default it is only accessible locally (from the machine that started it).

@naisanza it is not stated because it can be run from any machine and we do not want to suggest anything, and not to make user think that some machine is the only one he can run it from.

Only requirements are kubectl and valid kubeconfig file.

PS. Master node does not require kubectl in any way. It's user's choice to install it there.

[colemickens]

@naisanza The reason I mention it is:

1. Many users in this thread seem to want to access the dashboard from machines that are not part of the cluster.

2. If you run kubectl proxy on the master, you either have to:

   a) tunnel traffic to the master through that proxy
   b) open the proxy to accept traffic from any host and put it on a publicly accessible port

The 2b option is suggested throughout this thread but that means that anyone in the world could then start writing to your cluster. The more secure option is to run kubectl proxy from the same node you're running the browser on.

I think there are additional options now that kube-dashboard offers some alternative authentication options, but that was my reason for recommendation in this thread.

[immarvin]

hi @miguelcastilho , kubectl proxy --disable-filter=true --address=0.0.0.0 works for me, thanks a lot

[k-modi]

I was able to get this working. Hence, sharing it for anyone interested.

BACKGROUND: K8s master on ubuntu VM. Couldn't access dashboard from my local machine (Windows 10).

Solution:
Taking pointers from @seeekr and @groyee, I installed kubectl on my machine and configured it to map to the cluster using https://kubernetes.io/docs/tasks/access-application-cluster/configure-access-multiple-clusters/.

Note: I had to WinSCP and copy over the cert files from ~/kube/certs/ on the ubuntu VM.

Thereafter,

1. Added a new cluster to $HOME/.kube/config using:
   kubectl config set-cluster <custom-cluster-name> --server=<https://{ip-of-ubuntu-master}> --certificate-authority=<path/to/apiserver.pem>
2. Added a new user with:
   kubectl config set-credentials <custom-user-name> --client-certificate=<path/to/ca.pem> --client-key=<path/to/ca-key.pem>
3. Added a context to map the cluster and user together using:
   kubectl config set-context <custom-context-name> --cluster=<custom-cluster-name-from-above> --user=<custom-user-name-from-above>
4. Change/set the current kubectl context to the newly created one:
   kubectl config use-context <custom-context-name>
5. Start proxy
   kubectl proxy
6. Browse to
   http://127.0.0.1:8001/api/v1/namespaces/kube-system/services/kubernetes-dashboard/proxy

Enjoy!!!

[soolaugust]

I try following way to get this working.

kubectl proxy --address 0.0.0.0 --accept-hosts '.*'

[wongoo]

port-forward also works:
sudo kubectl --namespace kube-system port-forward svc/kubernetes-dashboard 443

then access https://localhost

[mrichman]

@soolaugust got me working with kubectl proxy --address 0.0.0.0 --accept-hosts '.*' as well.

There really should be something more obvious like --accept-all-hosts.

[marcosmcp]

Great!

[maelfosso]

Hello,

I am a newbie on Kubernetes. I am running Kubernetes and minikube on Ubuntu server 16.04

I ran kubectl proxy and I got this

$ kubectl proxy
Starting to serve on 127.0.0.1:8001
E0907 13:25:13.009215    1817 proxy_server.go:144] Error while proxying request: dial tcp 192.168.99.100:8443: i/o timeout
E0907 13:25:43.287623    1817 proxy_server.go:144] Error while proxying request: dial tcp 192.168.99.100:8443: i/o timeout
E0907 13:27:57.516225    1817 proxy_server.go:144] Error while proxying request: dial tcp 192.168.99.100:8443: i/o timeout
E0907 13:28:27.599589    1817 proxy_server.go:144] Error while proxying request: dial tcp 192.168.99.100:8443: i/o timeout

I don't understand that error.

Here is my ip addresses

$ ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp7s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN group default qlen 1000
    link/ether 00:26:6c:28:ce:5f brd ff:ff:ff:ff:ff:ff
3: wlp8s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 24:ec:99:48:97:8b brd ff:ff:ff:ff:ff:ff
    inet 192.168.100.19/24 brd 192.168.100.255 scope global dynamic wlp8s0
       valid_lft 170760sec preferred_lft 170760sec
    inet6 fe80::bbd3:3733:b634:a9a0/64 scope link 
       valid_lft forever preferred_lft forever
4: vmnet1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 1000
    link/ether 00:50:56:c0:00:01 brd ff:ff:ff:ff:ff:ff
    inet 192.168.5.1/24 brd 192.168.5.255 scope global vmnet1
       valid_lft forever preferred_lft forever
    inet6 fe80::250:56ff:fec0:1/64 scope link 
       valid_lft forever preferred_lft forever
5: vmnet8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 1000
    link/ether 00:50:56:c0:00:08 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.1/24 brd 192.168.122.255 scope global vmnet8
       valid_lft forever preferred_lft forever
    inet6 fe80::250:56ff:fec0:8/64 scope link 
       valid_lft forever preferred_lft forever
6: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:37:fb:dc:65 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:37ff:fefb:dc65/64 scope link 
       valid_lft forever preferred_lft forever
10: vboxnet1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 0a:00:27:00:00:01 brd ff:ff:ff:ff:ff:ff
661: br-b03b4b67d036: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:e6:30:0f:1f brd ff:ff:ff:ff:ff:ff
    inet 172.28.0.1/16 brd 172.28.255.255 scope global br-b03b4b67d036
       valid_lft forever preferred_lft forever
662: br-54dc24dbbd29: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:af:a2:bb:cb brd ff:ff:ff:ff:ff:ff
    inet 172.29.0.1/16 brd 172.29.255.255 scope global br-54dc24dbbd29
       valid_lft forever preferred_lft forever
668: vboxnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 0a:00:27:00:00:00 brd ff:ff:ff:ff:ff:ff
    inet 192.168.56.1/24 brd 192.168.56.255 scope global vboxnet0
       valid_lft forever preferred_lft forever
    inet6 fe80::800:27ff:fe00:0/64 scope link 
       valid_lft forever preferred_lft forever

$ minikube start
😄  minikube v1.13.0 on Ubuntu 16.04
✨  Using the virtualbox driver based on existing profile
👍  Starting control plane node minikube in cluster minikube
🔄  Restarting existing virtualbox VM for "minikube" ...
🐳  Preparing Kubernetes v1.19.0 on Docker 19.03.12 ...
🔎  Verifying Kubernetes components...
🌟  Enabled addons: default-storageclass, storage-provisioner
🏄  Done! kubectl is now configured to use "minikube" by default

How can I solve it?

[vishal3108]

@dannyman I got the same problem. So, i try using NodePort and it's work. I can use dashboard right now on the browser.
My env.

• windows 7, vagrant + centos 7, kubernetes 1.4

hi ,
Everytime I login k8s dashboard , it ask the token . How did you fix it ?