nginx ingress controller not able to resolve namespace/service (kube 1.6)

[fritzb]

I tried to follow the example here: https://github.com/kubernetes/ingress/tree/master/examples/deployment/nginx to deploy nginx ingress controller, but not having a success with kube 1.6 (with kubeadm). The ingress controller failed to resolve the service backend, and the default backend is there. Any idea ? This is not a problem in Kube 1.5.2

kubectl logs nginx-ingress-controller-1543599588-6h81t -n kube-system

I0415 15:53:29.382927 7 launch.go:96] &{NGINX 0.9.0-beta.3 git-3dd7461 git@github.com:ixdy/kubernetes-ingress.git}
I0415 15:53:29.382976 7 launch.go:99] Watching for ingress class: nginx
I0415 15:53:29.383424 7 launch.go:245] Creating API server client for https://10.254.0.1:443
F0415 15:53:29.437150 7 launch.go:113] no service with name kube-system/backend found: the server does not allow access to the requested resource (get services backend)

kubectl get svc -n kube-system backend
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
backend 10.254.48.240 80/TCP 11m

[aledbf]

@fritzb this issue es related to RBAC. Please check #266

[perrefe]

I'm having the same issue in Kube 1.5 with a simple ABAC configuration policies, Do you know if there's an issue also?

[aledbf]

@perrefe please check the pod logs.

[perrefe]

The pod log is equivalent to the one reported by @fritzb

F0615 19:51:00.064955       1 main.go:121] no service with name nginx-ingress/default-http-backend found: the server does not allow access to the requested resource (get services default-http-backend)

Ingress controller image: gcr.io/google_containers/nginx-ingress-controller:0.8.3

As you can see here, that service is present:

$ kubectl get svc -n nginx-ingress
NAME                   CLUSTER-IP   EXTERNAL-IP   PORT(S)                      AGE
default-http-backend   10.3.0.14    <none>        80/TCP                       11h

This ingress used to work until I configured ABAC authorization mode in API server, mi policy file contains the following:

{"apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": {"user":"kube-admin", "namespace": "*", "resource": "*", "apiGroup": "*", "nonResourcePath": "*"}}
{"apiVersion":"abac.authorization.kubernetes.io/v1beta1","kind":"Policy","spec":{"user":"system:serviceaccount:kube-system:default","namespace":"*","resource":"*","apiGroup":"*"}}
{"apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": {"user":"kube-worker-1", "namespace": "*", "resource": "*", "apiGroup": "*", "nonResourcePath": "*"}}
{"apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": {"user":"kube-worker-2", "namespace": "*", "resource": "*", "apiGroup": "*", "nonResourcePath": "*"}}
{"apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": {"user":"kube-worker-3", "namespace": "*", "resource": "*", "apiGroup": "*", "nonResourcePath": "*"}}

Unfortunately, I haven't found anything about kube 1.5 w/ABAC + nginx-ingress-controller.

I'll appreciate any help!

[aledbf]

@perrefe please update to the latest beta 0.9-beta.8

[perrefe]

Done, but I'm getting the same error:

I0616 03:27:52.042401       7 launch.go:101] &{NGINX 0.9.0-beta.8 git-245e6b0 https://github.com/kubernetes/ingress}
I0616 03:27:52.042464       7 launch.go:104] Watching for ingress class: nginx
I0616 03:27:52.043117       7 launch.go:257] Creating API server client for https://10.3.0.1:443
F0616 03:27:52.140699       7 launch.go:118] no service with name nginx-ingress/default-http-backend found: Forbidden: "/api/v1/namespaces/nginx-ingress/services/default-http-backend" (get services default-http-backend)

[aledbf]

@perrefe I asked to liggit about ^^ and this was the response

when you enable authorization, you have to specifically allow each API client the example ABAC policy only allows system:serviceaccount:kube-system:default to do things not sure what service account the ingress controller runs as

[perrefe]

Thanks @aledbf, I finally made it work configuring these three policies:

Everything readonly (maybe this could be more specific):
{"apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": {"user":"system:serviceaccount:nginx-ingress:ingress", "namespace": "*", "resource": "*", "apiGroup": "*", "nonResourcePath": "*", "readonly": true}}

Permission to write on ingresses resources across cluster:
{"apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": {"user":"system:serviceaccount:nginx-ingress:ingress", "namespace": "*", "resource": "ingresses"}}

Permision to write configmaps on self namespace:
{"apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": {"user":"system:serviceaccount:nginx-ingress:ingress", "namespace": "nginx-ingress", "resource": "configmaps"}}

[pawelprazak]

This worked for me:

  { "apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": { "user": "system:serviceaccount:gateway:default",            "namespace": "*",                    "resource": "*", "apiGroup": "*", "readonly": true } }
  { "apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": { "user": "system:serviceaccount:gateway:default",            "namespace": "*",                    "resource": "ingresses", "apiGroup": "*" } }
  { "apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": { "user": "system:serviceaccount:gateway:default",            "namespace": "*",                    "resource": "events" } }
  { "apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": { "user": "system:serviceaccount:gateway:default",            "namespace": "gateway",              "resource": "configmaps" } }
  { "apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": { "user": "system:serviceaccount:gateway:default",            "namespace": "gateway",              "resource": "endpoints" } }

[Navruzjon]

@pawelprazak @perrefe Can you pls specify how to configure those? or any link could help. thanks

[pawelprazak]

@Navruzjon at this point in time you shouldn't, ABAC is deprecated

anyway, here are the docs: https://kubernetes.io/docs/admin/authorization/abac