Using oauth2_proxy with nginx k8s ingress

[nmiculinic]

kubernetes nginx controller supports external auth via:

https://github.com/kubernetes/ingress-nginx/blob/master/docs/user-guide/annotations.md#external-authentication

However, I'm not sure how can I use it since I'm a beginner regarding nginx and k8s nginx controller.

[bismarck]

There is an example in the ingress-nginx repository using external auth:

https://github.com/kubernetes/ingress-nginx/tree/master/docs/examples/external-auth

[ean]

To get deep linking to work I needed to add this patch, it is hacky as the nginx ingress does not encode the query pairs properly:

diff --git a/oauthproxy.go b/oauthproxy.go
index 21e5dfc..275a975 100644
--- a/oauthproxy.go
+++ b/oauthproxy.go
@@ -426,9 +426,27 @@ func (p *OAuthProxy) GetRedirect(req *http.Request) (redirect string, err error)
        }

        redirect = req.Form.Get("rd")
+       if req.URL.Query().Get("rd") != "" {
+               elements := strings.SplitN(req.URL.String(), "/start?rd=", 2)
+               if len(elements) == 2 {
+                       u, err := url.Parse(elements[1])
+                       if err == nil {
+                               redirect = u.Path
+                               if u.RawQuery != "" {
+                                       redirect += "?" + u.RawQuery
+                               }
+                               if u.Fragment != "" {
+                                       redirect += "#" + u.Fragment
+                               }
+                       } else {
+                               fmt.Printf("Parse failed: %s\n", req.URL.Query().Get("rd"))
+                       }
+               }
+       }
        if redirect == "" || !strings.HasPrefix(redirect, "/") || strings.HasPrefix(redirect, "//") {
                redirect = "/"
        }
+       fmt.Printf("Redirect is: %s\n", redirect)

        return
 }

[nmiculinic]

@ean

What exactly nginx does, and what it should do? Do you have an example?

[ean]

The nginx ingress controller requests auth using this statement:
error_page 401 = https://host/oauth2/start?rd=$scheme://$http_host$request_uri;
Where the $scheme://$http_host$request_uri should be url encoded, which would allow us to fetch the value using the standard request API in go instead of doing manual parsing of the url string.

[nmiculinic]

https://github.com/openresty/set-misc-nginx-module#set_escape_uri

Can we use something like this?

[nmiculinic]

Also I've tried your patched and it didn't help

[ean]

My ingress setup look like this:

---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    ingress.kubernetes.io/auth-signin: https://hostname.example.net/oauth2/start
    ingress.kubernetes.io/auth-url: https://hostname.example.net/oauth2/auth
    ingress.kubernetes.io/ssl-redirect: "true"
    kubernetes.io/ingress.class: nginx
    kubernetes.io/tls-acme: "true"
  labels:
    app: foo
  name: foo
spec:
  rules:
  - host: hostname.example.net
    http:
      paths:
      - path: /
        backend:
          serviceName: foo
          servicePort: 3000
  tls:
  - hosts:
    - hostname.example.net
    secretName: foo-tls
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    ingress.kubernetes.io/ssl-redirect: "true"
    kubernetes.io/ingress.class: nginx
    kubernetes.io/tls-acme: "true"
  labels:
    app: foo
  name: foo-auth
spec:
  rules:
  - host: hostname.example.net
    http:
      paths:
      - path: /oauth2
        backend:
          serviceName: auth-proxy-v1
          servicePort: 4180
  tls:
  - hosts:
    - hostname.example.net
    secretName: foo-tls

where auto-proxy-v1 is a service that exposes the oauth2-proxy instance. I will look at what is needed to get redirect to work if the oauth2-proxy is exposed on a separate external hostname to avoid having two ingresses for all exposed services. But that requires some more changes to the oauth2-proxy code it seems.

[nmiculinic]

EDIT: Deleted, since I misunderstood how cookies work. TIL

[chrisz100]

Having the same issue. When trying to login using github I end up in a loop to login. If I go to URL/oauth2/sign_in without the ?rd=URL nginx attaches the login works fine.

[KaustubhKhati]

I have a working setup.
A nginx is setup with a AWS Certificate.
The Oauth2 Deployment and Ingress for dashboard

Only thing is the nginx we are using is 0.9.0.

[theobolo]

@KaustubhKhati You'r DA MAN ! I'm on this since a long time now ... finally it's working by replacing nginx.ingress.kubernetes.io/auth-signin: https://hostname.example.net/oauth2/sign_in by nginx.ingress.kubernetes.io/auth-signin: https://hostname.example.net/oauth2/start, after that no more Loops at first login !

Tested on the last Nginx Ingress controller version 0.12.0.

[KaustubhKhati]

Anyone who might want to protect multiple domains with github and oauth2_proxy, nginx config file has to be modified. But since i didnt have time i wrote a small NodeJS app with a static file that will help around the problem. https://github.com/KaustubhKhati/oauth2_node_proxy

[CommanderJax]

Has anyone encountered an issue where the authentication works just fine(based on the oauth2_proxy logs), but the ingress controller still denies access with a 403? It doesn't make any sense, I feel like I have tried literally everything(including the configs provided by @KaustubhKhati) and the outcome is still the same.

[KaustubhKhati]

@AquaZ Can you post your Oauth2_proxy Deployment YAML. For me it was the cookie domain --cookie-domain flag that was causing the issue.

[CommanderJax]

@KaustubhKhati
I finally figured this out. In case anyone faces the same issue, in my case it was related to whitelisting a specific ip range on the nginx ingress controller. After removing the restriction, the issue disappeared.

[bhack]

Anybody tried to check if pass_access_token is working with this setup? I cannot see the X-Forwarded-Access-Toke entry in the header.

[f-ld]

@theobolo : setting nginx.ingress.kubernetes.io/auth-signin: https://$host/oauth2/start only works if you are accessing /.
Indeed, with that configuration, accessing /foobar will take you to / after signin.

If you want to go to /foobar after signin triggered trying to access /foobar then the right configuration is nginx.ingress.kubernetes.io/auth-signin: https://$host/oauth2/start?rd=$request_uri.

Indeed, if I summarize for everyone:

• nginx ingress does a 302 to the sign-in page and the header X-Auth-Request-Redirect is then not transmitted to the sign-in page
• the code of oauth_proxy v2.2 (see

  oauth2_proxy/oauthproxy.go

  Lines 367 to 373 in b90a234

  	redirect_url := req.URL.RequestURI()
  	if req.Header.Get("X-Auth-Request-Redirect") != "" {
  	redirect_url = req.Header.Get("X-Auth-Request-Redirect")
  	}
  	if redirect_url == p.SignInPath {
  	redirect_url = "/"
  	}

  ) says that if that header is not configured then we use the current request_uri for redirect url, so the sign-in page itself !!! Indeed, since nginx ingress adds rd parameter to the url, we will never have equality between sign-in page and request_uri
• at the end, we redirect to the redirect url which is set to the signin page that resets cookies -> loop

With nginx.ingress.kubernetes.io/auth-signin: https://$host/oauth2/start?rd=$request_uri, we skip that signin page and directly go to the redirection page (the one redirecting to google auth page) with the right redirection url set, i.e. the one requested by the user.

Hope that helps

[sneko]

Hi!

I noticed a few months ago that I was required to use Nginx v0.9.0-beta.11 to be compatible with oauth2_proxy v2.2 (image: a5huynh/oauth2_proxy:2.2).

None of the versions greater than this mentioned beta are working. It's written at the top of the Helm chart https://github.com/helm/charts/tree/master/stable/oauth2-proxy#oauth2-proxy .

I was wondering if you succeeded in using a more recent Nginx image... ? I just tried with 0.19.0 , 0.18.0 and v0.9.0-beta.12 but it still doesn't work. Maybe I need to use another OAuth2 proxy image?

Thank you,

[KaustubhKhati]

@sneko check a few post above 'theobolo' mentioned he was able to do it with 0.12.0. try building your own image or use https://hub.docker.com/r/colemickens/oauth2_proxy/