tlsPassClientCertificates() filter not working in kubernetes

[rickhlx]

Describe the bug
Client TLS certificates are not being passed through to the X-Forwarded-Tls-Client-Cert header when running Skipper for ingress in Kubernetes.

To Reproduce

1. Skipper CLI args:

skipper --kubernetes -address=:8443 -kubernetes-enable-tls

2. Ingress definition with TLS secret termination and tlsPassClientCertificates()

apiVersion: networking.k8s.io/v1                                                                                                       
kind: Ingress                                                                                                                          
metadata:                                                                                                                              
  annotations:                                                                                                                                                                                   
    zalando.org/skipper-filter: tlsPassClientCertificates() -> logHeader("request" , "response")                                                                                                                                                                                                                                                                                                                                                                                            
  name: foobar
  namespace: foobar
spec:                                                                                                                                  
  rules:                                                                                                                               
  - host: foobar.com
    http:                                                                                                                              
      paths:                                                                                                                           
      - backend:                                                                                                                       
          service:                                                                                                                     
            name: foobar
            port:                                                                                                                      
              name: web                                                                                                                
        path: /foo
        pathType: Prefix                                                                                                               
  tls:                                                                                                                                 
  - hosts:                                                                                                                             
    - foobar.com
    secretName: foo-tls
status:                                                                                                                                
  loadBalancer: {}   

3. Request

curl -X POST https://foobar.com/foo/bar--cert cert.crt --key cert.key

Expected behavior

The pod running the foobar application should receive the X-Forwarded-Tls-Client-Cert from Skipper and should be logged by skipper when using the logHeader() filter

Observed behavior

The pod running the application does not get the X-Forwarded-Tls-Client-Cert header with the encoded certificate.

Host: foobar.com
Accept: */*
Content-Length: 79
Content-Type: application/x-www-form-urlencoded
User-Agent: curl/7.81.0

[rickhlx]

We've been able to debug this issue and found that without the tls.Config.ClientAuth set to tls.RequestClientCert (instead of the default tls.NoClientCert setting), the client certificates are not loaded into the TLS context.

After we set tls.Config.ClientAuth to tls.RequestClientCert we are able to see the correctly forwarded header:

Host: foobar.com
Accept: */*
Content-Length: 79
Content-Type: application/x-www-form-urlencoded
X-Forwarded-Tls-Client-Cert: REDACTED

For reference we configured ClientAuth here in our tests:

skipper/skipper.go

Lines 1199 to 1201 in 1b4b204

	config := &tls.Config{
	MinVersion: o.TLSMinVersion,
	}

Should this be set by default or via a flag to enable client certificates to be passed through?

[webframp]

@szuecs this is another of our team with a slightly different environment. I mentioned it briefly in slack. Let us know how we can help debug/report. Happy to contribute fixes if we can

[AlexanderYastrebov]

Hello, thank you for investigating. I've created #3281 to add an option and a flag to configure TLS client authentication policy in backwards-compatible way.