Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ext_authz injected headers not propagating to HTTP router #12195

Closed
bruth opened this issue Jul 21, 2020 · 5 comments
Closed

ext_authz injected headers not propagating to HTTP router #12195

bruth opened this issue Jul 21, 2020 · 5 comments
Labels
area/ext_authz investigate Potential bug that needs verification stale stalebot believes this issue/PR has not been touched recently

Comments

@bruth
Copy link

bruth commented Jul 21, 2020 •
edited
Loading

Description:

What issue is being seen? Describe what should be happening instead of
the bug, for example: Envoy should not crash, the expected value isn't
returned, etc.

This was tested with v0.12.3 through v0.15.0 with both the v2 and v3 APIs.

I have an external authorization service configured which adds headers to the request on a successful check. This works correctly per the trace logs (see below). However when using header-based matching in HTTP routes, the header does not match, as if it is not present.

[2020-07-21 01:08:57.322][68][trace][filter] [source/extensions/filters/http/ext_authz/ext_authz.cc:156] [C3][S13608611734040639772] ext_authz filter added header(s) to the request:
[2020-07-21 01:08:57.322][68][trace][filter] [source/extensions/filters/http/ext_authz/ext_authz.cc:163] [C3][S13608611734040639772] 'x-principal-id':'abc123'
[2020-07-21 01:08:57.322][68][trace][filter] [source/extensions/filters/http/ext_authz/ext_authz.cc:163] [C3][S13608611734040639772] 'x-principal-type':'user'
[2020-07-21 01:08:57.322][68][trace][filter] [source/extensions/filters/http/ext_authz/ext_authz.cc:163] [C3][S13608611734040639772] 'x-principal-name':'John Doe'
[2020-07-21 01:08:57.322][68][trace][filter] [source/extensions/filters/http/ext_authz/ext_authz.cc:163] [C3][S13608611734040639772] 'x-principal-email':'jdoe@example.com'

Repro steps:

Include sample requests, environment, etc. All data and inputs
required to reproduce the bug.

Config:

Include the config used to configure Envoy.

admin:
  access_log_path: /tmp/admin_access.log
  address:
    socket_address:
      address: 0.0.0.0
      port_value: 9005

static_resources:
  # Listeners that will proxy traffic to upstream clusters.
  listeners:
    - address:
        socket_address:
          address: 0.0.0.0
          port_value: 9000

      filter_chains:
        - filters:
            - name: envoy.http_connection_manager
              typed_config:
                "@type": type.googleapis.com/envoy.config.filter.network.http_connection_manager.v2.HttpConnectionManager
                stat_prefix: ingress_http
                codec_type: AUTO

                upgrade_configs:
                  - upgrade_type: websocket

                http_filters:
                  # Auth check and inject x-principal-email header.
                  - name: envoy.filters.http.ext_authz
                    typed_config:
                      "@type": type.googleapis.com/envoy.config.filter.http.ext_authz.v2.ExtAuthz
                      grpc_service:
                        envoy_grpc:
                          cluster_name: auth

                  - name: envoy.filters.http.lua
                    typed_config:
                      "@type": type.googleapis.com/envoy.config.filter.http.lua.v2.Lua
                      inline_code: |
                        function envoy_on_request(handle)
                          for key, value in pairs(handle:headers()) do
                            handle:logTrace("key: "..key..", value: "..value)
                          end
                        end

                  - name: envoy.filters.http.router

                route_config:
                  virtual_hosts:
                    - name: test
                      domains: ["*"]
                      routes:
                        - match:
                            prefix: /
                            # This additional constraint will yield a 404 from envoy even though it is injected by the ext_authz filter.
                            headers:
                              - name: "x-principal-email"
                                safe_regex_match:
                                  google_re2: {}
                                  regex: "jdoe@example.com"

                          route:
                            cluster: workspace

  clusters:
    - name: auth
      connect_timeout: "0.25s"
      type: LOGICAL_DNS
      http2_protocol_options: {}
      load_assignment:
        cluster_name: auth
        endpoints:
          - lb_endpoints:
            - endpoint:
                address:
                  socket_address:
                    address: auth
                    port_value: 8000

    - name: workspace
      connect_timeout: "1s"
      type: STRICT_DNS
      load_assignment:
        cluster_name: workspace
        endpoints:
          - lb_endpoints:
            - endpoint:
                address:
                  socket_address:
                    address: 127.0.0.1
                    port_value: 3000

Logs:

Include the access logs and the Envoy logs.

I added the Lua filter just to confirm the headers are propagated to the next filter.

[2020-07-21 01:08:57.322][68][trace][http] [source/common/http/conn_manager_impl.cc:2047] [C3][S13608611734040639772] continuing filter chain: filter=0x55cab27fc960
[2020-07-21 01:08:57.322][68][trace][lua] [bazel-out/k8-opt/bin/source/extensions/filters/common/lua/_virtual_includes/lua_lib/extensions/filters/common/lua/lua.h:122] creating N5Envoy10Extensions11HttpFilters3Lua19StreamHandleWrapperE at 0x400ce7c0
[2020-07-21 01:08:57.322][68][trace][lua] [bazel-out/k8-opt/bin/source/extensions/filters/common/lua/_virtual_includes/lua_lib/extensions/filters/common/lua/lua.h:122] creating N5Envoy10Extensions11HttpFilters3Lua16HeaderMapWrapperE at 0x400cd210
[2020-07-21 01:08:57.322][68][trace][lua] [bazel-out/k8-opt/bin/source/extensions/filters/common/lua/_virtual_includes/lua_lib/extensions/filters/common/lua/lua.h:122] creating N5Envoy10Extensions11HttpFilters3Lua17HeaderMapIteratorE at 0x400cd2a8
[2020-07-21 01:08:57.322][68][trace][lua] [source/extensions/filters/http/lua/lua_filter.cc:673] script log: key: :authority, value: foo.example.com 
[2020-07-21 01:08:57.322][68][trace][lua] [source/extensions/filters/http/lua/lua_filter.cc:673] script log: key: :path, value: /bar/
[2020-07-21 01:08:57.322][68][trace][lua] [source/extensions/filters/http/lua/lua_filter.cc:673] script log: key: :method, value: GET
[2020-07-21 01:08:57.322][68][trace][lua] [source/extensions/filters/http/lua/lua_filter.cc:673] script log: key: cookie, value: foo=bar
[2020-07-21 01:08:57.322][68][trace][lua] [source/extensions/filters/http/lua/lua_filter.cc:673] script log: key: if-none-match, value: W/"30682294-1991-2020-07-20T21:28:30.030Z"
[2020-07-21 01:08:57.322][68][trace][lua] [source/extensions/filters/http/lua/lua_filter.cc:673] script log: key: accept, value: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
[2020-07-21 01:08:57.322][68][trace][lua] [source/extensions/filters/http/lua/lua_filter.cc:673] script log: key: if-modified-since, value: Mon, 20 Jul 2020 21:28:30 GMT
[2020-07-21 01:08:57.322][68][trace][lua] [source/extensions/filters/http/lua/lua_filter.cc:673] script log: key: user-agent, value: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.1.1 Safari/605.1.15
[2020-07-21 01:08:57.322][68][trace][lua] [source/extensions/filters/http/lua/lua_filter.cc:673] script log: key: accept-language, value: en-us
[2020-07-21 01:08:57.322][68][trace][lua] [source/extensions/filters/http/lua/lua_filter.cc:673] script log: key: accept-encoding, value: gzip, deflate, br
[2020-07-21 01:08:57.322][68][trace][lua] [source/extensions/filters/http/lua/lua_filter.cc:673] script log: key: x-forwarded-proto, value: https
[2020-07-21 01:08:57.322][68][trace][lua] [source/extensions/filters/http/lua/lua_filter.cc:673] script log: key: x-request-id, value: bec93cd6-adb1-46d0-8820-5d08b9b7f521
[2020-07-21 01:08:57.322][68][trace][lua] [source/extensions/filters/http/lua/lua_filter.cc:673] script log: key: content-length, value: 0
[2020-07-21 01:08:57.322][68][trace][lua] [source/extensions/filters/http/lua/lua_filter.cc:673] script log: key: x-principal-id, value: abc123
[2020-07-21 01:08:57.322][68][trace][lua] [source/extensions/filters/http/lua/lua_filter.cc:673] script log: key: x-principal-type, value: user
[2020-07-21 01:08:57.322][68][trace][lua] [source/extensions/filters/http/lua/lua_filter.cc:673] script log: key: x-principal-name, value: John Doe
[2020-07-21 01:08:57.322][68][trace][lua] [source/extensions/filters/http/lua/lua_filter.cc:673] script log: key: x-principal-email, value: jdoe@example.com
@zuercher zuercher added area/ext_authz investigate Potential bug that needs verification labels Jul 21, 2020
@zuercher
Copy link
Member

Envoy chooses and caches the route before executing http filters. There is a mechanism where an HTTP filter can cause the route to be recomputed (at which point I believe it should see the additional headers and select the route you expect). The tradeoff is the performance hit of recomputing the routes. I think the trick will be deciding how to modify the ext_authz filter to trigger route recomputation -- it probably shouldn't do it blindly.

@bruth
Copy link
Author

bruth commented Jul 21, 2020

@zuercher Thanks for the information. I do see there is clear_route_cache as an option. Is this what you are referring to?

Any idea of the magnitude of the performance hit? Are we talking microseconds per request or milliseconds?

@bruth
Copy link
Author

bruth commented Jul 21, 2020

That clear_route_cache option appeared to do the trick for routes that have an upstream.

However, a related question, in my real implementation I have an RBAC filter after the authorization filter. I am now noticing that HTTP routes that has a redirect appear to be blocked by the RBAC filter, while a route that has the route config to a cluster appears to work fine. Is this at all related or a different problem?

For example:

- match:    # fails due to RBAC
    path: /foo
  redirect:
    path_redirect: /foo/
 
- match:    # fine with RBAC
    prefix: /foo/
  route:
    cluster: foo
    timeout: 0s

@stale
Copy link

stale bot commented Aug 22, 2020

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or other activity occurs. Thank you for your contributions.

@stale stale bot added the stale stalebot believes this issue/PR has not been touched recently label Aug 22, 2020
@stale
Copy link

stale bot commented Aug 30, 2020

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted". Thank you for your contributions.

@stale stale bot closed this as completed Aug 30, 2020
@VirajSalaka VirajSalaka mentioned this issue Dec 8, 2020
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/ext_authz investigate Potential bug that needs verification stale stalebot believes this issue/PR has not been touched recently
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@zuercher @bruth