support HTTP/1.1 CONNECT for proxied upstreams #3664

bdecoste · 2020-09-24T22:40:44Z

Is your feature request related to a problem? Please describe.
I am unable to connect from envoy to an internet upstream through corporate proxy

Describe the solution you'd like
support HTTP/1.1 CONNECT for proxied upstreams

envoyproxy/envoy#11308

kdorosh · 2020-12-01T21:43:59Z

upstream envoy issue, not a gloo one

kdorosh · 2021-02-19T23:56:14Z

documenting config for HTTP request -> envoy (encapsulate in HTTP Connect) -> HTTP Connect-enabled proxy -> upstream.

this is useful, for example, if the HTTP Connect enabled proxy is in a DMZ with internet access, but envoy and your other services are not on the internet.

put the following envoy config on clipboard:

admin:
  access_log_path: /tmp/admin_access.log
  address:
    socket_address:
      protocol: TCP
      address: 0.0.0.0
      port_value: 9903
static_resources:
  listeners:
    - name: listener_0
      address:
        # use an in memory pipe here instead, see example in following comment below
        socket_address:
          protocol: TCP
          address: 0.0.0.0
          port_value: 10000
      filter_chains:
        - filters:
            - name: tcp
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
                stat_prefix: tcp_stats
                cluster: "cluster_0"
                tunneling_config:
                  hostname: host.com:443
    - name: listener_1
      address:
        socket_address:
          address: 0.0.0.0
          port_value: 9999
      filter_chains:
        - filters:
            - name: envoy.filters.network.http_connection_manager
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                codec_type: AUTO
                stat_prefix: prefix
                http_filters:
                  - name: envoy.filters.http.router
                route_config:
                  name: demo_route
                  virtual_hosts:
                    - name: demo_host
                      domains:
                        - "*"
                      routes:
                        - match:
                            prefix: "/"
#                            headers:
#                              - name: ":method"
#                                exact_match: GET
                          route:
                            cluster: tcp_self_cluster
  clusters:
    - name: cluster_0
      type: LOGICAL_DNS
      connect_timeout: 5s
      # This ensures HTTP/1.1 CONNECT is used for establishing the tunnel.
      typed_extension_protocol_options:
        envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
          "@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
          explicit_http_config:
            http_protocol_options: {}
      load_assignment:
        cluster_name: cluster_0
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      # in this example, this is the address of the proxy in DMZ (supports HTTP Connect)
                      address: host.docker.internal
                      port_value: 9876
    - name: tcp_self_cluster
      connect_timeout: 5.000s
      load_assignment:
        cluster_name: tcp_self_cluster
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: 127.0.0.1
                      port_value: 10000
      upstream_connection_options:
        tcp_keepalive: {}
      type: STATIC

and run envoy:

docker run -p 10000:10000 -p 9903:9903 -p 9999:9999 --rm envoyproxy/envoy-dev:b77157803df9a1e6dff53cc616b32ddbf79f83f2 --config-yaml "$(pbpaste)" -l trace

Then start a fake "echo server" as the upstream:

(printf "HTTP/1.1 200 OK\n\n";cat) | nc -l 9876

And make an HTTP request:

curl -v 127.0.0.1:9999

The flow here is:

HTTP request to envoy at port 9999 -> envoy forwards to itself at port 10000 (so TCP filter can match for HTTP Connect) -> envoy treats the HTTP request as TCP, sends everything after the headers as raw TCP in new encapsulated HTTP Connect request to upstream cluster (e.g., proxy in a DMZ)

Reopening because we need to expose some envoy config to support this flow

edit: upon further review, the second cluster and listener aren't necessary for POC. curl -v 127.0.0.1:10000 would have worked, although the "loopback" is still necessary to get matching for this tunneling config at the route level.

kdorosh · 2021-02-23T21:40:01Z

If you want to test using SSL:

admin:
  access_log_path: /tmp/admin_access.log
  address:
    socket_address:
      protocol: TCP
      address: 0.0.0.0
      port_value: 9903
static_resources:
  listeners:
    - name: listener_0
      address:
        pipe:
          path: "@/cluster_0"
      filter_chains:
        - filters:
            - name: tcp
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
                stat_prefix: tcp_stats
                cluster: "cluster_0"
                tunneling_config:
                  hostname: host.com:443
    - name: listener_1
      address:
        socket_address:
          address: 0.0.0.0
          port_value: 9999
      filter_chains:
        - filters:
            - name: envoy.filters.network.http_connection_manager
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                codec_type: AUTO
                stat_prefix: prefix
                http_filters:
                  - name: envoy.filters.http.router
                route_config:
                  name: demo_route
                  virtual_hosts:
                    - name: demo_host
                      domains:
                        - "*"
                      routes:
                        - match:
                            prefix: "/"
                          route:
                            cluster: tcp_self_cluster
  clusters:
    - name: cluster_0
      type: LOGICAL_DNS
      connect_timeout: 5s
      # This ensures HTTP/1.1 CONNECT is used for establishing the tunnel.
      typed_extension_protocol_options:
        envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
          "@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
          explicit_http_config:
            http_protocol_options: {}
      transport_socket:
        name: envoy.transport_sockets.tls
        typed_config:
          '@type': type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
          common_tls_context:
            tls_certificates:
              - certificate_chain:
                  # base64 encoded
                  inline_bytes: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUNsakNDQVg0Q0NRQ2ZPMXBKcXdQLzZEQU5CZ2txaGtpRzl3MEJBUXNGQURBTk1Rc3dDUVlEVlFRR0V3SlYKVXpBZUZ3MHlNVEF5TWpNeU1URTNNVEJhRncweU1UQXpNalV5TVRFM01UQmFNQTB4Q3pBSkJnTlZCQVlUQWxWVApNSUlCSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQXIxYUY2TWJwb2ZDOWVyODBHazloClJ5RlYyQjBEM1RkOVJ1VUYxYTFYNFhCVGdGYUxvN0RQbFlDSFVBSWRvUmRpSlVqeDZ2by8vY1h4TmcxRDRGMy8KY1owazFpTVROL0pvQm9QVitpa3RCMGJCUURFSGVWV2p4a0J4S1pFSGE0aEgrUyt2alhmSXRBSEJrY2tTcFZQcApBRmJzU0kvbi9XREZBbTlvNUlBZXR5eTJKdEhXbVlKa0IrSU90dFNtN2RZazRnMmVxS0lVSWh3dE5acTRZeU9SCkoyQ2g2aXV4VEtmb3Y4OEF5SDJxZEd5TkVlVlhVNndtNWdhek1XUXo4SmsvMExyYW1PVlVldGcxTEVBLys1VVEKZXJqU2NNWFhPQW5OaGpPdjBqdEJhWUhod25FVlg4R2NwWGFZc1ZETElhSWphMDVrODFqazFOcWtUbTFLc0Z2dQpUUUlEQVFBQk1BMEdDU3FHU0liM0RRRUJDd1VBQTRJQkFRQXVNSFEyVTE5SlA3d1d1VFdYS0NjenRtb1JWT2dDCmxLMlJQZzJYSWNPQjRMWXdkV2ExR3dSQkhLYVFDNWh5ME0rSEhhRTVSMVI5SURPaFJzblZTWGUzZTJucjZFRFIKRmUwdEtYWE5yL2pzNXhFR0xHV2xFYmZ6SFVEQWxtRU9SVXhuUFlFMU52eHBVVVVCMFg4c04zMTlPaG1uRDZkTApMZ25Pcy9LZXpJZFgwV2lrakZmR1c2d0lYZDBORmNmSjI0YjVJbHdDdHVSUW81cG5UVS9FWWM4SXZpaFdhL2VjCjBuM3R1WVlwMjBNUGxhY0JxQ2pQbU1QM1VnaEY2ZE1UMnFrRGtVcFkyc05Nc0JMVkZyYXh0L2lBMlhkbUlqdVMKMU8xRVBlZWludDlaQUNUcmVaQ2Rzblk3b0JJUzdwaWkva210U1BSV2lSMlNqWlRiMXZlcHFyMmYKLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
                private_key:
                  # base64 encoded
                  inline_bytes: LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tCk1JSUV2QUlCQURBTkJna3Foa2lHOXcwQkFRRUZBQVNDQktZd2dnU2lBZ0VBQW9JQkFRQ3ZWb1hveHVtaDhMMTYKdnpRYVQyRkhJVlhZSFFQZE4zMUc1UVhWclZmaGNGT0FWb3Vqc00rVmdJZFFBaDJoRjJJbFNQSHErai85eGZFMgpEVVBnWGY5eG5TVFdJeE0zOG1nR2c5WDZLUzBIUnNGQU1RZDVWYVBHUUhFcGtRZHJpRWY1TDYrTmQ4aTBBY0dSCnlSS2xVK2tBVnV4SWorZjlZTVVDYjJqa2dCNjNMTFltMGRhWmdtUUg0ZzYyMUtidDFpVGlEWjZvb2hRaUhDMDEKbXJoakk1RW5ZS0hxSzdGTXAraS96d0RJZmFwMGJJMFI1VmRUckNibUJyTXhaRFB3bVQvUXV0cVk1VlI2MkRVcwpRRC83bFJCNnVOSnd4ZGM0Q2MyR002L1NPMEZwZ2VIQ2NSVmZ3WnlsZHBpeFVNc2hvaU5yVG1UeldPVFUycVJPCmJVcXdXKzVOQWdNQkFBRUNnZ0VBR3FSRStKZXd6RUJ1S2daL2ZqZnRpZ2RRcWVQT3dJQTc4dGU5eVRmcmtIK20KMEd0V0VHcy9QM0ltSFV3VkFITXpJSU15N3dkVFRkWVR6cWk1VlpBeXppcVlmRWtjRkhZcXhFS244cmpxOUo0VgpveEZRRjJtaDF1L1h3ZjhkWjROdm5EZFlyMkF5Q05memQ1bGIwZThzQ2pjcTgrT1oxbkFydnpENmVTM1dmUEozCjFWQkVadUVNUCtsWnZqMmdkQWJWVTkrTURzS2hRUm44dS9kcnByby80NG5FbmFSNkpaQmNiRkZtYmJtOVNKODIKSjRHTy82YXllWCs0UnYrOHducTloQVpDY2tIR3JFWkNBUGxJUE9aY1pTMkRZYUxsZTRBeElFSS9CRjh6dkx0OApJTFRTY3cwZUFTc0J3TkMveEJkWUk4S0JjSklWWDk1Z2E3cHJIYlJ3NlFLQmdRRGIxZ3cyUWdVeFNhckV1YzNKCjlxUnpFRk90SURWY2d2alJ3NE9aVHNTZTlKdnc1YmFVRDRMVzFIODNMT2g3T01PZmlvQ3ZUUGJSUm1IRHRvWEUKMXp6dnRXdnBaWUtLUXVLQlRzcm5VSmtLQWZoS0x0QkVTOGFHVEhIaGQ4S3MvazlXMGcvNHY3M3ZuQUowelBHegp6VU9mQWJmN0JHRnMzWVdscERERi9GOUU1d0tCZ1FETUxvYkFmNmkzNytUZ3pVY1FWc3R4NUJ4YStxYzZia0xUCjkwRHVzWUFsR3dYaUNDN0lXWjIrV2M4Z0tjdUJWY1Exc3N1dVVidGwrQVgyam8rUmNIalpVUERqN0JEMmx0bUUKL2xUZkt0Y3RuRy9semh5c05aY2ZIQTRGNnd3SnMwQk0wdGVZaW01SkU3WDYrRFNBOTcyVVVyRGN0N1R3RXpTeQp1STlUNS9yWXF3S0JnQkxmS3lrMVBiRGZmQk5yUitsQWpSbDdtMU5QWW9WUWtQK2c0eGswRTlqNWNyR1paclVMCko5WTlZN1plMm81SmJ6R3crQjhVWWxQUTBPKzJLWUdlWUpMTFpqb2hCclhMekJvQm81Vjc2OGRhSi9CbVdHYWsKZ3FSZ3g1M3RZLzdwemZITUVxYnQvdVlrNUdrV2R1RlNpNjJJdmp5RVFjRFF1M3Bldi9rZkV5YWRBb0dBYjhhRQpwRHIrZlB0SjI3cmt6VkFHZmFtMzBDSnBpN1l5eFdFVTNkYXo1UncvQXpjaGpUSnFCWHVheE91OGFXZ3ZTb28vCnNJVXp4cVdaZGhrakJTYUNNYzE2b2Z0b2FlVzQzSEp5dWR6MUFSWWp0N1Uyd3h0dlNHdU9UYlVYMlBPVkhybEoKcDBzWkJzNVFrRHh5V2F2Rno3M1k3bXR2RlFJZWxqd3MwbTByU0pNQ2dZQitMeUM1andrMElNNEpaMXZ2SVFLMApROGExWFNQUUhXVytSNDZpL295QzRFdEN6QmE0M0d2dTh5Q2F3ZFozam5rS2Fuc1hOTko1ZlRDdlhsak1aV0p6CjdEODQvNVZxMWJMTXdLY1ZpamRScFVrMWh3MEdHTFVUZy9sUy9STmJ5Vk16Wm1yaUVoY1NQaGR5a292ZHhmSWQKSE9ub2RhdmthSmw0YmtnSWxyTk1Fdz09Ci0tLS0tRU5EIFBSSVZBVEUgS0VZLS0tLS0K
      load_assignment:
        cluster_name: cluster_0
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      # in this example, this is the address of the proxy in DMZ (supports HTTP Connect)
                      address: host.docker.internal
                      port_value: 9876
    - name: tcp_self_cluster
      connect_timeout: 5.000s
      load_assignment:
        cluster_name: tcp_self_cluster
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    pipe:
                      path: "@/cluster_0"
      upstream_connection_options:
        tcp_keepalive: {}
      type: STATIC

Write this as tls.key file:

Write TLS this as tls.crt file:

Then set up TLS listener

(printf "HTTP/1.1 200 OK\n\n";cat) | ncat -l 9876 --ssl-cert test.crt --ssl-key test.key

And make requests:

curl -v 127.0.0.1:9999

kdorosh · 2021-02-24T18:39:27Z

In the customer env, the TLS context must go on tcp_self_cluster instead of cluster_0 for this to work

gsustek · 2022-01-21T14:59:40Z

for me above envoy config did not work, but following one does:

static_resources:
listeners:
- name: listener_0
address:
pipe:
path: "@/cluster_0"
filter_chains:
- filters:
- name: tcp
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
stat_prefix: tcp_stats
cluster: "cluster_0"
tunneling_config:
hostname: jwt.io:443
headers_to_add:
- header:
key: Proxy-Authorization
value: Basic dXNlcjpwYXNzd29yZA== # replace with your credentials
- header:
key: Proxy-Connection
value: Keep-Alive
append: false
- name: listener_1
address:
socket_address:
address: 0.0.0.0
port_value: 10000
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
codec_type: AUTO
stat_prefix: prefix
http_filters:
- name: envoy.filters.http.router
route_config:
name: demo_route
virtual_hosts:
- name: demo_host
domains:
- "*"
routes:
- match:
prefix: "/"
route:
host_rewrite_literal: jwt.io
cluster: tcp_self_cluster
clusters:
- name: cluster_0
type: LOGICAL_DNS
connect_timeout: 5s
# This ensures HTTP/1.1 CONNECT is used for establishing the tunnel.
typed_extension_protocol_options:
envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
"@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
explicit_http_config:
http_protocol_options: {}
load_assignment:
cluster_name: cluster_0
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
# in this example, this is the address of the proxy in DMZ (supports HTTP Connect)
address: 192.168.168.211
port_value: 8080
- name: tcp_self_cluster
connect_timeout: 5.000s
load_assignment:
cluster_name: tcp_self_cluster
endpoints:
- lb_endpoints:
- endpoint:
address:
pipe:
path: "@/cluster_0"
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
sni: jwt.io

      curl -v  127.0.0.1:10000

scrocquesel · 2022-01-21T17:52:03Z

for me above envoy config did not work, but following one:

static_resources: listeners: - name: listener_0 address: pipe: path: "@/cluster_0" filter_chains: - filters: - name: tcp typed_config: "@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy stat_prefix: tcp_stats cluster: "cluster_0" tunneling_config: hostname: jwt.io:443 headers_to_add: - header: key: Proxy-Authorization value: Basic dXNlcjpwYXNzd29yZA== # replace with your credentials append: false - name: listener_1 address: socket_address: address: 0.0.0.0 port_value: 10000 filter_chains: - filters: - name: envoy.filters.network.http_connection_manager typed_config: "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager codec_type: AUTO stat_prefix: prefix http_filters: - name: envoy.filters.http.router route_config: name: demo_route virtual_hosts: - name: demo_host domains: - "*" routes: - match: prefix: "/" route: cluster: tcp_self_cluster clusters: - name: cluster_0 type: LOGICAL_DNS connect_timeout: 5s # This ensures HTTP/1.1 CONNECT is used for establishing the tunnel. typed_extension_protocol_options: envoy.extensions.upstreams.http.v3.HttpProtocolOptions: "@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions explicit_http_config: http_protocol_options: {} load_assignment: cluster_name: cluster_0 endpoints: - lb_endpoints: - endpoint: address: socket_address: # in this example, this is the address of the proxy in DMZ (supports HTTP Connect) address: 192.168.168.211 port_value: 8080 - name: tcp_self_cluster connect_timeout: 5.000s load_assignment: cluster_name: tcp_self_cluster endpoints: - lb_endpoints: - endpoint: address: pipe: path: "@/cluster_0" transport_socket: name: envoy.transport_sockets.tls typed_config: "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext sni: jwt.io
      curl -v --header 'Host: jwt.io' 127.0.0.1:10000

For improved perf, you may use a pipe loopback see https://gist.github.com/cortex93/6d7cd9738b077d0447ab891cbacd0d5e

gsustek · 2022-01-24T23:30:38Z

i adapted yaml once more time

this configuration supports only reaching https url's via forward proxy.
"curl --insecure -v -x http://user:password@localhost:8080 -L https://jwt.io "
looking in tcpdump we can see there is two http request:

curl is using HTTP CONNECT VERB to connect jwt.io:443 via FW Proxy service by making Tunnel:
"CONNECT jwt.io:443 HTTP/1.1
Host: jwt.io:443
Proxy-Authorization: Basic dXNlcjpwYXNzd29yZA==
User-Agent: curl/7.77.0
Proxy-Connection: Keep-Alive

HTTP/1.1 200 Connection established"

second:
TLS tunnel inside proxy-protocol TLS Client Hello
third: HTTP get jwt.io

so in gloo case
0. curl to locally exposed virtual service. curl -v localhost:10000

HTTP/1.1 CONNECT is used for establishing the tunnel with proxy-auth and proxy-conn headers
create TLS connection and rewrite hostname
connect to jwt.io:443 via FW proxy

i assume that this structure would fit gloo virtualservice / upstream via FW proxy paradigm

bdecoste added the Type: Enhancement New feature or request label Sep 24, 2020

kdorosh closed this as completed Dec 1, 2020

kdorosh reopened this Feb 19, 2021

kdorosh mentioned this issue Feb 24, 2021

Remove some unused proto imports #4260

Closed

8 tasks

solo-changelog-bot bot mentioned this issue Feb 26, 2021

Http connect proxy #4358

Merged

8 tasks

soloio-bulldozer bot closed this as completed in #4358 Mar 2, 2021

kdorosh mentioned this issue Mar 2, 2021

Http connect proxy 1.6 #4377

Merged

8 tasks

pwoelfle mentioned this issue Feb 1, 2022

Support Proxy-Authorization HTTP Header on HTTP Connect Proxy #5859

Closed

bdecoste mentioned this issue Jul 15, 2022

When upstream httpProxyHostname is set then static.useTls and sslConfig: {} only apply to httpProxyHostname #6720

Closed

github-actions bot mentioned this issue Nov 13, 2024

[Migrated] support HTTP/1.1 CONNECT for proxied upstreams solo-io/gloo#3664

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

support HTTP/1.1 CONNECT for proxied upstreams #3664

support HTTP/1.1 CONNECT for proxied upstreams #3664

bdecoste commented Sep 24, 2020

kdorosh commented Dec 1, 2020

kdorosh commented Feb 19, 2021 •

edited

Loading

kdorosh commented Feb 23, 2021 •

edited

Loading

kdorosh commented Feb 24, 2021 •

edited

Loading

gsustek commented Jan 21, 2022 •

edited

Loading

scrocquesel commented Jan 21, 2022

gsustek commented Jan 24, 2022 •

edited

Loading

support HTTP/1.1 CONNECT for proxied upstreams #3664

support HTTP/1.1 CONNECT for proxied upstreams #3664

Comments

bdecoste commented Sep 24, 2020

kdorosh commented Dec 1, 2020

kdorosh commented Feb 19, 2021 • edited Loading

kdorosh commented Feb 23, 2021 • edited Loading

kdorosh commented Feb 24, 2021 • edited Loading

gsustek commented Jan 21, 2022 • edited Loading

scrocquesel commented Jan 21, 2022

gsustek commented Jan 24, 2022 • edited Loading

kdorosh commented Feb 19, 2021 •

edited

Loading

kdorosh commented Feb 23, 2021 •

edited

Loading

kdorosh commented Feb 24, 2021 •

edited

Loading

gsustek commented Jan 21, 2022 •

edited

Loading

gsustek commented Jan 24, 2022 •

edited

Loading