PROXY Protocol transformed to Forwarded HTTP header

doc/admin-guide/configuration/index.en.rst

@@ -32,3 +32,4 @@ Proxy Cache Configuration
    transparent-proxy.en
    transparent-forward-proxying.en
    hierachical-caching.en
+   proxy-protocol.en

doc/admin-guide/configuration/proxy-protocol.en.rst

@@ -0,0 +1,100 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+.. include:: ../../common.defs
+
+.. _proxy-protocol:
+
+Proxy Protocol
+****************
+
+The `PROXY protocol <https://www.haproxy.com/blog/haproxy/proxy-protocol/>`_
+provides a means of passing connection information between layers of the proxy
+infrastructure.  Without the PROXY protocol, |TS| would only have connection
+information from the previous hop connecting to |TS| and not the actual
+originating client connection information.  This can be done over either HTTP or
+TLS connections.
+
+.. note::
+
+    The current version only supports transforming client IP from PROXY Version 1 
+    header to the Forwarded: header.
+
+In the current implementation, the client IP address in the PROXY protocol header
+is passed to the origin server via an HTTP `Forwarded:
+<https://tools.ietf.org/html/rfc7239>`_ header.
+
+The Proxy Protocol must be enabled on each port.  See
+:ts:cv:`proxy.config.http.server_ports` for information on how to enable the
+Proxy Protocol on a port.  Once enabled, all incoming requests must be prefaced
+with the PROXY v1 header.  Any request not preface by this header will be
+dropped.
+
+As a security measure, an optional whitelist of trusted IP addresses may be
+configured with :ts:cv:`proxy.config.http.proxy_protocol_whitelist`.
+
+   .. important::
+
+       If the whitelist is configured, requests will only be accepted from these
+       IP addressses and must be prefaced with the PROXY v1 header.
+
+See :ts:cv:`proxy.config.http.insert_forwarded` for configuration information.
+Detection of the PROXY protocol header is automatic.  If the PROXY header
+precludes the request, it will automatically be parse and made available to the
+Forwarded: request header sent to the origin server.
+
+Example
+-------
+
+As an example, consider the following topology:
+
+.. figure:: ../../static/images/admin/proxy-protocol.png
+   :align: center
+   :alt: PROXY protocol transformed into a Forwarded: header
+
+   PROXY protocol header flow
+
+Without the PROXY protocol header, the client IP would only be reported
+accurately to the Load Balancer.  |TS| would only see the connection from the
+Load Balancer.  Similarly, the Web Server would only see the connection from
+|TS|.  In the example above, if the client initiated a TLS connection, the Web
+Server would see the connection originating from |TS| at ``10.0.0.2``:
+
+.. code-block:: lua
+
+    Forwarded: for=10.0.0.2;by=10.0.0.1;proto=https;host=test000001.com
+
+If the Load Balancer has the Proxy Protocol enabled, requests sent through the
+Load Balancer will be preceded with the PROXY header.  |TS| will detect the
+PROXY header and transform that into the Forwarded: HTTP header if configured to
+insert the Forwarded: header with the ``for`` paramter.  In the example above,
+if the client initiated a TLS connection, the Web Server can use the Forwarded:
+header to determine the TLS connection originated from the client at ``192.168.1.100``:
+
+.. code-block:: lua
+
+    Forwarded: for=192.168.2.100;by=10.0.0.2;proto=https;host=test000001.com
+
+
+References
+==========
+
+-  `The PROXY protocol Versions 1 & 2
+   <https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt>`_
+
+-  `Forwarded HTTP Extension
+   <https://tools.ietf.org/html/rfc7239#page-6>`_

doc/admin-guide/files/records.config.en.rst

@@ -608,6 +608,7 @@ HTTP Engine
    ip-out      Value           Local outbound IP address.
    ip-resolve  Value           IP address resolution style.
    proto       Value           List of supported session protocols.
+   pp                          Enable Proxy Protocol.
    ssl                         SSL terminated.
    tr-full                     Fully transparent (inbound and outbound)
    tr-in                       Inbound transparent.
@@ -643,6 +644,12 @@ proto
    all available protocols. For non-TLS proxy ports the default is HTTP
    only.
 
+pp
+   Enables Proxy Protocol on the port.  If Proxy Protocol is enabled on the
+   port, all incoming requests must be prefaced with the PROXY header.  See
+   :ref:`Proxy Protocol <proxy-protocol>` for more details on how to configure
+   this option properly.
+
 tr-full
    Fully transparent. This is a convenience option and is identical to specifying both ``tr-in`` and ``tr-out``.
 
@@ -1600,6 +1607,30 @@ Proxy User Variables
    is prohibited by RFC 7239. Currently, for the ``host`` parameter to provide the original host from the
    incoming client request, `proxy.config.url_remap.pristine_host_hdr`_ must be enabled.
 
+.. ts:cv:: CONFIG proxy.config.http.proxy_protocol_whitelist STRING ```<ip list>```
+
+   This defines a whitelist of server IPs that are trusted to provide
+   connections with Proxy Protocol information.  This is a comma delimited list
+   of IP addresses.  Addressed may be listed individually, in a range separated
+   by a dash or by using CIDR notation.
+
+   ======================= ===========================================================
+   Example  Effect
+   ======================= ===========================================================
+   ``10.0.2.123``          A single IP Address.
+   ``10.0.3.1-10.0.3.254`` A range of IP address.
+   ``10.0.4.0/24``         A range of IP address specified by CIDR notation.
+   ======================= ============================================================
+
+   .. important::
+
+       If Proxy Protocol is enabled on the port, but this directive is not
+       defined any server may initiate a connection with Proxy Protocol
+       information.
+       See :ts:cv:`proxy.config.http.server_ports` for information on how to enable Proxy Protocol on a port.
+
+   See :ref:`proxy-protocol` for more discussion on how |TS| tranforms the `Forwarded: header.
+
 .. ts:cv:: CONFIG proxy.config.http.normalize_ae INT 1
    :reloadable:
    :overridable:
@@ -2501,7 +2532,7 @@ HostDB
    Set the file path for an external host file.
 
    If this is set (non-empty) then the file is presumed to be a hosts file in
-   the standard `host file format <http://tools.ietf.org/html/rfc1123#page-13>`_.
+   the standard .
    It is read and the entries there added to the HostDB. The file is
    periodically checked for a more recent modification date in which case it is
    reloaded. The interval is set with :ts:cv:`proxy.config.hostdb.host_file.interval`.

iocore/net/Connection.cc

@@ -241,6 +241,10 @@ Server::setup_fd_for_listen(bool non_blocking, const NetProcessor::AcceptOptions
 #endif
   }
 
+  if (opt.f_proxy_protocol) {
+    Debug("proxyprotocol", "Proxy Protocol enabled.");
+  }
+
 #if defined(TCP_MAXSEG)
   if (NetProcessor::accept_mss > 0) {
     if ((res = safe_setsockopt(fd, IPPROTO_TCP, TCP_MAXSEG, (char *)&NetProcessor::accept_mss, sizeof(int))) < 0) {

iocore/net/I_NetProcessor.h

@@ -24,6 +24,7 @@
 
 #pragma once
 
+#include "ts/IpMap.h"
 #include "I_EventSystem.h"
 #include "I_Socks.h"
 struct socks_conf_struct;
@@ -96,6 +97,9 @@ class NetProcessor : public Processor
     */
     bool f_inbound_transparent;
 
+    /// Proxy Protocol enabled
+    bool f_proxy_protocol;
+
     /// Default constructor.
     /// Instance is constructed with default values.
     AcceptOptions() { this->reset(); }

iocore/net/I_NetVConnection.h

@@ -33,6 +33,8 @@
 #include "I_Socks.h"
 #include <ts/apidefs.h>
 #include <string_view>
+#include "ts/TextView.h"
+#include "ts/IpMap.h"
 
 #define CONNECT_SUCCESS 1
 #define CONNECT_FAILURE 0
@@ -129,10 +131,12 @@ struct NetVCOptions {
       @see ip_family
   */
   IpAddr local_ip;
+
   /** Local port for connection.
       Set to 0 for "don't care" (default).
-   */
+  */
   uint16_t local_port;
+
   /// How to bind the local address.
   /// @note Default is @c ANY_ADDR.
   addr_bind_style addr_binding;
@@ -601,6 +605,9 @@ class NetVConnection : public AnnotatedVConnection
   /** Set remote sock addr struct. */
   virtual void set_remote_addr() = 0;
 
+  /** Set remote sock addr struct. */
+  virtual void set_remote_addr(const sockaddr *) = 0;
+
   // for InkAPI
   bool
   get_is_internal_request() const
@@ -627,6 +634,19 @@ class NetVConnection : public AnnotatedVConnection
     is_transparent = state;
   }
 
+  /// Get the proxy protocol enabled flag
+  bool
+  get_is_proxy_protocol() const
+  {
+    return is_proxy_protocol;
+  }
+  /// Set the proxy protocol enabled flag on the port
+  void
+  set_is_proxy_protocol(bool state = true)
+  {
+    is_proxy_protocol = state;
+  }
+
   virtual int
   populate_protocol(std::string_view *results, int n) const
   {
@@ -643,6 +663,113 @@ class NetVConnection : public AnnotatedVConnection
   NetVConnection(const NetVConnection &) = delete;
   NetVConnection &operator=(const NetVConnection &) = delete;
 
+  enum class ProxyProtocolVersion {
+    UNDEFINED,
+    V1,
+    V2,
+  };
+
+  enum class ProxyProtocolData {
+    UNDEFINED,
+    SRC,
+    DST,
+  };
+
+  int
+  set_proxy_protocol_addr(const ProxyProtocolData src_or_dst, ts::TextView &ip_addr_str)
+  {
+    int ret = -1;
+
+    if (src_or_dst == ProxyProtocolData::SRC) {
+      ret = ats_ip_pton(ip_addr_str, &pp_info.src_addr);
+    } else {
+      ret = ats_ip_pton(ip_addr_str, &pp_info.dst_addr);
+    }
+    return ret;
+  }
+
+  int
+  set_proxy_protocol_src_addr(ts::TextView src)
+  {
+    return set_proxy_protocol_addr(ProxyProtocolData::SRC, src);
+  }
+
+  int
+  set_proxy_protocol_dst_addr(ts::TextView src)
+  {
+    return set_proxy_protocol_addr(ProxyProtocolData::DST, src);
+  }
+
+  int
+  set_proxy_protocol_port(const ProxyProtocolData src_or_dst, in_port_t port)
+  {
+    if (src_or_dst == ProxyProtocolData::SRC) {
+      pp_info.src_addr.port() = htons(port);
+    } else {
+      pp_info.dst_addr.port() = htons(port);
+    }
+    return port;
+  }
+
+  int
+  set_proxy_protocol_src_port(in_port_t port)
+  {
+    return set_proxy_protocol_port(ProxyProtocolData::SRC, port);
+  }
+
+  int
+  set_proxy_protocol_dst_port(in_port_t port)
+  {
+    return set_proxy_protocol_port(ProxyProtocolData::DST, port);
+  }
+
+  void
+  set_proxy_protocol_version(const ProxyProtocolVersion ver)
+  {
+    pp_info.proxy_protocol_version = ver;
+  }
+
+  ProxyProtocolVersion
+  get_proxy_protocol_version()
+  {
+    return pp_info.proxy_protocol_version;
+  }
+
+  sockaddr const *get_proxy_protocol_addr(const ProxyProtocolData);
+
+  sockaddr const *
+  get_proxy_protocol_src_addr()
+  {
+    return get_proxy_protocol_addr(ProxyProtocolData::SRC);
+  }
+
+  uint16_t
+  get_proxy_protocol_src_port()
+  {
+    return ats_ip_port_host_order(this->get_proxy_protocol_addr(ProxyProtocolData::SRC));
+  }
+
+  sockaddr const *
+  get_proxy_protocol_dst_addr()
+  {
+    return get_proxy_protocol_addr(ProxyProtocolData::DST);
+  }
+
+  uint16_t
+  get_proxy_protocol_dst_port()
+  {
+    return ats_ip_port_host_order(this->get_proxy_protocol_addr(ProxyProtocolData::DST));
+  };
+
+  typedef struct _ProxyProtocol {
+    ProxyProtocolVersion proxy_protocol_version = ProxyProtocolVersion::UNDEFINED;
+    uint16_t ip_family;
+    IpEndpoint src_addr;
+    IpEndpoint dst_addr;
+  } ProxyProtocol;
+
+  ProxyProtocol pp_info;
+
 protected:
   IpEndpoint local_addr;
   IpEndpoint remote_addr;
@@ -653,6 +780,8 @@ class NetVConnection : public AnnotatedVConnection
   bool is_internal_request;
   /// Set if this connection is transparent.
   bool is_transparent;
+  /// Set if proxy protocol is enabled
+  bool is_proxy_protocol;
   /// Set if the next write IO that empties the write buffer should generate an event.
   int write_buffer_empty_event;
   /// NetVConnection Context.
@@ -667,6 +796,7 @@ inline NetVConnection::NetVConnection()
     got_remote_addr(false),
     is_internal_request(false),
     is_transparent(false),
+    is_proxy_protocol(false),
     write_buffer_empty_event(0),
     netvc_context(NET_VCONNECTION_UNSET)
 {

iocore/net/Makefile.am

@@ -131,6 +131,8 @@ libinknet_a_SOURCES = \
 	P_UnixNetVConnection.h \
 	P_UnixPollDescriptor.h \
 	P_UnixUDPConnection.h \
+	ProxyProtocol.h \
+	ProxyProtocol.cc \
 	Socks.cc \
 	SNIActionPerformer.cc \
 	SSLCertLookup.cc \