router, formatter: make address formats consistent in substitution_formatter & header_formatter

docs/root/configuration/http/http_conn_man/headers.rst

@@ -587,22 +587,73 @@ Supported variable names are:
       <config_http_conn_man_headers_x-forwarded-for>`.
 
 %DOWNSTREAM_REMOTE_ADDRESS_WITHOUT_PORT%
-    Same as **%DOWNSTREAM_REMOTE_ADDRESS%** excluding port if the address is an IP address.
+    Remote address of the downstream connection, without any port component.
+    IP addresses are the only address type with a port component.
+
+    .. note::
+
+      This may not be the physical remote address of the peer if the address has been inferred from
+      :ref:`Proxy Protocol filter <config_listener_filters_proxy_protocol>` or :ref:`x-forwarded-for
+      <config_http_conn_man_headers_x-forwarded-for>`.
+
+%DOWNSTREAM_REMOTE_PORT%
+    Remote port of the downstream connection.
+    IP addresses are the only address type with a port component.
+
+    .. note::
+
+      This may not be the physical remote address of the peer if the address has been inferred from
+      :ref:`Proxy Protocol filter <config_listener_filters_proxy_protocol>` or :ref:`x-forwarded-for
+      <config_http_conn_man_headers_x-forwarded-for>`.
+
+%DOWNSTREAM_DIRECT_REMOTE_ADDRESS%
+    Direct remote address of the downstream connection. If the address is an IP address it includes both
+    address and port.
+
+    .. note::
+
+      This is always the physical remote address of the peer even if the downstream remote address has
+      been inferred from :ref:`Proxy Protocol filter <config_listener_filters_proxy_protocol>`
+      or :ref:`x-forwarded-for <config_http_conn_man_headers_x-forwarded-for>`.
+
+%DOWNSTREAM_DIRECT_REMOTE_ADDRESS_WITHOUT_PORT%
+    Direct remote address of the downstream connection, without any port component.
+    IP addresses are the only address type with a port component.
+
+    .. note::
+
+      This is always the physical remote address of the peer even if the downstream remote address has
+      been inferred from :ref:`Proxy Protocol filter <config_listener_filters_proxy_protocol>`
+      or :ref:`x-forwarded-for <config_http_conn_man_headers_x-forwarded-for>`.
+
+%DOWNSTREAM_DIRECT_REMOTE_PORT%
+    Direct remote port of the downstream connection.
+    IP addresses are the only address type with a port component.
+
+    .. note::
+
+      This is always the physical remote address of the peer even if the downstream remote address has
+      been inferred from :ref:`Proxy Protocol filter <config_listener_filters_proxy_protocol>`
+      or :ref:`x-forwarded-for <config_http_conn_man_headers_x-forwarded-for>`.
+
 
 %DOWNSTREAM_LOCAL_ADDRESS%
     Local address of the downstream connection. If the address is an IP address it includes both
     address and port.
+
     If the original connection was redirected by iptables REDIRECT, this represents
     the original destination address restored by the
     :ref:`Original Destination Filter <config_listener_filters_original_dst>` using SO_ORIGINAL_DST socket option.
     If the original connection was redirected by iptables TPROXY, and the listener's transparent
     option was set to true, this represents the original destination address and port.
 
 %DOWNSTREAM_LOCAL_ADDRESS_WITHOUT_PORT%
-    Same as **%DOWNSTREAM_LOCAL_ADDRESS%** excluding port if the address is an IP address.
+    Local address of the downstream connection, without any port component.
+    IP addresses are the only address type with a port component.
 
 %DOWNSTREAM_LOCAL_PORT%
-    Similar to **%DOWNSTREAM_LOCAL_ADDRESS_WITHOUT_PORT%**, but only extracts the port portion of the **%DOWNSTREAM_LOCAL_ADDRESS%**
+    Local port of the downstream connection.
+    IP addresses are the only address type with a port component.
 
 %DOWNSTREAM_LOCAL_URI_SAN%
   HTTP
@@ -726,11 +777,36 @@ Supported variable names are:
 
     This works both on request and response headers.
 
+%UPSTREAM_LOCAL_ADDRESS%
+    Local address of the upstream connection. If the address is an IP address it includes both
+    address and port.
+
+    The upstream local address cannot be added to request headers as the upstream host
+    hremote as not been selected when custom request headers are generated.
+
+%UPSTREAM_LOCAL_ADDRESS_WITHOUT_PORT%
+    Local address of the upstream connection, without any port component.
+    IP addresses are the only address type with a port component.
+
+%UPSTREAM_LOCAL_PORT%
+    Local port of the upstream connection.
+    IP addresses are the only address type with a port component.
+
 %UPSTREAM_REMOTE_ADDRESS%
-    Remote address of the upstream host. If the address is an IP address it includes both address
-    and port. The upstream remote address cannot be added to request headers as the upstream host
+    Remote address of the upstream connection. If the address is an IP address it includes both
+    address and port.
+
+    The upstream remote address cannot be added to request headers as the upstream host
     has not been selected when custom request headers are generated.
 
+%UPSTREAM_REMOTE_ADDRESS_WITHOUT_PORT%
+    Remote address of the upstream connection, without any port component.
+    IP addresses are the only address type with a port component.
+
+%UPSTREAM_REMOTE_PORT%
+    Remote port of the upstream connection.
+    IP addresses are the only address type with a port component.
+
 %PER_REQUEST_STATE(reverse.dns.data.name)%
     Populates the header with values set on the stream info filterState() object. To be
     usable in custom request/response headers, these values must be of type

docs/root/configuration/observability/access_log/usage.rst

@@ -414,6 +414,26 @@ The following command operators are supported:
   Local address of the upstream connection. If the address is an IP address it includes both
   address and port.
 
+%UPSTREAM_LOCAL_ADDRESS_WITHOUT_PORT%
+  Local address of the upstream connection, without any port component.
+  IP addresses are the only address type with a port component.
+
+%UPSTREAM_LOCAL_PORT%
+  Local port of the upstream connection.
+  IP addresses are the only address type with a port component.
+
+%UPSTREAM_REMOTE_ADDRESS%
+  Remote address of the upstream connection. If the address is an IP address it includes both
+  address and port.
+
+%UPSTREAM_REMOTE_ADDRESS_WITHOUT_PORT%
+  Remote address of the upstream connection, without any port component.
+  IP addresses are the only address type with a port component.
+
+%UPSTREAM_REMOTE_PORT%
+  Remote port of the upstream connection.
+  IP addresses are the only address type with a port component.
+
 .. _config_access_log_format_upstream_transport_failure_reason:
 
 %UPSTREAM_TRANSPORT_FAILURE_REASON%
@@ -436,8 +456,18 @@ The following command operators are supported:
     <config_http_conn_man_headers_x-forwarded-for>`.
 
 %DOWNSTREAM_REMOTE_ADDRESS_WITHOUT_PORT%
-  Remote address of the downstream connection. If the address is an IP address the output does
-  *not* include port.
+  Remote address of the downstream connection, without any port component.
+  IP addresses are the only address type with a port component.
+
+  .. note::
+
+    This may not be the physical remote address of the peer if the address has been inferred from
+    :ref:`Proxy Protocol filter <config_listener_filters_proxy_protocol>` or :ref:`x-forwarded-for
+    <config_http_conn_man_headers_x-forwarded-for>`.
+
+%DOWNSTREAM_REMOTE_PORT%
+  Remote port of the downstream connection.
+  IP addresses are the only address type with a port component.
 
   .. note::
 
@@ -456,8 +486,18 @@ The following command operators are supported:
     or :ref:`x-forwarded-for <config_http_conn_man_headers_x-forwarded-for>`.
 
 %DOWNSTREAM_DIRECT_REMOTE_ADDRESS_WITHOUT_PORT%
-  The direct remote address of the downstream connection. If the address is an IP address the output does
-  *not* include port.
+  Direct remote address of the downstream connection, without any port component.
+  IP addresses are the only address type with a port component.
+
+  .. note::
+
+    This is always the physical remote address of the peer even if the downstream remote address has
+    been inferred from :ref:`Proxy Protocol filter <config_listener_filters_proxy_protocol>`
+    or :ref:`x-forwarded-for <config_http_conn_man_headers_x-forwarded-for>`.
+
+%DOWNSTREAM_DIRECT_REMOTE_PORT%
+  Direct remote port of the downstream connection.
+  IP addresses are the only address type with a port component.
 
   .. note::
 
@@ -468,14 +508,20 @@ The following command operators are supported:
 %DOWNSTREAM_LOCAL_ADDRESS%
   Local address of the downstream connection. If the address is an IP address it includes both
   address and port.
+
   If the original connection was redirected by iptables REDIRECT, this represents
   the original destination address restored by the
   :ref:`Original Destination Filter <config_listener_filters_original_dst>` using SO_ORIGINAL_DST socket option.
   If the original connection was redirected by iptables TPROXY, and the listener's transparent
   option was set to true, this represents the original destination address and port.
 
 %DOWNSTREAM_LOCAL_ADDRESS_WITHOUT_PORT%
-    Same as **%DOWNSTREAM_LOCAL_ADDRESS%** excluding port if the address is an IP address.
+  Local address of the downstream connection, without any port component.
+  IP addresses are the only address type with a port component.
+
+%DOWNSTREAM_LOCAL_PORT%
+  Local port of the downstream connection.
+  IP addresses are the only address type with a port component.
 
 .. _config_access_log_format_connection_id:
 
@@ -489,9 +535,6 @@ The following command operators are supported:
 %GRPC_STATUS%
   gRPC status code which is easy to interpret with text message corresponding with number.
 
-%DOWNSTREAM_LOCAL_PORT%
-    Similar to **%DOWNSTREAM_LOCAL_ADDRESS_WITHOUT_PORT%**, but only extracts the port portion of the **%DOWNSTREAM_LOCAL_ADDRESS%**
-
 .. _config_access_log_format_req:
 
 %REQ(X?Y):Z%

source/common/formatter/substitution_formatter.cc

@@ -881,6 +881,61 @@ const StreamInfoFormatter::FieldExtractorLookupTbl& StreamInfoFormatter::getKnow
                 return nullptr;
               });
         }},
+       {"UPSTREAM_LOCAL_ADDRESS_WITHOUT_PORT",
+        []() {
+          return StreamInfoAddressFieldExtractor::withoutPort(
+              [](const StreamInfo::StreamInfo& stream_info)
+                  -> std::shared_ptr<const Envoy::Network::Address::Instance> {
+                if (stream_info.upstreamInfo().has_value()) {
+                  return stream_info.upstreamInfo().value().get().upstreamLocalAddress();
+                }
+                return nullptr;
+              });
+        }},
+       {"UPSTREAM_LOCAL_PORT",
+        []() {
+          return StreamInfoAddressFieldExtractor::justPort(
+              [](const StreamInfo::StreamInfo& stream_info)
+                  -> std::shared_ptr<const Envoy::Network::Address::Instance> {
+                if (stream_info.upstreamInfo().has_value()) {
+                  return stream_info.upstreamInfo().value().get().upstreamLocalAddress();
+                }
+                return nullptr;
+              });
+        }},
+       {"UPSTREAM_REMOTE_ADDRESS",
+        []() {
+          return StreamInfoAddressFieldExtractor::withPort(
+              [](const StreamInfo::StreamInfo& stream_info)
+                  -> std::shared_ptr<const Envoy::Network::Address::Instance> {
+                if (stream_info.upstreamInfo() && stream_info.upstreamInfo()->upstreamHost()) {
+                  return stream_info.upstreamInfo()->upstreamHost()->address();
+                }
+                return nullptr;
+              });
+        }},
+       {"UPSTREAM_REMOTE_ADDRESS_WITHOUT_PORT",
+        []() {
+          return StreamInfoAddressFieldExtractor::withoutPort(
+              [](const StreamInfo::StreamInfo& stream_info)
+                  -> std::shared_ptr<const Envoy::Network::Address::Instance> {
+                if (stream_info.upstreamInfo() && stream_info.upstreamInfo()->upstreamHost()) {
+                  return stream_info.upstreamInfo()->upstreamHost()->address();
+                }
+                return nullptr;
+              });
+        }},
+       {"UPSTREAM_REMOTE_PORT",
+        []() {
+          return StreamInfoAddressFieldExtractor::justPort(
+              [](const StreamInfo::StreamInfo& stream_info)
+                  -> std::shared_ptr<const Envoy::Network::Address::Instance> {
+                if (stream_info.upstreamInfo() && stream_info.upstreamInfo()->upstreamHost()) {
+                  return stream_info.upstreamInfo()->upstreamHost()->address();
+                }
+                return nullptr;
+              });
+        }},
        {"UPSTREAM_REQUEST_ATTEMPT_COUNT",
         []() {
           return std::make_unique<StreamInfoUInt64FieldExtractor>(
@@ -923,6 +978,13 @@ const StreamInfoFormatter::FieldExtractorLookupTbl& StreamInfoFormatter::getKnow
                 return stream_info.downstreamAddressProvider().remoteAddress();
               });
         }},
+       {"DOWNSTREAM_REMOTE_PORT",
+        []() {
+          return StreamInfoAddressFieldExtractor::justPort(
+              [](const Envoy::StreamInfo::StreamInfo& stream_info) {
+                return stream_info.downstreamAddressProvider().remoteAddress();
+              });
+        }},
        {"DOWNSTREAM_DIRECT_REMOTE_ADDRESS",
         []() {
           return StreamInfoAddressFieldExtractor::withPort(
@@ -937,6 +999,13 @@ const StreamInfoFormatter::FieldExtractorLookupTbl& StreamInfoFormatter::getKnow
                 return stream_info.downstreamAddressProvider().directRemoteAddress();
               });
         }},
+       {"DOWNSTREAM_DIRECT_REMOTE_PORT",
+        []() {
+          return StreamInfoAddressFieldExtractor::justPort(
+              [](const Envoy::StreamInfo::StreamInfo& stream_info) {
+                return stream_info.downstreamAddressProvider().directRemoteAddress();
+              });
+        }},
        {"CONNECTION_ID",
         []() {
           return std::make_unique<StreamInfoUInt64FieldExtractor>(

source/common/router/header_formatter.cc

@@ -263,8 +263,27 @@ StreamInfoHeaderFormatter::StreamInfoHeaderFormatter(absl::string_view field_nam
       return StreamInfo::Utility::formatDownstreamAddressNoPort(
           *stream_info.downstreamAddressProvider().remoteAddress());
     };
+  } else if (field_name == "DOWNSTREAM_REMOTE_PORT") {
+    field_extractor_ = [](const Envoy::StreamInfo::StreamInfo& stream_info) {
+      return StreamInfo::Utility::formatDownstreamAddressJustPort(
+          *stream_info.downstreamAddressProvider().remoteAddress());
+    };
+  } else if (field_name == "DOWNSTREAM_DIRECT_REMOTE_ADDRESS") {
+    field_extractor_ = [](const Envoy::StreamInfo::StreamInfo& stream_info) {
+      return stream_info.downstreamAddressProvider().directRemoteAddress()->asString();
+    };
+  } else if (field_name == "DOWNSTREAM_DIRECT_REMOTE_ADDRESS_WITHOUT_PORT") {
+    field_extractor_ = [](const Envoy::StreamInfo::StreamInfo& stream_info) {
+      return StreamInfo::Utility::formatDownstreamAddressNoPort(
+          *stream_info.downstreamAddressProvider().directRemoteAddress());
+    };
+  } else if (field_name == "DOWNSTREAM_DIRECT_REMOTE_PORT") {
+    field_extractor_ = [](const Envoy::StreamInfo::StreamInfo& stream_info) {
+      return StreamInfo::Utility::formatDownstreamAddressJustPort(
+          *stream_info.downstreamAddressProvider().directRemoteAddress());
+    };
   } else if (field_name == "DOWNSTREAM_LOCAL_ADDRESS") {
-    field_extractor_ = [](const StreamInfo::StreamInfo& stream_info) {
+    field_extractor_ = [](const Envoy::StreamInfo::StreamInfo& stream_info) {
       return stream_info.downstreamAddressProvider().localAddress()->asString();
     };
   } else if (field_name == "DOWNSTREAM_LOCAL_ADDRESS_WITHOUT_PORT") {
@@ -337,13 +356,55 @@ StreamInfoHeaderFormatter::StreamInfoHeaderFormatter(absl::string_view field_nam
     field_extractor_ = parseSubstitutionFormatField(field_name, formatter_map_);
   } else if (absl::StartsWith(field_name, "DOWNSTREAM_PEER_CERT_V_END")) {
     field_extractor_ = parseSubstitutionFormatField(field_name, formatter_map_);
+  } else if (field_name == "UPSTREAM_LOCAL_ADDRESS") {
+    field_extractor_ = [](const Envoy::StreamInfo::StreamInfo& stream_info) -> std::string {
+      if (stream_info.upstreamInfo().has_value() &&
+          stream_info.upstreamInfo()->upstreamLocalAddress()) {
+        return stream_info.upstreamInfo()->upstreamLocalAddress()->asString();
+      }
+      return "";
+    };
+  } else if (field_name == "UPSTREAM_LOCAL_ADDRESS_WITHOUT_PORT") {
+    field_extractor_ = [](const Envoy::StreamInfo::StreamInfo& stream_info) -> std::string {
+      if (stream_info.upstreamInfo().has_value() &&
+          stream_info.upstreamInfo()->upstreamLocalAddress()) {
+        return StreamInfo::Utility::formatDownstreamAddressNoPort(
+            *stream_info.upstreamInfo()->upstreamLocalAddress());
+      }
+      return "";
+    };
+  } else if (field_name == "UPSTREAM_LOCAL_PORT") {
+    field_extractor_ = [](const Envoy::StreamInfo::StreamInfo& stream_info) -> std::string {
+      if (stream_info.upstreamInfo().has_value() &&
+          stream_info.upstreamInfo()->upstreamLocalAddress()) {
+        return StreamInfo::Utility::formatDownstreamAddressJustPort(
+            *stream_info.upstreamInfo()->upstreamLocalAddress());
+      }
+      return "";
+    };
   } else if (field_name == "UPSTREAM_REMOTE_ADDRESS") {
     field_extractor_ = [](const Envoy::StreamInfo::StreamInfo& stream_info) -> std::string {
       if (stream_info.upstreamInfo() && stream_info.upstreamInfo()->upstreamHost()) {
         return stream_info.upstreamInfo()->upstreamHost()->address()->asString();
       }
       return "";
     };
+  } else if (field_name == "UPSTREAM_REMOTE_ADDRESS_WITHOUT_PORT") {
+    field_extractor_ = [](const Envoy::StreamInfo::StreamInfo& stream_info) -> std::string {
+      if (stream_info.upstreamInfo() && stream_info.upstreamInfo()->upstreamHost()) {
+        return StreamInfo::Utility::formatDownstreamAddressNoPort(
+            *stream_info.upstreamInfo()->upstreamHost()->address());
+      }
+      return "";
+    };
+  } else if (field_name == "UPSTREAM_REMOTE_PORT") {
+    field_extractor_ = [](const Envoy::StreamInfo::StreamInfo& stream_info) -> std::string {
+      if (stream_info.upstreamInfo() && stream_info.upstreamInfo()->upstreamHost()) {
+        return StreamInfo::Utility::formatDownstreamAddressJustPort(
+            *stream_info.upstreamInfo()->upstreamHost()->address());
+      }
+      return "";
+    };
   } else if (absl::StartsWith(field_name, "START_TIME")) {
     field_extractor_ = parseSubstitutionFormatField(field_name, formatter_map_);
   } else if (absl::StartsWith(field_name, "UPSTREAM_METADATA")) {