Skip to content

Use PP SRC IP as Client IP #12761

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
cmcfarlen wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Use PP SRC IP as Client IP #12761

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1391a66
Restore old feature to copy PP src IP to remote ip of NetVConn, but t…
cmcfarlen Dec 16, 2025
547969e
Use get_client_addr in some places it might matter.
cmcfarlen Dec 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions doc/admin-guide/configuration/proxy-protocol.en.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,9 @@ By default, |TS| uses client's IP address that is from the peer when it applies
enable PROXY protocol and want to apply ACL against the IP address delivered by PROXY protocol, you need to have ``PROXY`` in
:ts:cv:`proxy.config.acl.subjects`.

If you specify the server_ports flag `pp-clnt` then the client IP address used for the
transaction will be the one provided by proxy protocol.

1. HTTP Forwarded Header

The client IP address in the PROXY protocol header is passed to the origin server via an HTTP `Forwarded:
Expand Down
1 change: 1 addition & 0 deletions doc/admin-guide/files/records.yaml.en.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -786,6 +786,7 @@ HTTP Engine
ip-resolve Value IP address resolution style.
proto Value List of supported session protocols.
pp Enable Proxy Protocol.
pp-clnt Use the Proxy Protocol SRC IP as the client IP.
ssl SSL terminated.
quic QUIC terminated.
tr-full Fully transparent (inbound and outbound)
Expand Down
8 changes: 5 additions & 3 deletions doc/admin-guide/logging/formatting.en.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -499,16 +499,19 @@ incoming/outgoing ports, and network interfaces used during transactions.
Field Source Description
===== ============== ==========================================================
chi Client IP address of the client's host. If :ref:`Proxy Protocol <proxy-protocol>`
is used, this represents the IP address of the peer, rather than
the client IP behind the peer.
is configured with the pp-clnt flag, this represents the proxy protocol SRC IP address, otherwise
the IP is that of the connected peer.
chih Client IP address of the client's host, in hexadecimal.
chiv Client IP address of the client's host verified by a plugin. If not available,
``chi`` is used.
rchi Remote Client This is alway the IP address of the inbound remote peer.
rchh Remote Client The IP address of the inbound remote peer in hexadecimal.
hii Proxy IP address for the proxy's incoming interface (to which
the client connected).
hiih Proxy IP address for the proxy's incoming interface (to which
the client connected), in hexadecimal.
chp Client Port number of the client's host.
rchp Remote Client Port number of the inbound remote peer.
php Proxy Response TCP port number from which |TS| serviced the request.
pqsi Proxy Request IP address from which |TS| issued the proxy request to the
origin server. Cache hits will result in a value of ``0``.
Expand All @@ -529,7 +532,6 @@ ppd Proxy Protocol Destination IP received via Proxy Protocol context from the
Dest IP to the |TS|
ppa Proxy Protocol The Authority TLV from Proxy Protocol context from the LB
Authority to the |TS|

===== ============== ==========================================================

.. note::
Expand Down
3 changes: 2 additions & 1 deletion include/iocore/net/AcceptOptions.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,5 +85,6 @@ struct AcceptOptions {
bool f_mptcp = false;

/// Proxy Protocol enabled
bool f_proxy_protocol = false;
bool f_proxy_protocol = false;
bool f_proxy_protocol_client_src = false;
};
51 changes: 47 additions & 4 deletions include/iocore/net/NetVConnection.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
#include "iocore/net/NetVCOptions.h"
#include "iocore/net/ProxyProtocol.h"

#include <cstdint>
#include <string_view>
#include <optional>

Expand Down Expand Up @@ -282,6 +283,12 @@ class NetVConnection : public VConnection, public PluginUserArgs<TS_USER_ARGS_VC
/** Returns local port. */
uint16_t get_local_port();

/** Returns remote sockaddr storage. */
sockaddr const *get_client_addr();
IpEndpoint const &get_client_endpoint();

uint16_t get_client_port();

/** Returns remote sockaddr storage. */
sockaddr const *get_remote_addr();
IpEndpoint const &get_remote_endpoint();
Expand Down Expand Up @@ -426,11 +433,18 @@ class NetVConnection : public VConnection, public PluginUserArgs<TS_USER_ARGS_VC
{
return is_proxy_protocol;
}
/// Get the proxy protocol copy src IP flag
bool
get_is_proxy_protocol_cp_src() const
{
return is_proxy_protocol_cp_src;
}
/// Set the proxy protocol enabled flag on the port
void
set_is_proxy_protocol(bool state = true)
set_is_proxy_protocol(bool state, bool cp_src_ip)
{
is_proxy_protocol = state;
is_proxy_protocol = state;
is_proxy_protocol_cp_src = cp_src_ip;
}

virtual int
Expand Down Expand Up @@ -510,7 +524,8 @@ class NetVConnection : public VConnection, public PluginUserArgs<TS_USER_ARGS_VC
N_SERVICES,
};

IpEndpoint local_addr;
IpEndpoint local_addr;
// This can be the remote peer address or the proxy protocol SRC IP address
IpEndpoint remote_addr;
ProxyProtocol pp_info;

Expand All @@ -523,7 +538,8 @@ class NetVConnection : public VConnection, public PluginUserArgs<TS_USER_ARGS_VC
/// Set if this connection is transparent.
bool is_transparent = false;
/// Set if proxy protocol is enabled
bool is_proxy_protocol = false;
bool is_proxy_protocol = false;
bool is_proxy_protocol_cp_src = false;
/// This is essentially a tri-state, we leave it undefined to mean no MPTCP support
std::optional<bool> mptcp_state;
/// Set if the next write IO that empties the write buffer should generate an event.
Expand Down Expand Up @@ -693,6 +709,33 @@ NetVConnection::_set_service(QUICSupport *instance)
this->_set_service(NetVConnection::Service::QUIC, instance);
}

inline sockaddr const *
NetVConnection::get_client_addr()
{
if (pp_info.version != ProxyProtocolVersion::UNDEFINED && is_proxy_protocol_cp_src) {
return get_proxy_protocol_src_addr();
} else {
return get_remote_addr();
}
}

inline IpEndpoint const &
NetVConnection::get_client_endpoint()
{
if (pp_info.version != ProxyProtocolVersion::UNDEFINED && is_proxy_protocol_cp_src) {
return pp_info.src_addr;
} else {
return remote_addr;
}
}

/// @return The remote port in host order.
inline uint16_t
NetVConnection::get_client_port()
{
return ats_ip_port_host_order(this->get_client_addr());
}

inline sockaddr const *
NetVConnection::get_remote_addr()
{
Expand Down
3 changes: 3 additions & 0 deletions include/proxy/ProxySession.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@

#pragma once

#include "tscore/ink_inet.h"
#include "tscore/ink_platform.h"
#include "tscore/ink_resolver.h"
#include "tscore/TSSystemState.h"
Expand Down Expand Up @@ -116,6 +117,8 @@ class ProxySession : public VConnection, public PluginUserArgs<TS_USER_ARGS_SSN>
virtual uint64_t get_received_frame_count(uint64_t type) const;

// Replicate NetVConnection API
virtual sockaddr const *get_client_addr() const;
virtual uint16_t get_client_port() const;
virtual sockaddr const *get_remote_addr() const;
virtual sockaddr const *get_local_addr();

Expand Down
17 changes: 17 additions & 0 deletions include/proxy/ProxyTransaction.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
#pragma once

#include "proxy/ProxySession.h"
#include <cstdint>
#include <string_view>

class HttpSM;
Expand Down Expand Up @@ -131,6 +132,10 @@ class ProxyTransaction : public VConnection
void set_verified_client_addr(const sockaddr *addr);
sockaddr const *get_verified_client_addr() const;

// this is the client address for the transaction, which may not be the same as the connection remote address
sockaddr const *get_client_addr() const;
uint16_t get_client_port() const;

// This function must return a non-negative number that is different for two in-progress transactions with the same proxy_ssn
// session.
//
Expand Down Expand Up @@ -336,6 +341,18 @@ ProxyTransaction::get_verified_client_addr() const
return reinterpret_cast<const struct sockaddr *>(&_verified_addr);
}

inline sockaddr const *
ProxyTransaction::get_client_addr() const
{
return _proxy_ssn ? _proxy_ssn->get_client_addr() : nullptr;
}

inline uint16_t
ProxyTransaction::get_client_port() const
{
return _proxy_ssn ? _proxy_ssn->get_client_port() : 0;
}

inline void
ProxyTransaction::set_verified_client_addr(const sockaddr *addr)
{
Expand Down
2 changes: 2 additions & 0 deletions include/proxy/logging/LogAccess.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,8 @@ class LogAccess
int marshal_host_interface_ip(char *); // STR
int marshal_client_host_ip_verified(char *); // STR
int marshal_client_host_port(char *); // INT
int marshal_remote_host_ip(char *); // STR
int marshal_remote_host_port(char *); // STR
int marshal_client_auth_user_name(char *); // STR
int marshal_client_req_timestamp_sec(char *); // INT
int marshal_client_req_timestamp_ms(char *); // INT
Expand Down
41 changes: 22 additions & 19 deletions include/records/RecHttp.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -265,6 +265,8 @@ struct HttpProxyPort {
uint8_t m_family = AF_INET; ///< IP address family.
/// True if proxy protocol is required on incoming requests.
bool m_proxy_protocol = false;
/// True if the port should use the proxy protocol src address as the connection remote address
bool m_proxy_protocol_client_src = false;
/// True if inbound connects (from client) are transparent.
bool m_inbound_transparent_p = false;
/// True if outbound connections (to origin servers) are transparent.
Expand Down Expand Up @@ -415,25 +417,26 @@ struct HttpProxyPort {
static const char *const DEFAULT_VALUE;

// Keywords (lower case versions, but compares should be case insensitive)
static const char *const OPT_FD_PREFIX; ///< Prefix for file descriptor value.
static const char *const OPT_OUTBOUND_IP_PREFIX; ///< Prefix for inbound IP address.
static const char *const OPT_INBOUND_IP_PREFIX; ///< Prefix for outbound IP address.
static const char *const OPT_IPV6; ///< IPv6.
static const char *const OPT_IPV4; ///< IPv4
static const char *const OPT_TRANSPARENT_INBOUND; ///< Inbound transparent.
static const char *const OPT_TRANSPARENT_OUTBOUND; ///< Outbound transparent.
static const char *const OPT_TRANSPARENT_FULL; ///< Full transparency.
static const char *const OPT_TRANSPARENT_PASSTHROUGH; ///< Pass-through non-HTTP.
static const char *const OPT_ALLOW_PLAIN; ///< Backup to plain HTTP.
static const char *const OPT_SSL; ///< SSL (experimental)
static const char *const OPT_QUIC; ///< QUIC (experimental)
static const char *const OPT_PROXY_PROTO; ///< Proxy Protocol
static const char *const OPT_PLUGIN; ///< Protocol Plugin handle (experimental)
static const char *const OPT_BLIND_TUNNEL; ///< Blind tunnel.
static const char *const OPT_COMPRESSED; ///< Compressed.
static const char *const OPT_HOST_RES_PREFIX; ///< Set DNS family preference.
static const char *const OPT_PROTO_PREFIX; ///< Transport layer protocols.
static const char *const OPT_MPTCP; ///< MPTCP.
static const char *const OPT_FD_PREFIX; ///< Prefix for file descriptor value.
static const char *const OPT_OUTBOUND_IP_PREFIX; ///< Prefix for inbound IP address.
static const char *const OPT_INBOUND_IP_PREFIX; ///< Prefix for outbound IP address.
static const char *const OPT_IPV6; ///< IPv6.
static const char *const OPT_IPV4; ///< IPv4
static const char *const OPT_TRANSPARENT_INBOUND; ///< Inbound transparent.
static const char *const OPT_TRANSPARENT_OUTBOUND; ///< Outbound transparent.
static const char *const OPT_TRANSPARENT_FULL; ///< Full transparency.
static const char *const OPT_TRANSPARENT_PASSTHROUGH; ///< Pass-through non-HTTP.
static const char *const OPT_ALLOW_PLAIN; ///< Backup to plain HTTP.
static const char *const OPT_SSL; ///< SSL (experimental)
static const char *const OPT_QUIC; ///< QUIC (experimental)
static const char *const OPT_PROXY_PROTO; ///< Proxy Protocol
static const char *const OPT_PLUGIN; ///< Protocol Plugin handle (experimental)
static const char *const OPT_BLIND_TUNNEL; ///< Blind tunnel.
static const char *const OPT_COMPRESSED; ///< Compressed.
static const char *const OPT_HOST_RES_PREFIX; ///< Set DNS family preference.
static const char *const OPT_PROTO_PREFIX; ///< Transport layer protocols.
static const char *const OPT_MPTCP; ///< MPTCP.
static const char *const OPT_PROXY_PROTO_CLIENT_SRC_IP; ///< The Proxy protocol SRC IP address is used as the client's IP address

static std::vector<self> &m_global; ///< Global ("default") data.

Expand Down
1 change: 1 addition & 0 deletions include/ts/ts.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2036,6 +2036,7 @@ TSVConn TSTransformOutputVConnGet(TSVConn connp);
/* --------------------------------------------------------------------------
Net VConnections */
struct sockaddr const *TSNetVConnRemoteAddrGet(TSVConn vc);
struct sockaddr const *TSNetVConnClientAddrGet(TSVConn vc);

/**
Opens a network connection to the host specified by ip on the port
Expand Down
10 changes: 9 additions & 1 deletion src/api/InkAPI.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4692,7 +4692,7 @@ TSHttpSsnClientAddrGet(TSHttpSsn ssnp)
if (cs == nullptr) {
return nullptr;
}
return cs->get_remote_addr();
return cs->get_client_addr();
}
sockaddr const *
TSHttpTxnClientAddrGet(TSHttpTxn txnp)
Expand Down Expand Up @@ -6168,6 +6168,14 @@ TSNetVConnRemoteAddrGet(TSVConn connp)
return vc->get_remote_addr();
}

sockaddr const *
TSNetVConnClientAddrGet(TSVConn connp)
{
sdk_assert(sdk_sanity_check_iocore_structure(connp) == TS_SUCCESS);
NetVConnection *vc = reinterpret_cast<NetVConnection *>(connp);
return vc->get_client_addr();
}

TSAction
TSNetConnect(TSCont contp, sockaddr const *addr)
{
Expand Down
2 changes: 1 addition & 1 deletion src/iocore/net/SNIActionPerformer.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -417,7 +417,7 @@ SNI_IpAllow::SNIAction(SSL &ssl, ActionItem::Context const & /* ctx ATS_UNUSED *
const sockaddr *client_ip = nullptr;
for (int i = 0; i < IpAllow::Subject::MAX_SUBJECTS; ++i) {
if (IpAllow::Subject::PEER == IpAllow::subjects[i]) {
client_ip = ssl_vc->get_remote_addr();
client_ip = ssl_vc->get_client_addr();
Copy link
Member

@maskit maskit Dec 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I don't think we should do this. If the setting value were CLIENT then this would make sense, but "PEER" was picked as a term that means the direct peer to distinguish it from the ambiguous term "client".

Also, this is unrelated to the change, I found that this if-else lacks the support for PLUGIN (verified address). I'll add it.

Copy link
Contributor Author

@cmcfarlen cmcfarlen Dec 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The point of this PR and the new pp-clnt flag (if configured) is to use the actual/true client address (from proxy protocol) for PEER or Client instead of the literal peer host's address. So, in all places where PEER is used, we need to check if the PP SRC IP should be used instead. This is what get_client_addr does. I'm open to better names for this function.

break;
} else if (IpAllow::Subject::PROXY == IpAllow::subjects[i] &&
ssl_vc->get_proxy_protocol_version() != ProxyProtocolVersion::UNDEFINED) {
Expand Down
8 changes: 4 additions & 4 deletions src/iocore/net/SSLClientUtils.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ verify_callback(int signature_ok, X509_STORE_CTX *ctx)
Dbg(dbg_ctl_ssl_verify, "verification error:num=%d:%s:depth=%d", err, X509_verify_cert_error_string(err), depth);
const char *sni_name;
char buff[INET6_ADDRSTRLEN];
ats_ip_ntop(netvc->get_remote_addr(), buff, INET6_ADDRSTRLEN);
ats_ip_ntop(netvc->get_client_addr(), buff, INET6_ADDRSTRLEN);
Copy link
Member

@maskit maskit Dec 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this function is for origin connections. If so, remote_addr means origin's address, and client_addr means ATS's address.

Copy link
Member

@maskit maskit Dec 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, it may not. But it's confusing because the client is ATS on origin connections.

Copy link
Contributor Author

@cmcfarlen cmcfarlen Dec 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is a better name? effective_remote_addr? I agree it's odd if the connection is not the incoming request one, but it looked like this function is generic. The only important function call change in this last commit is for ACLs. That is client side and does require get_client_addr.

if (netvc->options.sni_servername) {
sni_name = netvc->options.sni_servername.get();
} else {
Expand Down Expand Up @@ -110,15 +110,15 @@ verify_callback(int signature_ok, X509_STORE_CTX *ctx)
sni_name = reinterpret_cast<unsigned char *>(netvc->options.sni_servername.get());
} else {
sni_name = reinterpret_cast<unsigned char *>(buff);
ats_ip_ntop(netvc->get_remote_addr(), buff, INET6_ADDRSTRLEN);
ats_ip_ntop(netvc->get_client_addr(), buff, INET6_ADDRSTRLEN);
}
if (validate_hostname(cert, sni_name, false, &matched_name)) {
Dbg(dbg_ctl_ssl_verify, "Hostname %s verified OK, matched %s", sni_name, matched_name);
ats_free(matched_name);
} else { // Name validation failed
// Get the server address if we did't already compute it
if (netvc->options.sni_servername) {
ats_ip_ntop(netvc->get_remote_addr(), buff, INET6_ADDRSTRLEN);
ats_ip_ntop(netvc->get_client_addr(), buff, INET6_ADDRSTRLEN);
}
// If we got here the verification failed
Warning("SNI (%s) not in certificate. Action=%s server=%s(%s)", sni_name, enforce_mode ? "Terminate" : "Continue",
Expand All @@ -141,7 +141,7 @@ verify_callback(int signature_ok, X509_STORE_CTX *ctx)
sni_name = reinterpret_cast<unsigned char *>(netvc->options.sni_servername.get());
} else {
sni_name = reinterpret_cast<unsigned char *>(buff);
ats_ip_ntop(netvc->get_remote_addr(), buff, INET6_ADDRSTRLEN);
ats_ip_ntop(netvc->get_client_addr(), buff, INET6_ADDRSTRLEN);
}
Warning("TS_EVENT_SSL_VERIFY_SERVER plugin failed the origin certificate check for %s. Action=%s SNI=%s",
netvc->options.ssl_servername.get(), enforce_mode ? "Terminate" : "Continue", sni_name);
Expand Down
2 changes: 1 addition & 1 deletion src/iocore/net/SSLDiags.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,7 @@ SSLDiagnostic(const SourceLocation &loc, bool debug, SSLNetVConnection *vc, cons
ip_text_buffer ip_buf = {'\0'};

if (vc) {
ats_ip_ntop(vc->get_remote_addr(), ip_buf, sizeof(ip_buf));
ats_ip_ntop(vc->get_client_addr(), ip_buf, sizeof(ip_buf));
Copy link
Member

@maskit maskit Dec 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above. The name doesn't make sense on origin connections.

}

es = reinterpret_cast<unsigned long>(pthread_self());
Expand Down
6 changes: 2 additions & 4 deletions src/iocore/net/SSLNetVConnection.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -356,10 +356,8 @@ SSLNetVConnection::read_raw_data()
if (pp_ipmap->count() > 0) {
Dbg(dbg_ctl_proxyprotocol, "proxy protocol has a configured allowlist of trusted IPs - checking");

// At this point, using get_remote_addr() will return the ip of the
// proxy source IP, not the Proxy Protocol client ip. Since we are
// checking the ip of the actual source of this connection, this is
// what we want now.
// Using get_remote_addr() will return the ip of the
// proxy source IP, not the Proxy Protocol client ip.
if (!pp_ipmap->contains(swoc::IPAddr(get_remote_addr()))) {
Dbg(dbg_ctl_proxyprotocol, "Source IP is NOT in the configured allowlist of trusted IPs - closing connection");
r = -ENOTCONN; // Need a quick close/exit here to refuse the connection!!!!!!!!!
Expand Down
6 changes: 5 additions & 1 deletion src/iocore/net/Server.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -291,7 +291,11 @@ Server::setup_fd_for_listen(bool non_blocking, const NetProcessor::AcceptOptions
}

if (opt.f_proxy_protocol) {
Dbg(dbg_ctl_proxyprotocol, "Proxy Protocol enabled.");
if (opt.f_proxy_protocol_client_src) {
Dbg(dbg_ctl_proxyprotocol, "Proxy Protocol enabled(Using src IP as remote IP).");
} else {
Dbg(dbg_ctl_proxyprotocol, "Proxy Protocol enabled.");
}
}

#if defined(TCP_MAXSEG)
Expand Down
Loading