Skip to content

Commit d9cd13c

Browse files
committed
detailed design for session persistence
1 parent 97ebb04 commit d9cd13c

File tree

1 file changed

+159
-0
lines changed

1 file changed

+159
-0
lines changed

docs/proposals/session-persistence.md

Lines changed: 159 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,8 +18,167 @@ Enable NGINX Gateway Fabric to support session persistence for both NGINX Plus a
1818
- Extend session persistence support to TLSRoutes or other Layer 4 route types.
1919
- Supporting the `sameSite` cookie directive for NGINX Plus session persistence, which may be considered in the future as the Gateway API `sessionPersistence` specification evolves.
2020

21+
## Introduction
22+
23+
For NGINX OSS, session persistence is enabled by setting `loadBalancingMethod: ip_hash` on UpstreamSettingsPolicy, which adds the `ip_hash` directive to upstreams and provides IP-based affinity.
24+
For NGINX Plus, session persistence defined on `HTTPRouteRule/GRPCRouteRule` is translated into sticky cookie upstream configuration with secure, httpOnly, host-only cookies and a path derived from HTTPRoute matches (or defaulted for GRPCRoutes), so sessions stick to a chosen backend.
25+
26+
### Understanding the NGINX directives
27+
28+
**ip_hash**
29+
30+
The [ip_hash](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#ip_hash) directive enables session persistence by routing requests from the same client IP address to the same upstream server. It uses the client’s IP address as a hash key to determine the target server, ensuring consistent routing for users behind a single IP. If the chosen server becomes unavailable, NGINX automatically selects the next available upstream server.
31+
32+
Syntax:
33+
34+
```bash
35+
ip_hash;
36+
```
37+
38+
**sticky (cookie method)**
39+
40+
The [sticky](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#sticky) directive enables session persistence using a cookie to identify the upstream server handling a client’s session. When configured with the cookie parameter, NGINX sends a cookie token in the client response in a `Set-Cookie` header, allowing the browser to route subsequent requests with that cookie to the same upstream server.
41+
42+
Syntax:
43+
44+
```bash
45+
sticky cookie name [expires=time] [domain=domain] [httponly] [samesite=strict|lax|none|$variable] [secure] [path=path];
46+
```
47+
48+
Key Parameters:
49+
cookie <name> – Defines the session cookie name.
50+
expires=<time> – Sets cookie lifetime; omit to make it session-based. `max` – Special value for expires that sets expiry to `31 Dec 2037 23:55:55 GMT`.
51+
domain=<domain> - Sets the domain for the cookie scope.
52+
path=<path> - Sets the path for the cookie scope.
53+
samesite=[strict|lax|none|$variable] - Sets the sameSite attribute for the cookie.
54+
secure - Sets the `secure` attribute for the cookie.
55+
httpOnly - Sets the `httpOnly` attribute for the cookie.
56+
57+
### Session Persistence for NGINX OSS users
58+
59+
In OSS, session persistence is provided by configuring upstreams to use the `ip_hash` load-balancing method. NGINX hashes the client IP to select an upstream server, so requests from the same IP are routed to the same upstream as long as it is available. If that server becomes unavailable, NGINX automatically selects another server in the upstream group. Session affinity quality with `ip_hash` depends on NGINX seeing the real client IP. In environments with external load balancers or proxies, operators must ensure appropriate `real_ip_header/set_real_ip_from` configuration so that `$remote_addr` reflects the end-user address otherwise, stickiness will be determined by the address of the front-end proxy rather than the actual client.
60+
61+
To surface this behavior, UpstreamSettingsPolicy is extended with a load-balancing method field:
62+
63+
```go
64+
// UpstreamSettingsPolicySpec defines the desired state of the UpstreamSettingsPolicy.
65+
type UpstreamSettingsPolicySpec struct {
66+
// ZoneSize is the size of the shared memory zone used by the upstream. This memory zone is used to share
67+
// the upstream configuration between nginx worker processes. The more servers that an upstream has,
68+
// the larger memory zone is required.
69+
// Default: OSS: 512k, Plus: 1m.
70+
// Directive: https://nginx.org/en/docs/http/ngx_http_upstream_module.html#zone
71+
//
72+
// +optional
73+
ZoneSize *Size `json:"zoneSize,omitempty"`
74+
75+
// KeepAlive defines the keep-alive settings.
76+
//
77+
// +optional
78+
KeepAlive *UpstreamKeepAlive `json:"keepAlive,omitempty"`
79+
80+
// LoadBalancingMethod specifies the load balancing algorithm to be used for the upstream.
81+
//
82+
// +optional
83+
// +kubebuilder:default:=random two least_conn
84+
LoadBalancingMethod *LoadBalancingType `json:"loadBalancingMethod,omitempty"`
85+
86+
// TargetRefs identifies API object(s) to apply the policy to.
87+
// Objects must be in the same namespace as the policy.
88+
// Support: Service
89+
//
90+
// TargetRefs must be _distinct_. The `name` field must be unique for all targetRef entries in the UpstreamSettingsPolicy.
91+
//
92+
// +kubebuilder:validation:MinItems=1
93+
// +kubebuilder:validation:MaxItems=16
94+
// +kubebuilder:validation:XValidation:message="TargetRefs Kind must be: Service",rule="self.all(t, t.kind=='Service')"
95+
// +kubebuilder:validation:XValidation:message="TargetRefs Group must be core",rule="self.exists(t, t.group=='') || self.exists(t, t.group=='core')"
96+
// +kubebuilder:validation:XValidation:message="TargetRef Name must be unique",rule="self.all(p1, self.exists_one(p2, p1.name == p2.name))"
97+
//nolint:lll
98+
TargetRefs []gatewayv1alpha2.LocalPolicyTargetReference `json:"targetRefs"`
99+
}
100+
101+
// LoadBalancingType defines supported load balancing methods.
102+
//
103+
// +kubebuilder:validation:Enum=ip_hash;random two least_conn
104+
type LoadBalancingType string
105+
106+
const (
107+
// LoadBalancingTypeIPHash enables IP hash-based load balancing,
108+
// ensuring requests from the same client IP are routed to the same upstream server.
109+
// NGINX directive: https://nginx.org/en/docs/http/ngx_http_upstream_module.html#ip_hash
110+
LoadBalancingTypeIPHash LoadBalancingType = "ip_hash"
111+
112+
// LoadBalancingTypeRandomTwoLeastConnection enables a variation of least-connections
113+
// balancing that randomly selects two servers and forwards traffic to the one with
114+
// fewer active connections.
115+
// NGINX directive least_conn: https://nginx.org/en/docs/http/ngx_http_upstream_module.html#least_conn
116+
// NGINX directive random: https://nginx.org/en/docs/http/ngx_http_upstream_module.html#random
117+
LoadBalancingTypeRandomTwoLeastConnection LoadBalancingType = "random two least_conn"
118+
)
119+
```
120+
121+
Note: `LoadBalancingMethod` is optional and defaults to `random two least_conn`. Adding this optional field is a non-breaking change and does not require a version bump in alignment with the [Kubernetes API compatibility guidelines](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api_changes.md#on-compatibility).
122+
123+
### Session Persistence for NGINX Plus users
124+
125+
In NGINX Plus, session persistence is implemented with the `sticky` directive.The directive supports cookie, header, and learn modes; this design only discusses the cookie-based method and the rest are out of scope.
126+
Users can configure [sessionPersistence](https://gateway-api.sigs.k8s.io/reference/spec/?h=sessionpersistence#sessionpersistence) on HTTPRouteRule or GRPCRouteRule, and NGINX Gateway Fabric will map that configuration to `sticky cookie` and associated cookie attributes as described below. The current specification for Session Persistence can be found [here](https://gateway-api.sigs.k8s.io/reference/spec/#sessionpersistence).
127+
128+
#### Mapping the Gateway API fields to NGINX directives
129+
130+
| Spec Field | NGINX Directive | Notes / Limitations |
131+
|----------------------------------------|----------------------------|-------------------------------------------------------------------------------------------------------------------------------------|
132+
| `sessionName` | `name` | Direct mapping to `sticky cookie` name. |
133+
| `absoluteTimeout` | `expires` | Only used when `cookieConfig.lifetimeType=Permanent`; not enforced for `Session` cookies. |
134+
| `idleTimeout` | _not supported_ | NGINX does not support idle-based invalidation for sticky cookies. Sessions expire only when the cookie expires or the session ends.|
135+
| `type` | `cookie` | Only cookie-based persistence is supported. If Header is specified, the sessionPersistence spec is ignored and a warning/status message is reported on the route, but the route itself remains valid. |
136+
| `cookieConfig.lifetimeType=Session` | _no `expires` set_ | Session cookies expire when the browser session ends. |
137+
| `cookieConfig.lifetimeType=Permanent` | `expires=<absoluteTimeout>`| Cookie persists until the specified timeout. `absoluteTimeout` is required when `lifetimeType` is `Permanent`. |
138+
| no matching spec field | _no `domain` attribute_ | Cookies are host-only for both `HTTPRoute` and `GRPCRoute`. |
139+
| no matching spec field | `path` | Behavior is described separately for `HTTPRoute` below. |
140+
| no matching spec field | `secure` | Enabled by default for all routes. |
141+
| no matching spec field | `httpOnly` | Enabled by default for all routes. |
142+
143+
144+
145+
#### Domain and Path selection for Routes
146+
147+
Cookies use the [domain](https://datatracker.ietf.org/doc/html/rfc6265?#section-5.1.3) and [path](https://datatracker.ietf.org/doc/html/rfc6265?#section-5.1.4) attributes to control when the browser sends them back to the server. Domain limits the cookie to a host (and its subdomains, if set), while path limits it to URLs under a specific path prefix. Together they control where the browser sends the cookie, and therefore where session persistence actually applies.
148+
149+
For **HTTPRoutes**, we do not set the `domain` attribute. Deriving a broader domain (for example, a common suffix across hostnames or a parent domain) would widen the cookie scope to sibling subdomains and increase the risk of cross-host leakage. Since users cannot explicitly configure this field, inferring a shared domain would also be vulnerable to abuse. Leaving domain unset ensures each cookie is scoped to the exact host that issued it.
150+
151+
To determine the cookie `path` for HTTPRoutes, we handle the simple case where there is a single path match as follows:
152+
153+
| Path Value | Path Match Type | Cookie `Path` Value | Cookie Match Expectations |
154+
|-------------------------------------|-----------------|---------------------|---------------------------------------------------------------------------------------------------------------------------------------------------|
155+
| `/hello-exact` | Exact | `/hello-exact` | Cookie header is sent for `/hello-exact` path only. |
156+
| `/hello-prefix` | Prefix | `/hello-prefix` | Cookie header is sent for `/hello-prefix` and any subpath starting with `/hello-prefix` (e.g. `/hello-prefix/foo`). |
157+
| `/hello-regex/[a-zA-Z0-9_-]+$` | Regex | `/hello-regex` | Cookie header is sent for any request whose path starts with `/hello-regex` and matches the regex in the location block (e.g. `/hello-regex/a`, `/hello-regex/abc123`). The regex still determines which requests match the route on the server side. |
158+
159+
When there are multiple path matches that share the same sessionPersistence configuration, we derive a single cookie path by computing the longest common prefix that ends on a path-segment boundary `/`. If no non-empty common prefix on a segment boundary exists, we fall back to `/` which is allowing all paths.
160+
161+
For **GRPCRoutes**, we do not set explicit cookie `domain` or `path` attributes. Leaving `domain` unset keeps cookies host-only, and omitting `path` means the user agent applies its default path derivation. This avoids guessing a cookie scope from gRPC routing metadata. gRPC routing is driven by a combination of listener hostnames, methods, and header matches, none of which map cleanly onto a single stable cookie scope: methods are too granular, hostnames may be broad or wildcarded, and header-based matches are inherently dynamic. Any attempt to derive a `domain` or `path` from this information would likely be ambiguous or over-scoped.
162+
163+
164+
These decisions let HTTPRoute traffic benefit from path-scoped cookies while keeping cookie domain to host-only for both HTTPRoutes and GRPCRoutes to avoid cross-host leakage.
165+
For GRPCRoutes, we only provide basic sessionPersistence because typical gRPC clients do not implement browser-style cookie storage and replay. Cookies are treated as ordinary headers, so applications must handle them explicitly rather than relying on an automatic client-side cookie store.
166+
167+
### Edge Cases
168+
169+
- If both Kubernetes Service-level session affinity and Gateway API sessionPersistence are configured for the same traffic, the route MUST be rejected, with a status condition explaining that the two mechanisms are incompatible.
170+
- For traffic-splitting configurations, if cookie-based session persistence is enabled, sessions must remain pinned consistently across the split backends.
171+
172+
### Future work
173+
174+
- Define clear precedence and additional restrictions when SessionPersistence is configured via a separate policy.
175+
- Add support for the `sameSite` cookie attribute in a way that remains compliant with the Gateway API specification.
176+
21177
## Useful Links
22178

23179
- Session Persistence [specification](https://gateway-api.sigs.k8s.io/reference/spec/#sessionpersistence).
24180
- Extended Session Persistence [GEP](https://gateway-api.sigs.k8s.io/geps/gep-1619).
25181
- RFC standard for [Set-Cookie](https://datatracker.ietf.org/doc/html/rfc6265#section-4.1) header.
182+
- [Security risks with subdomain](https://blog.stackademic.com/session-security-risks-with-subdomains-2802c56d681f).
183+
- [Cookie Security](https://www.appsecmonkey.com/blog/cookie-security), read the section `Malicious Subdomains`.
184+
- [gRPC Metadata](https://grpc.io/docs/guides/metadata/)

0 commit comments

Comments
 (0)