Enable Gateway Spec.Listener sharding

[evankanderson]

As part of my quest to implement Gateway API for Knative, I've been looking at how to enable TLS support for potentially 10k Knative Services in a single cluster. As a reminder, we're looking for the following properties, which seem reasonable:

1. Support all Services with a single *.example.com DNS entry pointing to a single pool of proxies
2. For HTTP traffic, we may be able to use a shared port 80 Gateway entry and a permissive Listener (all namespaces)
3. Enable TLS for each Service by provisioning a hostname Listener in a shared namespace and Service namespace-scoped TLS Secrets, a ReferenceGrant and HTTPRoute allowing that namesppace-specific hostname to be used only by the namespace in question
   1.Support a second Gateway of the same gatewayClass, which is only exposed for sevice.namespace.svc-scoped names, and is not available off-cluster

The current design of a Spec.Listener list on the Gateway resource probably limits this for a single gateway to around 1-2k Knative Services -- I'm guessing that the Listener component is around 300 bytes, and the managedFields tracking probably adds another 250 bytes or so. With a 1MB limit on etcd entities for storage, this means 1M / 450 ~= 2.2k Listeners, assuming no other usage or status information that grows with the number of entities.

Additionally, it was pointed out during code-review that this Gateway object may become an apiserver hot-spot for updates, either requiring careful queueing on the controller side, or introducing a lot of "concurrent modification" errors.

Talking with @youngnick, he suggested that an implementation could choose to coalesce multiple Gateways to serve off a single IP address, but the current Contour implementation does not do this. Doing this would enable creating a separate Gateway for each Knative Service, which would also make cleanup during object deletion and updates easier.

Furthermore, taking point 4 into account, you would need some way to ensure that the external TLS Gateway entries provisioned in 3 were associated with the Gateway address from 2, not the Gateway address from 4.

[daixiang0]

https://github.com/knative-sandbox/gateway-api is 404 :(

[evankanderson]

Whoops, it is https://github.com/knative-sandbox/net-gateway-api (typo)

[robscott]

We've left room for this in the spec, but I'm not sure any implementations have decided to merge Gateways yet:

gateway-api/apis/v1beta1/gateway_types.go

Lines 76 to 78 in 541e9fc

	// determines that the Listeners in the group are "compatible". An
	// implementation MAY also group together and collapse compatible
	// Listeners belonging to different Gateways.

At a high level, it seems like it would be difficult/impossible to require implementations to support this kind of concept because some have very real limits in terms of the underlying layers they are translating this API to.

[evankanderson]

Do the real underlying limits in the layers correspond to the 1MB etcd row limit? I'm very sympathetic to the idea that some implementations might not want to have to distribute 5k certificates to each proxy instance ahead-of-time based on the listener definition, but that seems like it should be separate from the orchestration limitations of a single Kubernetes object and managing concurrent updates to that object.

[robscott]

Do the real underlying limits in the layers correspond to the 1MB etcd row limit?

I'm sure that would be a factor for some. For others like cloud providers it may be as simple as underlying quotas or limits that are difficult to exceed.

So to clarify, I can be OK with adding some guidance for how implementations could handle thousands of listeners, but I'm not sure how many implementations will actually be able to support that, and because of that, I don't think we can require any sharding support.

[evankanderson]

I'm fine with there being limits, but I'm uncomfortable with there being no consistent, supported mechanism for more than about 2k listeners on a single IP address. I'm okay with some implementations saying "can't go above 500 listeners in any case", but having 3 different ways of going past the etcd row limit would be unfortunate.

[youngnick]

It seems like the big thing here is the behavior if you specify an IP address, right? If you want many thousands of Listeners attached to a single IP address, then you need to be able to merge Gateways.

Maybe we should add something that says "Practically, Gateways should not really go above x Listeners per Gateway (because etcd size limits, apiserver hotspot concerns, etc). If you want more than x Listeners on a single IP address, you should use an implementation that supports merging Gateways with the same IP address", where x is a number like 500 or 1000. I'd probably err on the 500 side, personally, but we can talk there.

Yes, I know that no implementation supports that yet, but there's a pretty clear market segment that people can strive to serve. (I know at least Envoy Gateway's design expects some level of Gateway merging to occur).

[dprotaso]

I've created a GEP issue to help address this - #1713