xds: generate listeners directly from API gateway snapshot #17398

sarahalsmiller · 2023-05-17T21:51:06Z

Description

Instead of converting each API gateway config snapshot into an ingress gateway config snapshot and then generating xDS resources from that, generate xDS resources directly from the API gateway config snapshot.

Testing & Reproduction steps

Links

PR Checklist

updated test coverage
external facing docs updated
appropriate backport labels added
not a security concern

* XDS primitive generation for endpoints and clusters Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> * server_test * deleted extra file * add missing parents to test --------- Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* XDS primitive generation for endpoints and clusters Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> * server_test * deleted extra file * add missing parents to test * checkpoint * delete extra file * httproute flattening code * linting issue * so close on this, calling for tonight * unit test passing * add in header manip to virtual host * upstream rebuild commented out * Use consistent upstream name whether or not we're rebuilding * Start working through route naming logic * Fix typos in test descriptions * Simplify route naming logic * Simplify RebuildHTTPRouteUpstream * Merge additional compiled discovery chains instead of overwriting * Use correct chain for flattened route, clean up + add TODOs * Remove empty conditional branch * Restore previous variable declaration Limit the scope of this PR * Clean up, improve TODO * add logging, clean up todos * clean up function --------- Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

Co-authored-by: John Maguire <john.maguire@hashicorp.com>

Each listener would previously have all upstreams from any route that bound to the listener. This is problematic when a route bound to one listener also binds to other listeners and so includes upstreams for multiple listeners. The list for a given listener would then wind up including upstreams for other listeners.

…-makeAPIGatewayListeners

nathancoleman · 2023-05-19T14:48:28Z

agent/consul/discoverychain/gateway.go

+	return consolidateHTTPRoutes(matches, listener.Name, gateway)
+}
+
+func RebuildHTTPRouteUpstream(route structs.HTTPRouteConfigEntry, listener structs.APIGatewayListener) structs.Upstream {


"rebuild" makes me feel like we're doing something a lot more intense than just initializing a struct. Thoughts on changing to just "build"?

Suggested change

func RebuildHTTPRouteUpstream(route structs.HTTPRouteConfigEntry, listener structs.APIGatewayListener) structs.Upstream {

func BuildHTTPRouteUpstream(route structs.HTTPRouteConfigEntry, listener structs.APIGatewayListener) structs.Upstream {

t-eckert

I love how much more clear the logic reads now! There are still some TODOs which I'm not sure are for after this PR or not. I'll approve anyway so you can move forward.

agent/consul/discoverychain/gateway.go

agent/structs/config_entry_gateways.go

agent/xds/listeners_apigateway.go

nathancoleman

This is looking great! I think there's still a little cleanup that you and I can do to make it polished and shiny

agent/consul/discoverychain/gateway.go

agent/proxycfg/snapshot.go

nathancoleman · 2023-05-19T15:41:31Z

agent/xds/listeners.go

-			// TODO Find a cleaner solution, can't currently pass unexported property types
-			var err error
-			cfgSnap.IngressGateway, err = cfgSnap.APIGateway.ToIngress(cfgSnap.Datacenter)
-			if err != nil {
-				return nil, err
-			}
-			listeners, err := s.makeIngressGatewayListeners(a.Address, cfgSnap)
+			listeners, err := s.makeAPIGatewayListeners(a.Address, cfgSnap)


This is the crux of the whole change that we're making: instead of converting to an ingress gateway snapshot and generating xDS resources from that, we generate xDS resources directly from our API gateway snapshot 🎉

agent/xds/listeners_apigateway.go

agent/consul/discoverychain/gateway.go

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

…teUpstream to BuildHTTPRouteUpstream

…rp/consul into NET-3671-makeAPIGatewayListeners

agent/consul/discoverychain/gateway.go

agent/proxycfg/snapshot.go

…rp/consul into NET-3671-makeAPIGatewayListeners

* API Gateway XDS Primitives, endpoints and clusters (#17002) * XDS primitive generation for endpoints and clusters Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> * server_test * deleted extra file * add missing parents to test --------- Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> * Routes for API Gateway (#17158) * XDS primitive generation for endpoints and clusters Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> * server_test * deleted extra file * add missing parents to test * checkpoint * delete extra file * httproute flattening code * linting issue * so close on this, calling for tonight * unit test passing * add in header manip to virtual host * upstream rebuild commented out * Use consistent upstream name whether or not we're rebuilding * Start working through route naming logic * Fix typos in test descriptions * Simplify route naming logic * Simplify RebuildHTTPRouteUpstream * Merge additional compiled discovery chains instead of overwriting * Use correct chain for flattened route, clean up + add TODOs * Remove empty conditional branch * Restore previous variable declaration Limit the scope of this PR * Clean up, improve TODO * add logging, clean up todos * clean up function --------- Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> * checkpoint, skeleton, tests not passing * checkpoint * endpoints xds cluster configuration * resources test fix * fix reversion in resources_test * checkpoint * Update agent/proxycfg/api_gateway.go Co-authored-by: John Maguire <john.maguire@hashicorp.com> * unit tests passing * gofmt * add deterministic sorting to appease the unit test gods * remove panic * Find ready upstream matching listener instead of first in list * Clean up, improve TODO * Modify getReadyUpstreams to filter upstreams by listener (#17410) Each listener would previously have all upstreams from any route that bound to the listener. This is problematic when a route bound to one listener also binds to other listeners and so includes upstreams for multiple listeners. The list for a given listener would then wind up including upstreams for other listeners. * clean up todos, references to api gateway in listeners_ingress * merge in Nathan's fix * Update agent/consul/discoverychain/gateway.go * cleanup current todos, remove snapshot manipulation from generation code * Update agent/structs/config_entry_gateways.go Co-authored-by: Thomas Eckert <teckert@hashicorp.com> * Update agent/consul/discoverychain/gateway.go Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> * Update agent/consul/discoverychain/gateway.go Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> * Update agent/proxycfg/snapshot.go Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> * clarified header comment for FlattenHTTPRoute, changed RebuildHTTPRouteUpstream to BuildHTTPRouteUpstream * simplify cert logic * Delete scratch * revert route related changes in listener PR * Update agent/consul/discoverychain/gateway.go * Update agent/proxycfg/snapshot.go * clean up uneeded extra lines in endpoints --------- Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> Co-authored-by: John Maguire <john.maguire@hashicorp.com> Co-authored-by: Thomas Eckert <teckert@hashicorp.com>

sarahalsmiller and others added 11 commits April 26, 2023 10:17

checkpoint, skeleton, tests not passing

36c0d63

checkpoint

c7d6f79

endpoints xds cluster configuration

463c5fe

resources test fix

94188ea

fix reversion in resources_test

ed82d8c

checkpoint

e3f1e85

Update agent/proxycfg/api_gateway.go

8f5ea9e

Co-authored-by: John Maguire <john.maguire@hashicorp.com>

unit tests passing

890aaed

gofmt

8728cac

sarahalsmiller requested a review from nathancoleman May 17, 2023 21:51

github-actions bot added the theme/envoy/xds Related to Envoy support label May 17, 2023

sarahalsmiller and others added 6 commits May 18, 2023 10:07

add deterministic sorting to appease the unit test gods

562c789

remove panic

077741d

Find ready upstream matching listener instead of first in list

029ae91

Clean up, improve TODO

3204325

clean up todos, references to api gateway in listeners_ingress

b302597

sarahalsmiller changed the base branch from NET-2063-Implementation-API-GW-Use-XDS-primitives-instead-of-Ingress-GW-primitives to NET-3673-endpointsFromSnapshotAPIGateway May 18, 2023 19:43

Merge branch 'NET-3673-endpointsFromSnapshotAPIGateway' into NET-3671…

6cdd69a

…-makeAPIGatewayListeners

vercel bot deployed to Preview – consul May 18, 2023 19:51 View deployment

merge in Nathan's fix

61e78ee

nathancoleman changed the title ~~WIP- Net 3671 make api gateway listeners~~ xds: generate listeners directly from API gateway snapshot May 18, 2023

nathancoleman reviewed May 19, 2023

View reviewed changes

t-eckert approved these changes May 19, 2023

View reviewed changes

agent/consul/discoverychain/gateway.go Outdated Show resolved Hide resolved

agent/structs/config_entry_gateways.go Outdated Show resolved Hide resolved

agent/xds/listeners_apigateway.go Outdated Show resolved Hide resolved

agent/xds/listeners_apigateway.go Outdated Show resolved Hide resolved

nathancoleman mentioned this pull request May 19, 2023

xds: generate routes directly from API gateway snapshot #17392

Merged

4 tasks

nathancoleman requested changes May 19, 2023

View reviewed changes

andrewstucki reviewed May 19, 2023

View reviewed changes

agent/xds/listeners_apigateway.go Outdated Show resolved Hide resolved

sarahalsmiller commented May 19, 2023

View reviewed changes

agent/consul/discoverychain/gateway.go Outdated Show resolved Hide resolved

sarahalsmiller and others added 4 commits May 19, 2023 13:29

Update agent/consul/discoverychain/gateway.go

d1bf338

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

Update agent/proxycfg/snapshot.go

257ac0e

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

clarified header comment for FlattenHTTPRoute, changed RebuildHTTPRou…

b73a1ce

…teUpstream to BuildHTTPRouteUpstream

Merge branch 'NET-3671-makeAPIGatewayListeners' of github.com:hashico…

0d3454c

…rp/consul into NET-3671-makeAPIGatewayListeners

Base automatically changed from NET-3673-endpointsFromSnapshotAPIGateway to main May 19, 2023 18:51

Merge branch 'main' into NET-3671-makeAPIGatewayListeners

606afa2

sarahalsmiller added pr/no-changelog PR does not need a corresponding .changelog entry pr/no-backport labels May 19, 2023

vercel bot deployed to Preview – consul May 19, 2023 18:57 View deployment

sarahalsmiller and others added 3 commits May 19, 2023 15:26

simplify cert logic

b9b4e1c

Delete scratch

1b413e1

revert route related changes in listener PR

c905897

sarahalsmiller commented May 22, 2023

View reviewed changes

agent/consul/discoverychain/gateway.go Outdated Show resolved Hide resolved

Update agent/consul/discoverychain/gateway.go

ff3f6b3

sarahalsmiller commented May 22, 2023

View reviewed changes

agent/proxycfg/snapshot.go Outdated Show resolved Hide resolved

sarahalsmiller and others added 4 commits May 22, 2023 11:08

Update agent/proxycfg/snapshot.go

7180fbc

clean up uneeded extra lines in endpoints

1700631

Merge branch 'NET-3671-makeAPIGatewayListeners' of github.com:hashico…

9c30c40

…rp/consul into NET-3671-makeAPIGatewayListeners

Merge branch 'main' into NET-3671-makeAPIGatewayListeners

c47dd2a

sarahalsmiller enabled auto-merge (squash) May 22, 2023 16:55

nathancoleman requested review from andrewstucki and nathancoleman May 22, 2023 19:36

sarahalsmiller and others added 4 commits May 22, 2023 14:45

Merge branch 'main' into NET-3671-makeAPIGatewayListeners

c4c8e9d

clean up extra function

967fe11

Replace TODO with FUTURE comment for removal

3826310

Remove inapplicable TODO

92351d5

nathancoleman approved these changes May 22, 2023

View reviewed changes

sarahalsmiller merged commit e2a81aa into main May 22, 2023

sarahalsmiller deleted the NET-3671-makeAPIGatewayListeners branch May 22, 2023 21:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

xds: generate listeners directly from API gateway snapshot #17398

xds: generate listeners directly from API gateway snapshot #17398

sarahalsmiller commented May 17, 2023 •

edited by nathancoleman

Loading

nathancoleman May 19, 2023

t-eckert left a comment

nathancoleman left a comment •

edited

Loading

nathancoleman May 19, 2023

	func RebuildHTTPRouteUpstream(route structs.HTTPRouteConfigEntry, listener structs.APIGatewayListener) structs.Upstream {
	func BuildHTTPRouteUpstream(route structs.HTTPRouteConfigEntry, listener structs.APIGatewayListener) structs.Upstream {

xds: generate listeners directly from API gateway snapshot #17398

xds: generate listeners directly from API gateway snapshot #17398

Conversation

sarahalsmiller commented May 17, 2023 • edited by nathancoleman Loading

Description

Testing & Reproduction steps

Links

PR Checklist

nathancoleman May 19, 2023

Choose a reason for hiding this comment

t-eckert left a comment

Choose a reason for hiding this comment

nathancoleman left a comment • edited Loading

Choose a reason for hiding this comment

nathancoleman May 19, 2023

Choose a reason for hiding this comment

sarahalsmiller commented May 17, 2023 •

edited by nathancoleman

Loading

nathancoleman left a comment •

edited

Loading