Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement traffic redirection exclusion based on proxy config and user-provided values #10134

Merged
merged 6 commits into from
Apr 29, 2021

Conversation

ishustava
Copy link
Contributor

  • If -proxy-id is provided to the redirect-traffic command, exclude any listener ports from inbound traffic redirection. This includes envoy_prometheus_bind_addr, envoy_stats_bind_addr, and the ListenerPort from the Expose configuration.
  • Allow users to provide additional inbound and outbound ports, outbound CIDRs and additional user IDs to be excluded from traffic redirection. This affects both the traffic-redirect command and the iptables SDK package.
  • Additionally, use the proxy outbound port from the proxy config if it's set.

…r-provided values.

* If -proxy-id is provided to the redirect-traffic command, exclude any listener ports
  from inbound traffic redirection. This includes envoy_prometheus_bind_addr,
  envoy_stats_bind_addr, and the ListenerPort from the Expose configuration.
* Allow users to provide additional inbound and outbound ports, outbound CIDRs
  and additional user IDs to be excluded from traffic redirection.
  This affects both the traffic-redirect command and the iptables SDK package.
@github-actions github-actions bot added the theme/cli Flags and documentation for the CLI interface label Apr 27, 2021
@ishustava ishustava added backport/1.10 theme/connect Anything related to Consul Connect, Service Mesh, Side Car Proxies type/enhancement Proposed improvement or new feature and removed theme/cli Flags and documentation for the CLI interface labels Apr 27, 2021
@vercel vercel bot temporarily deployed to Preview – consul April 27, 2021 19:45 Inactive
@vercel vercel bot temporarily deployed to Preview – consul-ui-staging April 27, 2021 19:45 Inactive
Copy link
Contributor

@freddygv freddygv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for expanding this! I made a couple minor comments and one less trivial one about the Expose config. Depending on timing we could push the solution to that issue to after beta2 though.

"Outbound port to exclude from traffic redirection. May be provided multiple times.")
c.flags.Var((*flags.AppendSliceValue)(&c.excludeOutboundCIDRs), "exclude-outbound-cidr",
"Outbound CIDR to exclude from traffic redirection. May be provided multiple times.")
c.flags.Var((*flags.AppendSliceValue)(&c.excludeUIDs), "exclude-uid",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be valuable to also be able to exclude GIDs?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it could be. The main reason we're adding this is to better support the VM use case when you might want to exclude other processes/uids/gids from traffic redirection. I'm inclined to wait for users' feedback on this one because I'm not sure what would be valuable there.

command/connect/redirecttraffic/redirect_traffic.go Outdated Show resolved Hide resolved
}

// Exclude the ListenerPort from Expose configs from inbound traffic redirection.
for _, exposePath := range svc.Proxy.Expose.Paths {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing I'm realizing is missing is this bit of magic:
https://www.consul.io/docs/connect/registration/service-registration#checks

If someone enables this, Consul will dynamically expose checks on a listener for each check:

func (a *Agent) listenerPortLocked(svcID structs.ServiceID, checkID structs.CheckID) (int, error) {

We don't expose what port is allocated in any of our APIs, similarly to how we don't expose check definitions via any of our APIs, only check statuses. See this comment for more information on why: #7330 (comment)

There are a few potential solutions to discover those ports off the top of my head:

  1. Expand the structs.HealthCheck type to include the exposed path.
  2. Add a Meta map to checks, like we have for services, and include the exposed path there.
  3. Include all check definition details in structs.HealthCheck. That comes with the security risk mentioned above.

Copy link
Contributor

@freddygv freddygv Apr 28, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So based on the discussion today we should go with #1.

The exposed ports are generated in a few places.

One is where we add checks:

port, err := a.listenerPortLocked(sid, cid)

port, err := a.listenerPortLocked(sid, cid)

In addCheck it's easy to expand the HealthCheck type, since we already have the check pointer to it.

And the other is where we add services with checks:

func (a *Agent) rerouteExposedChecks(serviceID structs.ServiceID, proxyAddr string) error {

func (a *Agent) resetExposedChecks(serviceID structs.ServiceID) {

In these two other methods we'll need to query a.State.Check, I think, and modify that. (Assuming the state lock is held)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a great find! Let me address this in a separate PR so that this one can be included in beta2.

command/connect/redirecttraffic/redirect_traffic.go Outdated Show resolved Hide resolved
@vercel vercel bot temporarily deployed to Preview – consul-ui-staging April 28, 2021 19:30 Inactive
@vercel vercel bot temporarily deployed to Preview – consul April 28, 2021 19:30 Inactive
@ishustava ishustava requested a review from freddygv April 28, 2021 19:32
Copy link
Contributor

@freddygv freddygv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@ishustava ishustava merged commit 8dffb89 into master Apr 29, 2021
@ishustava ishustava deleted the redirect-traffic-exclusions branch April 29, 2021 16:21
@hc-github-team-consul-core
Copy link
Contributor

🍒 If backport labels were added before merging, cherry-picking will start automatically.

To retroactively trigger a backport after merging, add backport labels and re-run https://circleci.com/gh/hashicorp/consul/359312.

@hc-github-team-consul-core
Copy link
Contributor

🍒✅ Cherry pick of commit 8dffb89 onto release/1.10.x succeeded!

hc-github-team-consul-core pushed a commit that referenced this pull request Apr 29, 2021
…er-provided values (#10134)

* Use proxy outbound port from TransparentProxyConfig if provided
* If -proxy-id is provided to the redirect-traffic command, exclude any listener ports
  from inbound traffic redirection. This includes envoy_prometheus_bind_addr,
  envoy_stats_bind_addr, and the ListenerPort from the Expose configuration.
* Allow users to provide additional inbound and outbound ports, outbound CIDRs
  and additional user IDs to be excluded from traffic redirection.
  This affects both the traffic-redirect command and the iptables SDK package.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
theme/connect Anything related to Consul Connect, Service Mesh, Side Car Proxies type/enhancement Proposed improvement or new feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants