Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement outbound drain #898

Merged
merged 1 commit into from
Apr 9, 2024

Conversation

bleggett
Copy link
Contributor

@bleggett bleggett commented Apr 9, 2024

Similar to #894 - there are some other spawns that we need to guarantee terminate, to avoid stuck tasks.

Most of these are in Outbound, which unlike the others never really had per-connection drain.

This adds drains such that all outbound spawns are guaranteed to terminate when the workload is drained, this avoids several scenarios where task leaks are possible.

What we really need beyond this PR is something like a drain_type and more complex drain channels:

  • Drain(reason: workload gone) -> workload gone, don't wait for connections to "unstick" - just drop.
  • Drain(reason: proxy terminating) -> workloads are not gone, but whole proxy is going - start refusing conns and give workloads a grace period.

This PR is just a stopgap to handle some workload termination corner cases.

@bleggett bleggett requested a review from a team as a code owner April 9, 2024 00:17
@istio-testing istio-testing added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Apr 9, 2024
@bleggett bleggett force-pushed the bleggett/outbound-drain branch from dd34c38 to a13ca3b Compare April 9, 2024 01:32
@bleggett bleggett force-pushed the bleggett/outbound-drain branch from a13ca3b to ddbd7e1 Compare April 9, 2024 03:22
@bleggett
Copy link
Contributor Author

bleggett commented Apr 9, 2024

What we really need beyond this PR is something like a drain_type and more complex drain channels:

  • Drain(reason: workload gone) -> workload gone, don't wait for connections to "unstick" - just drop.
  • Drain(reason: proxy terminating) -> workloads are not gone, but whole proxy is going - start refusing conns and give workloads a grace period.

This PR is just a stopgap to handle some workload termination corner cases.

Signed-off-by: Benjamin Leggett <benjamin.leggett@solo.io>
@bleggett bleggett force-pushed the bleggett/outbound-drain branch from ddbd7e1 to 7efba68 Compare April 9, 2024 03:33
@istio-testing istio-testing merged commit fa7b1aa into istio:master Apr 9, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants