Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(kuma-dp): rework on the virtual probes to support probing tcp and grpc ports #10624

Merged
merged 50 commits into from
Aug 14, 2024

Conversation

jijiechen
Copy link
Member

@jijiechen jijiechen commented Jun 25, 2024

Checklist prior to review

Implements this MADR: gRPC and TCP probes

  • Link to relevant issue as well as docs and UI issues
  • This will not break child repos: it doesn't hardcode values (.e.g "kumahq" as a image registry) and it will work on Windows, system specific functions like syscall.Mkfifo have equivalent implementation on the other OS --
  • Tests (Unit test, E2E tests, manual test on universal and k8s)
    • tested with unit and manual tests
    • Don't forget ci/ labels to run additional/fewer tests
  • Do you need to update UPGRADE.md? --
    • No, it's compatible
  • Does it need to be backported according to the backporting policy? (this GH action will add "backport" label based on these file globs, if you want to prevent it from adding the "backport" label use no-backport-autolabel label)
    • No

Changelog: feat(kuma-dp): support TCP and gRPC probes for data planes running on Kubernetes

jijiechen added 15 commits June 11, 2024 15:20
Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
…nager

Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
… configuration

Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
@jijiechen jijiechen added the ci/run-full-matrix PR: Runs all possible e2e test combination (expensive use carefully) label Jun 25, 2024
@jijiechen jijiechen requested a review from a team as a code owner June 25, 2024 03:19
@jijiechen jijiechen requested review from bartsmykla and lukidzi and removed request for a team June 25, 2024 03:19
Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
@jijiechen
Copy link
Member Author

Hi reviewers, sorry that this is a large PR. To help you review the code easily, let me explain what are included in this PR.
This PR basicly refactors the existing HTTP-only "virtual probes" and implements the new model of "virtual probe" that support all three kinds of pod probes: TCP, gRPC and HTTP.

Here are the break down tasks:

  1. introduce a new component into kuma-dp to start the virtual probe server, it's an HTTP server. This is placed at directory app/kuma-dp/pkg/dataplane/probes
  2. configure kuma-dp from the injector.go to run the virtual probe server according to user's auto-injector configuration
  3. in the function of OverridePodProbes that is called by injector.go, change the existing virtual probe overriding to support all the three kinds of probes
  4. exclude the virtual probe server port on the ways of settings of transparent proxy:
    1. kuma-sidecar: PodRedirect in file pkg/transparentproxy/kubernetes/kubernetes.go
    2. kuma-cni: NewIntermediateConfig in file app/cni/pkg/cni/annotations_linux.go

@jijiechen
Copy link
Member Author

The PR also includes some minor/revelant refactorings.

Here are some of them:

  1. Removed Probes to be on dataplane objects sinec we don't generate an insecure listener for the virtual probes server any more. So I removed this field and its message definition.
  2. Removed grpc health check proto files from the repo and use the package from GitHub since it was included in the testserver package previously and if I introduce the same proto into a package under kuma-dp, it will generate a conflict panic.
  3. Added the capabilities of supporting scheme and httpHeaders for HTTP probes, supporting of timeout is added for all three kinds of probes.

@jijiechen jijiechen requested review from lukidzi and lobkovilya and removed request for lukidzi July 12, 2024 03:40
…e application-probe-proxy feature

Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
…into pods

Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
…oxy enabled at the same time

Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
@jijiechen jijiechen disabled auto-merge August 14, 2024 09:19
@jijiechen jijiechen requested review from lukidzi and lahabana August 14, 2024 09:20
Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
@jijiechen
Copy link
Member Author

According to recent review suggestions, here are the latest changes:

  1. Introduced a new concept of "application probe proxy", and corresponding annotations/configurations
  2. Brought back previous versions of "virtual probes"
  3. Built the compatibility of using "virtual probes" by disabling "application probe proxy"
  4. Marked "virtual probes" related annotations as Deprecated

Basically, the idea is, "application probe proxy" will be enabled defaultly and it's successor of "virtual probes".
User can still use "virtual probes" by disabling "application probe proxy" (set the port to 0)
When virtual probes are previously disabled using CP configuration or pod annotations, "application probe proxy" will also be disabled defaultly.

Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
@jijiechen jijiechen merged commit dc1daad into kumahq:master Aug 14, 2024
35 checks passed
@jijiechen jijiechen deleted the tcp-grpc-probes branch August 14, 2024 15:51
lahabana pushed a commit that referenced this pull request Dec 6, 2024
## Motivation

<!-- Why are we doing this change -->

Add Jay Jijie Chen as Kuma maintainer.

### Nominee's first and last name

Jay Chen

### Nominee's email address and GitHub user name

Email: jijie.chen@outlook.com
Github: jijiechen

### An explanation of why the nominee should be a maintainer/reviewer

I wish to become a Kuma project maintainer to better engage in the
discussions, contributions, code reviews for substantial time.

Here are my significant contribution PRs and Issues for this project:
- [ci(actions): add the 'build-test-distribute' GitHub
Action](https://github.com/kumahq/kuma/pull/8360/files)
- [feat(transparent-proxy): deprecate argument
'redirect-inbound-port-v6' and introduce 'ip-family-mode'
](#8939)
- [feat(kuma-cni): add a init container to validate that iptables rules
are applied](#9699)
- [feat(kuma-dp): rework on the virtual probes to support probing tcp
and grpc ports](#10624)
- [test(framework): inspect clusters, stats and policies when DebugKube
on E2E failures](#11746)

## Implementation information

<!-- Explain how this was done and potentially alternatives considered
and discarded -->

## Supporting documentation

<!-- Is there a MADR? An Issue? A related PR? -->

<!--
> Changelog: skip
-->
<!--
Uncomment the above section to explicitly set a [`> Changelog:` entry
here](https://github.com/kumahq/kuma/blob/master/CONTRIBUTING.md#submitting-a-patch)?
-->

Signed-off-by: Jay Chen <1180092+jijiechen@users.noreply.github.com>
Signed-off-by: Bart Smykla <bartek@smykla.com>
Co-authored-by: Bart Smykla <bartek@smykla.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci/run-full-matrix PR: Runs all possible e2e test combination (expensive use carefully)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support TCP and grpc liveness probes
4 participants