Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged metrics with consul-dataplane #1635

Merged
merged 9 commits into from
Oct 22, 2022

Conversation

curtbushko
Copy link
Contributor

@curtbushko curtbushko commented Oct 19, 2022

Changes proposed in this PR:

This PR is best reviewed by commit(ish).

f9fe189 - passes metrics flags to the consul-dataplane binary.
ea62a71 - Re-enables the metrics acceptance tests and gets them running with consul-dataplane.
87630a7 - Removes consul-sidecar as it is no longer needed at all for merging metrics.
dde818d6536020 - Please ignore. Git/me being silly.
32cb989 - Update CHANGELOG.md

How I've tested this PR:

  • Ran through the tests including a million times manually.

How I expect reviewers to test this PR:

👀

Checklist:

  • Tests added
  • CHANGELOG entry added

    HashiCorp engineers only, community PRs should not add a changelog entry.
    Entries should use present tense (e.g. Add support for...)

@curtbushko curtbushko force-pushed the curtbushko/agentless-metrics-merging branch from 09c0e45 to d377b69 Compare October 19, 2022 18:57
@curtbushko curtbushko requested a review from ishustava October 19, 2022 19:15
@curtbushko curtbushko self-assigned this Oct 19, 2022
@curtbushko curtbushko marked this pull request as ready for review October 19, 2022 19:48
@curtbushko curtbushko force-pushed the curtbushko/agentless-metrics-merging branch 3 times, most recently from 62d1d90 to 32cb989 Compare October 19, 2022 23:10
Copy link
Contributor

@ishustava ishustava left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking pretty good so far! I left some comments in-line, nothing super-major though. It's mostly about some potential extraneous configuration and missing tests.

acceptance/tests/metrics/metrics_test.go Outdated Show resolved Hide resolved
charts/consul/templates/mesh-gateway-deployment.yaml Outdated Show resolved Hide resolved
charts/consul/values.yaml Show resolved Hide resolved
control-plane/commands.go Show resolved Hide resolved
control-plane/connect-inject/consul_dataplane_sidecar.go Outdated Show resolved Hide resolved
control-plane/connect-inject/consul_dataplane_sidecar.go Outdated Show resolved Hide resolved
// if data.PrometheusKeyFile == "" {
// return corev1.Container{}, fmt.Errorf("must set %q when providing prometheus TLS config", annotationPrometheusKeyFile)
// }
//}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are there any tests for this that we need to remove/migrate?

Copy link
Contributor Author

@curtbushko curtbushko Oct 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tests for this have been migrated over to consul_dataplane_test.go.

Pass metrics flags to consul-dataplane
Cleanup: remove consul-sidecar code, annotations and tests
remove consul-sidecar from values.yaml and bats tests
Get acceptance tests running
@curtbushko curtbushko force-pushed the curtbushko/agentless-metrics-merging branch from 6536020 to c305e76 Compare October 21, 2022 23:16
@curtbushko curtbushko requested a review from ishustava October 21, 2022 23:20
Copy link
Contributor

@wilkermichael wilkermichael left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work!

env := suite.Environment()
cfg := suite.Config()
ctx := env.DefaultContext(t)
ns := ctx.KubectlOptions(t).Namespace

helmValues := map[string]string{
// Remove before merging
"global.imageConsulDataplane": "curtbushko/consul-dataplane:latest",
"global.imageK8S": "curtbushko/consul-k8s-control-plane-dev:latest",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't forget to remove!

env := suite.Environment()
cfg := suite.Config()
ctx := env.DefaultContext(t)
ns := ctx.KubectlOptions(t).Namespace

helmValues := map[string]string{
"global.datacenter": "dc1",
"global.metrics.enabled": "true",
// Remove before merging
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't forget to remove!

Copy link
Contributor

@ishustava ishustava left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!! Just one more comment about /metrics path for gateways, but I won't block on that, assuming it's addressed.

CHANGELOG.md Outdated Show resolved Hide resolved
-log-json={{ $root.Values.global.logJSON }}
-log-json={{ $root.Values.global.logJSON }} \
{{- if (and $root.Values.global.metrics.enabled $root.Values.global.metrics.enableGatewayMetrics) }}
-telemetry-prom-scrape-path={{ $root.Values.connectInject.metrics.defaultPrometheusScrapePath }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to hard-code this one to /metrics (that's how it worked before). This value is controlled by the value from the connectInject stanza which usually does not control gateways. Also, if someone changes this value, the prometheus annotations on the gateway deployment will not change because we're not consuming this value in the gateway annotations.

curtbushko and others added 2 commits October 21, 2022 21:40
Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com>
@curtbushko curtbushko merged commit 754f3ac into main Oct 22, 2022
@curtbushko curtbushko deleted the curtbushko/agentless-metrics-merging branch October 22, 2022 04:09
t-eckert pushed a commit that referenced this pull request Oct 31, 2022
Changes proposed in this PR:
- Add support for merged metrics with consul-dataplane.
- Merging metrics now takes place in the consul-dataplane itself instead of relying on the consul-sidecar
- Pairs with a new flag added to consul-dataplane in PR: Add telemetry-prom-merge-port flag for merged metrics
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants