Jsonnet: improve querier HPA to scale up and down more gradually #6971

pracucci · 2023-12-20T10:10:08Z

What this PR does

Few months ago we rolled out some improvements to querier HPA at Grafana Labs. In this PR I'm proposing to upstream them.

The main idea of these changes is avoiding the querier HPA scales up too aggressively when it wouldn't be required. The main issue looks to be the number of desired replicas measured looking at the 75p number of inflight queries in the query-scheduler queue.

The number of enqueued requests can spike a lot if (a) we receive a sudden burst of queries or (b) read part is experiencing a temporarily (short) slow down. However, in many cases we process these queries even before new queriers are scaled up, but the metric is recorded and the scale up will happen anyway (to a very high number) not being much effective in the end.

The pattern described above is not the only one. We have cases where scaling up queriers is effective, but we should try to improve our querier HPA metric to reduce the ineffective scale up spikes.

For this reason, the changes we did in the querier HPA are:

Scale up
- Max 50% every 2m
- Max 15 pods every 2m (covers the case the current number of pods is small, e.g. 3 + 50% = 5 so +2 pods, but would scale up to +15 pods thanks to this policy)
- NOTE: When multiple policies are specified the policy which allows the highest amount of change is the policy which is selected by default
Scale down
- Max 10% every 2 minutes
Scaling metrics
- 50p (instead of 75p) in-flight queries tracked by query-scheduler
- sum(rate(cortex_querier_request_duration_seconds_sum))

Which issue(s) this PR fixes or relates to

N/A

Checklist

Tests updated.
Documentation added.
CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX].
about-versioning.md updated with experimental features.

dimitarvdimitrov

LGTM with two minor comments

CHANGELOG.md

dimitarvdimitrov · 2023-12-21T09:38:20Z

operations/mimir/autoscaling.libsonnet

@@ -153,7 +153,7 @@
  // `weight` param can be used to control just a portion of the expected queriers with the generated scaled object.
  // For example, if you run multiple querier deployments on different node types, you can use the weight to control which portion of them runs on which nodes.
  // The weight is a number between 0 and 1, where 1 means 100% of the expected queriers.
-  newQuerierScaledObject(name, query_scheduler_container, querier_max_concurrent, min_replicas, max_replicas, target_utilization, weight=1):: self.newScaledObject(name, $._config.namespace, {
+  newQuerierScaledObject(name, query_scheduler_container, querier_container, querier_max_concurrent, min_replicas, max_replicas, target_utilization, weight=1):: self.newScaledObject(name, $._config.namespace, {


querier_container_name sounds more accurate, but should probably change query_scheduler_container too to be consistent. Happy to keep it as it is if you think it's outside of scope

We can do it in this PR.

Done in 236cc5c

Signed-off-by: Marco Pracucci <marco@pracucci.com>

@grafanabot

* feat(helm): Adding KEDA autoscaling support * fix: update changelog * feat(helm): Porting the changes from #6971 into helm chart * feat: Adding better changelog & values documentation outlining the experimental state of the feature, using the same autoscaling rules & behaviours as jsonnet, only supporting newest version of KEDA * fix: Remove duplicate query field, add base url in CHANGELOG.md * helm: align grpc server connection lifetime settings with jsonnet (#7269) * helm: align grpc server connection lifetime settings with jsonnet Co-authored-by: Marco Pracucci <marco@pracucci.com> Signed-off-by: Vladimir Varankin <vladimir.varankin@grafana.com> * helm: rebuild tests Signed-off-by: Vladimir Varankin <vladimir.varankin@grafana.com> --------- Signed-off-by: Vladimir Varankin <vladimir.varankin@grafana.com> Co-authored-by: Marco Pracucci <marco@pracucci.com> * querymiddleware: Fix race condition in shardActiveSeriesMiddleware (#7290) * querymiddleware: race condition in shardActiveSeriesMiddleware s2.Writer is not goroutine-safe to reuse between concurrent requests. Signed-off-by: Vladimir Varankin <vladimir.varankin@grafana.com> * querymiddleware: remove flaky ResponseBodyStreamed test from shardActiveSeriesMiddleware --------- Signed-off-by: Vladimir Varankin <vladimir.varankin@grafana.com> * version: add UserAgent() (#7264) Update pkg/util/version/info.go Update pkg/mimirtool/client/client.go Signed-off-by: Vladimir Varankin <vladimir.varankin@grafana.com> Co-authored-by: Andy Asp <90626759+andyasp@users.noreply.github.com> * helm: remove -server.grpc.keepalive.max-connection-idle from common config (#7298) Signed-off-by: Vladimir Varankin <vladimir.varankin@grafana.com> * Compactor: export estimated number of compaction jobs based on bucket-index (#7299) * Compute number of compaction jobs from bucket index and export it via cortex_bucket_index_compaction_jobs metric. Signed-off-by: Peter Štibraný <pstibrany@gmail.com> * Add PR number. Signed-off-by: Peter Štibraný <pstibrany@gmail.com> * Remove unused parameter name. Signed-off-by: Peter Štibraný <pstibrany@gmail.com> * Make linter happy. Signed-off-by: Peter Štibraný <pstibrany@gmail.com> * Address review feedback. Signed-off-by: Peter Štibraný <pstibrany@gmail.com> * Fix tests. Signed-off-by: Peter Štibraný <pstibrany@gmail.com> --------- Signed-off-by: Peter Štibraný <pstibrany@gmail.com> * Add KubePersistentVolumeFillingUp runbook (#7297) * Add KubePersistentVolumeFillingUp runbook Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> Signed-off-by: Marco Pracucci <marco@pracucci.com> * Added CHANGELOG entry Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed linter Signed-off-by: Marco Pracucci <marco@pracucci.com> --------- Signed-off-by: Marco Pracucci <marco@pracucci.com> Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * Internal: remove unnecessary parameter to NoCompactionMarkFilter (#7301) * Remove unnecessary parameter to NewNoCompactionMarkFilter. Signed-off-by: Peter Štibraný <pstibrany@gmail.com> * Remove unnecessary parameter to NewNoCompactionMarkFilter. Signed-off-by: Peter Štibraný <pstibrany@gmail.com> --------- Signed-off-by: Peter Štibraný <pstibrany@gmail.com> * Name query metrics for easier discovery (#7302) Changes the way query metrics are named to make them easier to search for. Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com> * fix(deps): update module github.com/aws/aws-sdk-go to v1.50.11 (#7288) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * fix(deps): update module github.com/klauspost/compress to v1.17.6 (#7291) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * chore(deps): update anchore/sbom-action action to v0.15.8 (#7286) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * chore(deps): update grafana/agent docker tag to v0.39.2 (#7287) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * chore(deps): update grafana/grafana docker tag to v10.3.1 (#7292) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * fix(deps): update module github.com/failsafe-go/failsafe-go to v0.4.4 (#7289) Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Chore: removed unused parameter from GenerateBlockFromSpec() (#7303) * Chore: removed unused parameter from GenerateBlockFromSpec() Signed-off-by: Marco Pracucci <marco@pracucci.com> * Removed unused variables Signed-off-by: Marco Pracucci <marco@pracucci.com> --------- Signed-off-by: Marco Pracucci <marco@pracucci.com> * Update mimir-prometheus (#7293) * Update mimir-prometheus Signed-off-by: Marco Pracucci <marco@pracucci.com> * Do not shard histogram_avg() Signed-off-by: Marco Pracucci <marco@pracucci.com> * Re-vendored Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fix TestBucketStore_Series_ShouldQueryBlockWithOutOfOrderChunks Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fix TestGroupCompactE2E Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed TestMultitenantCompactor_OutOfOrderCompaction Signed-off-by: Marco Pracucci <marco@pracucci.com> * Reworked TestBucketStore_Series_ShouldQueryBlockWithOutOfOrderChunks Signed-off-by: Marco Pracucci <marco@pracucci.com> * Simplify TestBucketStore_Series_ShouldQueryBlockWithOutOfOrderChunks Signed-off-by: Marco Pracucci <marco@pracucci.com> * Use filepath.Join() instead of hardcoding path separator Signed-off-by: Marco Pracucci <marco@pracucci.com> --------- Signed-off-by: Marco Pracucci <marco@pracucci.com> * Release mimir-distributed Helm chart 5.3.0-weekly.276 (#7294) * Update mimir-distributed chart to 5.2.0-weekly.276 * Update version to 5.3.0 Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> --------- Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> Co-authored-by: grafanabot <grafanabot@grafana.com> Co-authored-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> * Open circuit breakers on timeouts and per-instance limit errors only (#7310) * Open circuit breakers on timeouts and per-instance limit errors only Signed-off-by: Yuri Nikolic <durica.nikolic@grafana.com> * Update CHANGELOG Signed-off-by: Yuri Nikolic <durica.nikolic@grafana.com> --------- Signed-off-by: Yuri Nikolic <durica.nikolic@grafana.com> * Get rid of iterators.chunkIterator and iterators.chunkMergeIterator (#7313) * Get rid of iterators.chunkIterator and iterators.chunkMergeIterator Signed-off-by: Yuri Nikolic <durica.nikolic@grafana.com> * Fixing review findings Signed-off-by: Yuri Nikolic <durica.nikolic@grafana.com> --------- Signed-off-by: Yuri Nikolic <durica.nikolic@grafana.com> * Compactor: Language fixes (#7315) Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Do not register compat metrics in mimirtool (#7314) This commit does the same as we did for amtool in prometheus/alertmanager#3713. There is no need to register these metrics, so we use compat.NewMetrics(nil) instead of compat.RegisteredMetrics. * Compactor: Un-export symbols that don't need to be exported (#7317) Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Circuit breakers: add client.ErrCircuitBreakerOpen type (#7324) Signed-off-by: Yuri Nikolic <durica.nikolic@grafana.com> * Add mimirpb.CIRCUIT_BREAKER_OPEN error cause (#7330) * Add mimirpb.CIRCUIT_BREAKER_OPEN error cause Signed-off-by: Yuri Nikolic <durica.nikolic@grafana.com> * Fixing review findings Signed-off-by: Yuri Nikolic <durica.nikolic@grafana.com> --------- Signed-off-by: Yuri Nikolic <durica.nikolic@grafana.com> * store-gateway: remove cortex_bucket_store_blocks_loaded_by_duration (#7309) * store-gateway: remove cortex_bucket_store_blocks_loaded_by_duration The PR which added this metric 6074, the motivation was to help detect compactors which are falling behind. There is another metric `cortex_bucket_store_series_blocks_queried` added by jhalterman in 7112 which serves better the purpose of detecting uncompacted blocks. The reason is that it has an explicit label for uncompacted blocks, and it also is more sensitive to recent blocks (assuming those are queried much more often than old uncompacted blocks). Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> * Add CHANGELOG.md entry Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> * Update CHANGELOG.md Co-authored-by: Nick Pillitteri <56quarters@users.noreply.github.com> * Update code comment Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> --------- Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> Co-authored-by: Nick Pillitteri <56quarters@users.noreply.github.com> * ruler: don't retry on non-retriable error (#7216) * ruler: don't retry on non-retriable error Fix 6609 Signed-off-by: Vladimir Varankin <vladimir.varankin@grafana.com> * Update CHANGELOG.md Co-authored-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> --------- Signed-off-by: Vladimir Varankin <vladimir.varankin@grafana.com> Co-authored-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> * Update Alertmanager to f69a508 (#7332) * Update Alertmanager to f69a508 This commit updates Alertmanager to f69a508. The compat package metrics are removed, and the tests are updated to check that logs are emitted instead. * Fix lint * Fix lint again * Use strings.Trim instead of subslice * Remove integration test TestAlertmanagerMatchersMetrics * Fix lint * Prevent configurations with WebhookURLFile This commit updates the validation rules to prevent Alertmanager configurations for Discord and Microsoft Teams from using WebhookURLFile. * Add additional checks and tests for HTTPConfig * Fix missing nil check * Check for embedded HTTPConfig is not needed * Helm: add ruler specific service account (#7132) * Helm: add ruler specific service account Signed-off-by: QuantumEnigmaa <thibaud@giantswarm.io> * add suggestions from review Signed-off-by: QuantumEnigmaa <thibaud@giantswarm.io> * disable ruler sa by default * add rolebinding to ruler sa Signed-off-by: QuantumEnigmaa <thibaud@giantswarm.io> * remove trailing space Signed-off-by: QuantumEnigmaa <thibaud@giantswarm.io> * update handling of ruler sa name Signed-off-by: QuantumEnigmaa <thibaud@giantswarm.io> * add doc comment for ruler sa name Signed-off-by: QuantumEnigmaa <thibaud@giantswarm.io> --------- Signed-off-by: QuantumEnigmaa <thibaud@giantswarm.io> * frontend/transport: log non-2xx replies from downstream as non-successful (#7296) Signed-off-by: Vladimir Varankin <vladimir.varankin@grafana.com> * querymiddleware: Pool snappy writer in shard activity series (#7308) * querymiddleware: pool snappy writer in shard activity series Signed-off-by: Vladimir Varankin <vladimir.varankin@grafana.com> * test concurrent requests in shard active series --------- Signed-off-by: Vladimir Varankin <vladimir.varankin@grafana.com> * Helm: make PSP configurable (#7190) * Helm: make PSP configurable Signed-off-by: QuantumEnigmaa <thibaud@giantswarm.io> * fix changelog Signed-off-by: QuantumEnigmaa <thibaud@giantswarm.io> * fix psp rendering Signed-off-by: QuantumEnigmaa <thibaud@giantswarm.io> * Update operations/helm/charts/mimir-distributed/CHANGELOG.md --------- Signed-off-by: QuantumEnigmaa <thibaud@giantswarm.io> Co-authored-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> * Helm - Templatable host for gateway ingress/route (#7218) * feat: added host templating for gateway ingress * feat: added host templating for gateway route * fix: missing ctx ref * Revert "fix: missing ctx ref" This reverts commit cf956fb. * fix: missing ctx ref * ci: ran make build-helm-tests * updated CHANGELOG.md * fix: re-ordered changelog * feat: added templating to global ingress * fix: added templating to host rules * docs: values.yaml examples * test: rebuild tests according to examples --------- Co-authored-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> * [Docs] Update migrate-from-single-zone-with-helm.md (#7327) * Update migrate-from-single-zone-with-helm.md Fix typo beaching->breaching. * Update changelog * Always sort labels in distributors (#7326) * Always sort labels in distributors Previously, disabling metric relabelling would also disable label sorting. Fixes #7320 * Update comment about sorting * Update CHANGELOG * Do not check for ingester ring state before creating TSDB, or compacting / shipping blocks (#7322) * Do not check for ingester ring state before creating TSDB, or compacting / shipping blocks Signed-off-by: Marco Pracucci <marco@pracucci.com> * Removed unused param Signed-off-by: Marco Pracucci <marco@pracucci.com> --------- Signed-off-by: Marco Pracucci <marco@pracucci.com> * Compactor: String format compaction plan as comma separated blocks (#7321) * Compactor: Format directories as comma separated string Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Compactor: string format plan as comma separated blocks Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Harmonize log message field names Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add a lifetime manager for Vault authentication tokens (#7337) * Initial renewal implementation * Add metrics and integration test to test token renewal * remove unneeeded comment * Update CHANGELOG.md * Use promauto * address comments * Use global const for Vault image * Add failure metric * Update metrics * Update CHANGELOG and mark as experimental * include metric names in about-versioning * fix(deps): update github.com/grafana/dskit digest to f245b48 (#7283) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Packaging: remove reload from systemd file as mimir does not take into account SIGHUP (#7345) Signed-off-by: Wilfried Roset <wilfriedroset@users.noreply.github.com> * Docs: No longer mark OTLP endpoint as experimental (#7348) Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Update golang.org/x/exp digest to 2c58cdc (#7352) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update module github.com/aws/aws-sdk-go to v1.50.15 (#7353) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update module github.com/minio/minio-go/v7 to v7.0.67 (#7354) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update dependency puppeteer to v21.11.0 (#7355) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update helm/kind-action action to v1.9.0 (#7357) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update module cloud.google.com/go/storage to v1.37.0 (#7358) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Jsonnet / Helm: improve distributors graceful shutdown (#7361) * Jsonnet / Helm: improve distributors graceful shutdown Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fix Helm distributor termination grace period Signed-off-by: Marco Pracucci <marco@pracucci.com> --------- Signed-off-by: Marco Pracucci <marco@pracucci.com> * Release mimir-distributed Helm chart 5.3.0-weekly.277 (#7362) Co-authored-by: grafanabot <grafanabot@grafana.com> * Distributor: Make `-distributor.enable-otlp-metadata-storage` flag default to true, and deprecate (#7366) * Distributor: Change -distributor.enable-otlp-metadata-storage flag's default to true and deprecate --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Mark -ingester.limit-inflight-requests-using-grpc-method-limiter and -distributor.limit-inflight-requests-using-grpc-method-limiter as stable and enable it by default (#7360) * Mark -ingester.limit-inflight-requests-using-grpc-method-limiter and -distributor.limit-inflight-requests-using-grpc-method-limiter as stable and enable it by default Signed-off-by: Marco Pracucci <marco@pracucci.com> * Improved tests Signed-off-by: Marco Pracucci <marco@pracucci.com> * Improved tests Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed unit tests Signed-off-by: Marco Pracucci <marco@pracucci.com> * Mark config as deprecated Signed-off-by: Marco Pracucci <marco@pracucci.com> --------- Signed-off-by: Marco Pracucci <marco@pracucci.com> * Do not consider out-of-order blocks when filtering compactable jobs (#7342) * Do not consider out-of-order blocks when filtering compactable jobs Co-authored-by: Marco Pracucci <marco@pracucci.com> * mimir: Inject span profiler into tracer (#7363) * mimir: inject span profiler into tracer Signed-off-by: Vladimir Varankin <vladimir.varankin@grafana.com> * Update changelog Signed-off-by: Vladimir Varankin <vladimir.varankin@grafana.com> --------- Signed-off-by: Vladimir Varankin <vladimir.varankin@grafana.com> * Add experimental partitions ring lifecycler support (#7349) * Add experimental partitions ring lifecycler support Signed-off-by: Marco Pracucci <marco@pracucci.com> Co-authored-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> Co-authored-by: Peter Štibraný <pstibrany@gmail.com> * Use http.Error() Signed-off-by: Marco Pracucci <marco@pracucci.com> * Skip deprecated linter check for code that will be soon replaced Signed-off-by: Marco Pracucci <marco@pracucci.com> * Updated dskit Signed-off-by: Marco Pracucci <marco@pracucci.com> * Add partition id to logs Signed-off-by: Marco Pracucci <marco@pracucci.com> * Added comment Signed-off-by: Marco Pracucci <marco@pracucci.com> * Added TestIngester_ShouldNotCreatePartitionIfThereIsShutdownMarker Signed-off-by: Marco Pracucci <marco@pracucci.com> --------- Signed-off-by: Marco Pracucci <marco@pracucci.com> Co-authored-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> Co-authored-by: Peter Štibraný <pstibrany@gmail.com> * feat(helm): Adding KEDA autoscaling support * chore: rebase branch with main * chore: make build-helm-tests --------- Signed-off-by: Vladimir Varankin <vladimir.varankin@grafana.com> Signed-off-by: Peter Štibraný <pstibrany@gmail.com> Signed-off-by: Marco Pracucci <marco@pracucci.com> Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com> Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> Signed-off-by: Yuri Nikolic <durica.nikolic@grafana.com> Signed-off-by: QuantumEnigmaa <thibaud@giantswarm.io> Signed-off-by: Wilfried Roset <wilfriedroset@users.noreply.github.com> Co-authored-by: Vladimir Varankin <vladimir.varankin@grafana.com> Co-authored-by: Marco Pracucci <marco@pracucci.com> Co-authored-by: Andy Asp <90626759+andyasp@users.noreply.github.com> Co-authored-by: Peter Štibraný <pstibrany@gmail.com> Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Nick Pillitteri <56quarters@users.noreply.github.com> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Grot (@grafanabot) <43478413+grafanabot@users.noreply.github.com> Co-authored-by: grafanabot <grafanabot@grafana.com> Co-authored-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> Co-authored-by: Đurica Yuri Nikolić <durica.nikolic@grafana.com> Co-authored-by: George Robinson <george.robinson@grafana.com> Co-authored-by: Zirko <64951262+QuantumEnigmaa@users.noreply.github.com> Co-authored-by: Itay Kalfon <82172131+Itaykal@users.noreply.github.com> Co-authored-by: Éamon Ryan <eamonryan@users.noreply.github.com> Co-authored-by: Patrick Oyarzun <patrick.oyarzun@grafana.com> Co-authored-by: Fayzal Ghantiwala <114010985+fayzal-g@users.noreply.github.com> Co-authored-by: Wilfried ROSET <wilfriedroset@users.noreply.github.com> Co-authored-by: Jonathan Halterman <jonathan@grafana.com>

pracucci changed the title ~~Jsonnet: improve querier HPA~~ Jsonnet: improve querier HPA to scale up and down more gradually Dec 20, 2023

pracucci marked this pull request as ready for review December 20, 2023 10:44

pracucci requested a review from a team as a code owner December 20, 2023 10:44

dimitarvdimitrov approved these changes Dec 21, 2023

View reviewed changes

pracucci added 4 commits December 21, 2023 12:23

Jsonnet: improve querier HPA

ad542a9

Signed-off-by: Marco Pracucci <marco@pracucci.com>

Fixed query

9b98eca

Signed-off-by: Marco Pracucci <marco@pracucci.com>

Added CHANGELOG entry

67a7d5f

Signed-off-by: Marco Pracucci <marco@pracucci.com>

Addressed review comments

236cc5c

Signed-off-by: Marco Pracucci <marco@pracucci.com>

pracucci force-pushed the improve-querier-hpa branch from c7b2a50 to 236cc5c Compare December 21, 2023 11:27

pracucci enabled auto-merge (squash) December 21, 2023 11:35

pracucci merged commit be1f450 into main Dec 21, 2023
28 checks passed

pracucci deleted the improve-querier-hpa branch December 21, 2023 11:40

dimitarvdimitrov mentioned this pull request Jan 8, 2024

feat(helm): add keda autoscaling and fix dashboards #4687

Closed

3 tasks

beatkind added a commit to beatkind/mimir that referenced this pull request Feb 3, 2024

feat(helm): Porting the changes from grafana#6971 into helm chart

ce04869

beatkind mentioned this pull request Feb 3, 2024

feat(helm): Adding KEDA autoscaling support #7282

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Jsonnet: improve querier HPA to scale up and down more gradually #6971

Jsonnet: improve querier HPA to scale up and down more gradually #6971

pracucci commented Dec 20, 2023 •

edited

Loading

dimitarvdimitrov left a comment

dimitarvdimitrov Dec 21, 2023

pracucci Dec 21, 2023

pracucci Dec 21, 2023

Jsonnet: improve querier HPA to scale up and down more gradually #6971

Jsonnet: improve querier HPA to scale up and down more gradually #6971

Conversation

pracucci commented Dec 20, 2023 • edited Loading

What this PR does

Which issue(s) this PR fixes or relates to

Checklist

dimitarvdimitrov left a comment

Choose a reason for hiding this comment

dimitarvdimitrov Dec 21, 2023

Choose a reason for hiding this comment

pracucci Dec 21, 2023

Choose a reason for hiding this comment

pracucci Dec 21, 2023

Choose a reason for hiding this comment

pracucci commented Dec 20, 2023 •

edited

Loading