Merged Experimental Federated Rules API feature branch to master. #2200

bwplotka · 2020-02-28T14:26:33Z

Branch for Federated Rule API with @s-urbaniak based on following design

pkg/store/storepb/rpc.proto

stale · 2020-04-18T12:51:24Z

This issue/PR has been automatically marked as stale because it has not had recent activity. Please comment on status otherwise the issue will be closed in a week. Thank you for your contributions.

bwplotka · 2020-05-21T15:08:23Z

Ok, here we go!

We are ready to merge rules to master!

All flags are hidden for now.

The only missing bit is improved deduplication which can be done separately to the master: #2555

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* pkg/store/storepb: fix wrong rule reference Currently we recursively reference rules instead of recording rules. Signed-off-by: Sergiusz Urbaniak <sergiusz.urbaniak@gmail.com> * proto: regenerate Signed-off-by: Sergiusz Urbaniak <sergiusz.urbaniak@gmail.com>

…oto). Added tests. (#2242) Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

bwplotka · 2020-05-22T08:00:50Z

Ok I hate master pulls, doing full rebase, fasten the belt!

…romclient; Added rulesAPI RPC to sidecar. (#2243) * Use proto rules API instead of struct; Added rulesAPI RPC to sidecar. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed broken test. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Use proto rules API instead of struct; Moved as much as possible to promclient; Added rulesAPI RPC to sidecar. (#2291) * rules_custom_test: fix asserting labels Signed-off-by: Sergiusz Urbaniak <sergiusz.urbaniak@gmail.com> * TestPrometheusStore_Rules_e2e: fix test fixture Signed-off-by: Sergiusz Urbaniak <sergiusz.urbaniak@gmail.com> Co-authored-by: Sergiusz Urbaniak <sergiusz.urbaniak@gmail.com>

* cmd/thanos/query: add initial rules support Signed-off-by: Sergiusz Urbaniak <sergiusz.urbaniak@gmail.com> * pkg/query/api/v1: initial implementation Signed-off-by: Sergiusz Urbaniak <sergiusz.urbaniak@gmail.com> * e2e: initial implementation and fixes Signed-off-by: Sergiusz Urbaniak <sergiusz.urbaniak@gmail.com> * pkg/query: fix racy access to assert rules API store Signed-off-by: Sergiusz Urbaniak <sergiusz.urbaniak@gmail.com>

* Refactored proto generation and separate store from rules APIs. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Addressed comments. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed proto gen. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Addressed Serg comments. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

As per: https://thanos.io/proposals/202003_thanos_rules_federation.md/ Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

bwplotka · 2020-05-22T09:16:43Z

Now it should be good to go, just probably tests to fix. 💪

PTAL @povilasv @GiedriusS @s-urbaniak

ALL of the commits were reviewed and approved before, just last one + rebase could mess something up (:

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

bwplotka · 2020-05-23T07:21:55Z

(:

s-urbaniak · 2020-05-25T06:29:03Z

yay, thank you! 🎉
lgtm from my side ;-)

brancz

Going to go ahead and merge this as previously things had already been reviewed and merged into the feature branch, and the functionality is hidden for now.

This bug was already fixed in thanos-io#2615 but it got lost when we merged thanos-io#2200. Co-authored-by: johncming <johncming@yahoo.com> Co-authored-by: Lili Cosic <cosiclili@gmail.com> Signed-off-by: Simon Pasquier <spasquie@redhat.com>

This bug was already fixed in #2615 but it got lost when we merged #2200. Co-authored-by: johncming <johncming@yahoo.com> Co-authored-by: Lili Cosic <cosiclili@gmail.com> Signed-off-by: Simon Pasquier <spasquie@redhat.com> Co-authored-by: johncming <johncming@yahoo.com> Co-authored-by: Lili Cosic <cosiclili@gmail.com>

This bug was already fixed in thanos-io#2615 but it got lost when we merged thanos-io#2200. Co-authored-by: johncming <johncming@yahoo.com> Co-authored-by: Lili Cosic <cosiclili@gmail.com> Signed-off-by: Simon Pasquier <spasquie@redhat.com> Co-authored-by: johncming <johncming@yahoo.com> Co-authored-by: Lili Cosic <cosiclili@gmail.com>

This bug was already fixed in thanos-io#2615 but it got lost when we merged thanos-io#2200. Co-authored-by: johncming <johncming@yahoo.com> Co-authored-by: Lili Cosic <cosiclili@gmail.com> Signed-off-by: Simon Pasquier <spasquie@redhat.com> Co-authored-by: johncming <johncming@yahoo.com> Co-authored-by: Lili Cosic <cosiclili@gmail.com> Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

* Made sure old sse S3 option if specified, produces error. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Cut v0.15.0-rc.0 Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * tsdbstore: Optimized response framing if iterator finished. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Apply suggestions from code review Co-authored-by: Krasimir Georgiev <8903888+krasi-georgiev@users.noreply.github.com> Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Rule: update manager when all rule files are removed (#3095) This bug was already fixed in #2615 but it got lost when we merged #2200. Co-authored-by: johncming <johncming@yahoo.com> Co-authored-by: Lili Cosic <cosiclili@gmail.com> Signed-off-by: Simon Pasquier <spasquie@redhat.com> Co-authored-by: johncming <johncming@yahoo.com> Co-authored-by: Lili Cosic <cosiclili@gmail.com> Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * ui: Fix Block Viewer for Compactor and Store (#3098) * ui: Fix Block viewer for Compactor and Store Signed-off-by: Prem Kumar <prmsrswt@gmail.com> * Add an entry in CHANGELOG.md Signed-off-by: Prem Kumar <prmsrswt@gmail.com> * ui: react: Fixed tests for Blocks Signed-off-by: Prem Kumar <prmsrswt@gmail.com> Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * querier: Fix overwriting maxSourceResolution when auto downsampling is enabled (#3105) * Fix overwriting maxSourceResolution when auto downsampling is enabled Signed-off-by: Ben Ye <yb532204897@gmail.com> * add changelog Signed-off-by: Ben Ye <yb532204897@gmail.com> Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Author: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com> Date: Thu Aug 27 14:10:19 2020 +0300 (#3010) Make the PromQL lookBack configurable with a flag Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com> Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Cut release v0.15.0-rc.1 (#3103) * Cut release v0.15.0-rc.1 Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Address review issues Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Fix changelog Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Cut release v0.15.0 (#3129) Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Minor fix Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Address review comments Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Co-authored-by: Krasimir Georgiev <8903888+krasi-georgiev@users.noreply.github.com> Co-authored-by: Simon Pasquier <spasquie@redhat.com> Co-authored-by: johncming <johncming@yahoo.com> Co-authored-by: Lili Cosic <cosiclili@gmail.com> Co-authored-by: Prem Kumar <prmsrswt@gmail.com> Co-authored-by: Ben Ye <yb532204897@gmail.com>

* Made sure old sse S3 option if specified, produces error. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Cut v0.15.0-rc.0 Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * tsdbstore: Optimized response framing if iterator finished. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Apply suggestions from code review Co-authored-by: Krasimir Georgiev <8903888+krasi-georgiev@users.noreply.github.com> Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Rule: update manager when all rule files are removed (#3095) This bug was already fixed in thanos-io/thanos#2615 but it got lost when we merged thanos-io/thanos#2200. Co-authored-by: johncming <johncming@yahoo.com> Co-authored-by: Lili Cosic <cosiclili@gmail.com> Signed-off-by: Simon Pasquier <spasquie@redhat.com> Co-authored-by: johncming <johncming@yahoo.com> Co-authored-by: Lili Cosic <cosiclili@gmail.com> Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * ui: Fix Block Viewer for Compactor and Store (#3098) * ui: Fix Block viewer for Compactor and Store Signed-off-by: Prem Kumar <prmsrswt@gmail.com> * Add an entry in CHANGELOG.md Signed-off-by: Prem Kumar <prmsrswt@gmail.com> * ui: react: Fixed tests for Blocks Signed-off-by: Prem Kumar <prmsrswt@gmail.com> Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * querier: Fix overwriting maxSourceResolution when auto downsampling is enabled (#3105) * Fix overwriting maxSourceResolution when auto downsampling is enabled Signed-off-by: Ben Ye <yb532204897@gmail.com> * add changelog Signed-off-by: Ben Ye <yb532204897@gmail.com> Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Author: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com> Date: Thu Aug 27 14:10:19 2020 +0300 (#3010) Make the PromQL lookBack configurable with a flag Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com> Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Cut release v0.15.0-rc.1 (#3103) * Cut release v0.15.0-rc.1 Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Address review issues Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Fix changelog Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Cut release v0.15.0 (#3129) Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Minor fix Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Address review comments Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Co-authored-by: Krasimir Georgiev <8903888+krasi-georgiev@users.noreply.github.com> Co-authored-by: Simon Pasquier <spasquie@redhat.com> Co-authored-by: johncming <johncming@yahoo.com> Co-authored-by: Lili Cosic <cosiclili@gmail.com> Co-authored-by: Prem Kumar <prmsrswt@gmail.com> Co-authored-by: Ben Ye <yb532204897@gmail.com>

bwplotka mentioned this pull request Feb 28, 2020

Action Plan for Thanos User Interface and UI related APIs Improvements #2198

Closed

13 tasks

bwplotka commented Feb 28, 2020

View reviewed changes

pkg/store/storepb/rpc.proto Outdated Show resolved Hide resolved

bwplotka commented Mar 2, 2020

View reviewed changes

pkg/store/storepb/rpc.proto Outdated Show resolved Hide resolved

bwplotka mentioned this pull request Mar 9, 2020

Reduce number of Alerts/Rules boilerplate code structs. prometheus/prometheus#6948

Closed

GiedriusS reviewed Mar 11, 2020

View reviewed changes

pkg/store/storepb/rpc.proto Outdated Show resolved Hide resolved

bwplotka mentioned this pull request Mar 11, 2020

querier: Federated Thanos Targets Page #1375

Closed

s-urbaniak mentioned this pull request Mar 20, 2020

docs/proposals/202003_thanos_rules_federation: initial commit #2263

Merged

stale bot added the stale label Apr 18, 2020

stale bot closed this Apr 25, 2020

bwplotka reopened this Apr 27, 2020

stale bot removed the stale label Apr 27, 2020

bwplotka changed the title ~~feature branch: Federated Rules API via RuleAPI~~ Merged Experimental Federated Rules API feature branch to master. May 21, 2020

bwplotka marked this pull request as ready for review May 21, 2020 15:08

bwplotka requested review from kakkoyun, squat, brancz, FUSAKLA, pracucci and yeya24 May 21, 2020 15:08

bwplotka and others added 5 commits May 21, 2020 16:38

Added RulesAPI.

443fef7

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Added warnings.

2b0b308

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Added Type to rules requests as it is on HTTP API. (#2201)

62c0d79

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Made storepb.RuleGroups a source of truth for rules API (Go, JSON, pr…

13769f0

…oto). Added tests. (#2242) Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

bwplotka and others added 3 commits May 22, 2020 09:01

bwplotka added 2 commits May 22, 2020 10:01

Added Ruler support for RulesAPI; Refactored Manager. (#2562)

87bcc73

As per: https://thanos.io/proposals/202003_thanos_rules_federation.md/ Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Small fixes to changelog and flags. Do not add any.

50bc8dd

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

bwplotka force-pushed the features/rules-proxy branch from 42518a1 to d321dda Compare May 22, 2020 09:14

bwplotka force-pushed the features/rules-proxy branch 3 times, most recently from 714ac05 to 5f29313 Compare May 22, 2020 15:55

Fixed after rebase.

8191122

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

bwplotka force-pushed the features/rules-proxy branch from fe0fac4 to 8191122 Compare May 22, 2020 17:10

brancz approved these changes May 25, 2020

View reviewed changes

brancz merged commit c733564 into master May 25, 2020

simonpasquier mentioned this pull request Aug 28, 2020

Rule: update manager when all rule files are removed #3095

Merged

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Merged Experimental Federated Rules API feature branch to master. #2200

Merged Experimental Federated Rules API feature branch to master. #2200

bwplotka commented Feb 28, 2020 •

edited

Loading

stale bot commented Apr 18, 2020

bwplotka commented May 21, 2020

bwplotka commented May 22, 2020 •

edited

Loading

bwplotka commented May 22, 2020 •

edited

Loading

bwplotka commented May 23, 2020

s-urbaniak commented May 25, 2020

brancz left a comment

Merged Experimental Federated Rules API feature branch to master. #2200

Merged Experimental Federated Rules API feature branch to master. #2200

Conversation

bwplotka commented Feb 28, 2020 • edited Loading

stale bot commented Apr 18, 2020

bwplotka commented May 21, 2020

bwplotka commented May 22, 2020 • edited Loading

bwplotka commented May 22, 2020 • edited Loading

bwplotka commented May 23, 2020

s-urbaniak commented May 25, 2020

brancz left a comment

Choose a reason for hiding this comment

bwplotka commented Feb 28, 2020 •

edited

Loading

bwplotka commented May 22, 2020 •

edited

Loading

bwplotka commented May 22, 2020 •

edited

Loading