Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

query:thanos-query adaptive downsample by max_source_resolution #4526

Closed
wants to merge 0 commits into from

Conversation

t00350320
Copy link

  • I added CHANGELOG entry for this change.
  • Change is not relevant to the end user.

Changes

here is a case: when user don't know how thanos downsample datas comes such as [raw,5m,1h]. And they may use a not suitable duration parameter [15s] like:
rate(foo[15s]& max_source_resolution=1h,
the result may be empty or only a little raw data of latest time. because the promql funcrate must need two datas accross duration time, max_source_resolution=1h, means there is only one data in 1h, the final result of rate will fail or loss data.

So , we add an adaptive method, it check the real duration time, and compared with max_source_resolution, if less than max_so.., the duration time will be changed to a suitable value.

Verification

in our systerm, we test it by set :
a: rate(foo[15s]&start1627913400&end=1628086200&step=600&max_source_resolution=1h,
b:rate(foo[15s]&start1627913400&end=1628086200&step=600&max_source_resolution=5m,

a->a1:rate(foo[2h]&start1627913400&end=1628086200&step=600&max_source_resolution=1h,
b->b1:rate(foo[10m]&start1627913400&end=1628086200&step=600&max_source_resolution=5m,
the result datas of a1 and b1 are in expectation than previous version of thanos-query.

Copy link
Member

@bwplotka bwplotka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this!

Unfortunately, this is by design. You want to control your vector range. Normally you don't want something to change your vector range suddenly - this may surprise everyone: Suddenly you do rate over 1h not 5m. If you do increase that changes "units" kind of too.

Also, imagine you don't have downsampling but you have just long scrape interval (e.g 5m) and then you do 4m - you will also hit same issue. Also what about dynamic resolution.

What about something different: It would be nice to return error in such cases potentially (why not) if max resolution is set and unrealistic vector range is set. WDYT?

@t00350320
Copy link
Author

t00350320 commented Aug 6, 2021 via email

@t00350320
Copy link
Author

hi @bwplotka , i closed this request and submit a new one
#4529.
thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants