[new feature] logql: extrapolate unwrapped rate function #5013

liguozhong · 2021-12-29T07:22:49Z

ref issue:#1136
doc: https://grafana.com/docs/loki/latest/logql/metric_queries/
rate function doc: rate(unwrapped-range): calculates per second rate of all values in the specified interval.

logql: rate({_source="vector-internal-metrics" }|json|name="component_received_event_bytes_total"|tags_component_id="service_metrics"|unwrap counter_value [1m])

qps = 20,000 /s

liguozhong · 2021-12-29T14:04:37Z

@cyriltovena review pr please

cyriltovena · 2022-01-03T14:41:55Z

pkg/logql/functions.go

+// It calculates the rate (allowing for counter resets if isCounter is true),
+// extrapolates if the first/last sample is close to the boundary, and returns
+// the result as either per-second (if isRate is true) or overall.
+func extrapolatedRate(samples []promql.Point, selRange time.Duration, isCounter, isRate bool) float64 {


Could you please add a comment that this is taken from prometheus code base and where, this can be useful for everyone.

done ，thanks

cyriltovena · 2022-01-03T14:48:32Z

pkg/logql/engine_test.go

+			[]SelectSampleParams{
+				{&logproto.SampleQueryRequest{Start: time.Unix(30, 0), End: time.Unix(60, 0), Selector: `rate({app="foo"} | unwrap foo[30s])`}},
+			},
+			promql.Vector{promql.Sample{Point: promql.Point{T: 60 * 1000, V: 0.0}, Metric: labels.Labels{labels.Label{Name: "app", Value: "foo"}}}},


it would be better if we could have an example that returns a positive value.

done，thanks！

cyriltovena

Looking good, left just 2 small comments PTAL

cyriltovena

LGTM

cyriltovena · 2022-01-04T07:47:37Z

It might be worth adding a note of this behaviour in the doc,

This PR reverts the implementation done in #5013 to the original implementation that sums the extracted values from the log lines instead of treating them like a Prometheus counter metric. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>

`rate_counter()` takes an unwrapped range as input, but unlike `rate()` it treats the values extracted from the log lines as "counter metric" like in Prometheus' `rate()` function. This is a replacement for the reverted change of #5013 Signed-off-by: Christian Haudum <christian.haudum@gmail.com>

This PR reverts the implementation done in #5013 to the original implementation that sums the extracted values from the log lines instead of treating them like a Prometheus counter metric. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>

`rate_counter()` takes an unwrapped range as input, but unlike `rate()` it treats the values extracted from the log lines as "counter metric" like in Prometheus' `rate()` function. This is a replacement for the reverted change of #5013 Signed-off-by: Christian Haudum <christian.haudum@gmail.com>

This PR reverts the implementation done in #5013 to the original implementation that sums the extracted values from the log lines instead of treating them like a Prometheus counter metric. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>

`rate_counter()` takes an unwrapped range as input, but unlike `rate()` it treats the values extracted from the log lines as "counter metric" like in Prometheus' `rate()` function. This is a replacement for the reverted change of #5013 Signed-off-by: Christian Haudum <christian.haudum@gmail.com>

…counter (#6361) * Revert unwrapped rate aggregation to previous implementation This PR reverts the implementation done in #5013 to the original implementation that sums the extracted values from the log lines instead of treating them like a Prometheus counter metric. Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Move changelog entry Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Remove unused/dead code Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Clean changelog Signed-off-by: Christian Haudum <christian.haudum@gmail.com>

`rate_counter()` takes an unwrapped range as input, but unlike `rate()` it treats the values extracted from the log lines as "counter metric" like in Prometheus' `rate()` function. This is a replacement for the reverted change of #5013 Signed-off-by: Christian Haudum <christian.haudum@gmail.com>

…counter (#6361) * Revert unwrapped rate aggregation to previous implementation This PR reverts the implementation done in #5013 to the original implementation that sums the extracted values from the log lines instead of treating them like a Prometheus counter metric. Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Move changelog entry Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Remove unused/dead code Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Clean changelog Signed-off-by: Christian Haudum <christian.haudum@gmail.com> (cherry picked from commit b315ed0)

…counter (#6361) (#6555) * Revert unwrapped rate aggregation to previous implementation This PR reverts the implementation done in #5013 to the original implementation that sums the extracted values from the log lines instead of treating them like a Prometheus counter metric. Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Move changelog entry Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Remove unused/dead code Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Clean changelog Signed-off-by: Christian Haudum <christian.haudum@gmail.com> (cherry picked from commit b315ed0) Co-authored-by: Christian Haudum <christian.haudum@gmail.com>

`rate_counter()` takes an unwrapped range as input, but unlike `rate()` it treats the values extracted from the log lines as "counter metric" like in Prometheus' `rate()` function. This is a replacement for the reverted change of #5013 Signed-off-by: Christian Haudum <christian.haudum@gmail.com> (cherry picked from commit a56a88b)

* Add unwrapped aggregation `rate_counter()` (#6412) `rate_counter()` takes an unwrapped range as input, but unlike `rate()` it treats the values extracted from the log lines as "counter metric" like in Prometheus' `rate()` function. This is a replacement for the reverted change of #5013 Signed-off-by: Christian Haudum <christian.haudum@gmail.com> (cherry picked from commit a56a88b) * Update CHANGELOG Co-authored-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Susana Ferreira <susana.ferreira@grafana.com>

liguozhong added 2 commits December 27, 2021 17:49

[new feature] logql：reimplement metrics rate function

31757a0

[new feature] logql：reimplement metrics rate function

0328f99

liguozhong requested a review from a team as a code owner December 29, 2021 07:22

pull-request-size bot added the size/L label Dec 29, 2021

[new feature] logql：reimplement metrics rate function

a4a4419

cyriltovena reviewed Jan 3, 2022

View reviewed changes

cyriltovena changed the title ~~[new feature] logql：reimplement metrics rate function~~ [new feature] logql：extrapolate unwrapped rate function Jan 3, 2022

cyriltovena changed the title ~~[new feature] logql：extrapolate unwrapped rate function~~ [new feature] logql: extrapolate unwrapped rate function Jan 3, 2022

cyriltovena reviewed Jan 3, 2022

View reviewed changes

[new feature] logql: extrapolate unwrapped rate function grafana#5013

ef79b03

cyriltovena approved these changes Jan 4, 2022

View reviewed changes

cyriltovena merged commit a66e148 into grafana:main Jan 4, 2022

chaudum mentioned this pull request Jun 9, 2022

[RFC]: Sum values in unwrapped rate aggregation instead of treating them as counter #6351

Open

chaudum mentioned this pull request Jun 10, 2022

Sum values in unwrapped rate aggregation instead of treating them as counter #6361

Merged

4 tasks

chaudum mentioned this pull request Jun 16, 2022

Add unwrapped aggregation rate_counter() #6412

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[new feature] logql: extrapolate unwrapped rate function #5013

[new feature] logql: extrapolate unwrapped rate function #5013

liguozhong commented Dec 29, 2021 •

edited

Loading

liguozhong commented Dec 29, 2021

cyriltovena Jan 3, 2022

liguozhong Jan 4, 2022

cyriltovena Jan 3, 2022 •

edited

Loading

liguozhong Jan 4, 2022

cyriltovena left a comment

cyriltovena left a comment

cyriltovena commented Jan 4, 2022

[new feature] logql: extrapolate unwrapped rate function #5013

[new feature] logql: extrapolate unwrapped rate function #5013

Conversation

liguozhong commented Dec 29, 2021 • edited Loading

liguozhong commented Dec 29, 2021

cyriltovena Jan 3, 2022

Choose a reason for hiding this comment

liguozhong Jan 4, 2022

Choose a reason for hiding this comment

cyriltovena Jan 3, 2022 • edited Loading

Choose a reason for hiding this comment

liguozhong Jan 4, 2022

Choose a reason for hiding this comment

cyriltovena left a comment

Choose a reason for hiding this comment

cyriltovena left a comment

Choose a reason for hiding this comment

cyriltovena commented Jan 4, 2022

liguozhong commented Dec 29, 2021 •

edited

Loading

cyriltovena Jan 3, 2022 •

edited

Loading