Why is the number of records returned when calculating rate different?

[vinhthien1996]

I have 2 input data pushed to openTSDB

• For Input 1
  8/27/2024 2:56:25 1724727385 1326137
  8/27/2024 3:11:25 1724728285 1326345
  8/27/2024 3:26:25 1724729185 1326552
  8/27/2024 3:41:25 1724730085 1326759
  8/27/2024 3:56:25 1724730985 1326967
  8/27/2024 4:11:25 1724731885 1327175
  8/27/2024 4:26:25 1724732785 1327382
• query 3:10 -> 4:10:
  8/27/2024 3:11:20 1724728280 1326345
  8/27/2024 3:26:20 1724729180 1326552
  8/27/2024 3:41:20 1724730080 1326759
  8/27/2024 3:56:20 1724730980 1326967
• rate 3:10 -> 4:10:
  8/27/2024 3:26:20 1724729180 0.23
  8/27/2024 3:41:20 1724730080 0.23
  8/27/2024 3:56:20 1724730980 0.231111111

When querying 3:10 - > 4:10, the results from the original 4 records, after calculating the rate, there will be 3 records left, this is reasonable because openTSDB's rate calculation mechanism is (Vn - Vn-1) / ( Tn - Tn-1), so the first point has no previous point, so 4 -> 3 records:

• But for input 2:
  8/27/2024 2:45:44 1724726744 8.23952E+11
  8/27/2024 3:00:44 1724727644 8.23952E+11
  8/27/2024 3:15:44 1724728544 8.23952E+11
  8/27/2024 3:30:44 1724729444 8.23952E+11
  8/27/2024 3:45:44 1724730344 8.23952E+11
  8/27/2024 4:00:44 1724731244 8.23952E+11
  8/27/2024 4:15:44 1724732144 8.23952E+11
• query 3:10 -> 4:10:
  8/27/2024 3:15:40 1724728540 8.23952E+11
  8/27/2024 3:30:40 1724729440 8.23952E+11
  8/27/2024 3:45:40 1724730340 8.23952E+11
  8/27/2024 4:00:40 1724731240 8.23952E+11
• rate 3:10 -> 4:10:
  8/27/2024 3:15:40 1724728540 73.49333333
  8/27/2024 3:30:40 1724729440 73.14
  8/27/2024 3:45:40 1724730340 73.14
  8/27/2024 4:00:40 1724731240 73.14

When querying 3:10 - > 4:10, the results are from the original 4 records, after calculating the rate, the results are still 4 records.

According to the data I provided, I can understand that, if the query rate is 3:10 -> 4:10, openTSDB can expand the range from 3:10 to 3:00, so input 2 has from 3:00 -> 4:10 will have 5 points -> rate -> 4 points. And input 1, 3:00 -> 4:10 will still only have 4 points -> rate -> 3 points.
Am I understanding correctly?

[johann8384]

It has been a while but there are two things at play here from what I remember: 1) HBase returns extra data before and after the requested time rounded to the blocks, hourly iirc 2) rate also requests extra data if needed as it requires at least one or two data points before the start in order to do the rate calculation. Rate also calculates based on things like detecting a rollover and adding that to the previous value, etc. These factors can sometimes render unexpected (but in my opinion most correct) data.

…

On Tue, Aug 27, 2024, 1:37 PM vinhthien1996 ***@***.***> wrote: I have 2 input data pushed to openTSDB - For Input 1 8/27/2024 2:56:25 1724727385 1326137 8/27/2024 3:11:25 1724728285 1326345 8/27/2024 3:26:25 1724729185 1326552 8/27/2024 3:41:25 1724730085 1326759 8/27/2024 3:56:25 1724730985 1326967 8/27/2024 4:11:25 1724731885 1327175 8/27/2024 4:26:25 1724732785 1327382 - query 3:10 -> 4:10: 8/27/2024 3:11:20 1724728280 1326345 8/27/2024 3:26:20 1724729180 1326552 8/27/2024 3:41:20 1724730080 1326759 8/27/2024 3:56:20 1724730980 1326967 - rate 3:10 -> 4:10: 8/27/2024 3:26:20 1724729180 0.23 8/27/2024 3:41:20 1724730080 0.23 8/27/2024 3:56:20 1724730980 0.231111111 When querying 3:10 - > 4:10, the results from the original 4 records, after calculating the rate, there will be 3 records left, this is reasonable because openTSDB's rate calculation mechanism is (Vn - Vn-1) / ( Tn - Tn-1), so the first point has no previous point, so 4 -> 3 records: But for input 2: 8/27/2024 2:45:44 1724726744 8.23952E+11 8/27/2024 3:00:44 1724727644 8.23952E+11 8/27/2024 3:15:44 1724728544 8.23952E+11 8/27/2024 3:30:44 1724729444 8.23952E+11 8/27/2024 3:45:44 1724730344 8.23952E+11 8/27/2024 4:00:44 1724731244 8.23952E+11 8/27/2024 4:15:44 1724732144 8.23952E+11 - query 3:10 -> 4:10: 8/27/2024 3:15:40 1724728540 8.23952E+11 8/27/2024 3:30:40 1724729440 8.23952E+11 8/27/2024 3:45:40 1724730340 8.23952E+11 8/27/2024 4:00:40 1724731240 8.23952E+11 - rate 3:10 -> 4:10: 8/27/2024 3:15:40 1724728540 73.49333333 8/27/2024 3:30:40 1724729440 73.14 8/27/2024 3:45:40 1724730340 73.14 8/27/2024 4:00:40 1724731240 73.14 When querying 3:10 - > 4:10, the results are from the original 4 records, after calculating the rate, the results are still 4 records. According to the data I provided, I can understand that, if the query rate is 3:10 -> 4:10, openTSDB can expand the range from 3:10 to 3:00, so input 2 has from 3:00 -> 4:10 will have 5 points -> rate -> 4 points. And input 1, 3:00 -> 4:10 will still only have 4 points -> rate -> 3 points. Am I understanding correctly? — Reply to this email directly, view it on GitHub <#2296>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABFYMIQ7BIKWIU4PSOLR6TZTTBOFAVCNFSM6AAAAABNGWWL4OVHI2DSMVQWIX3LMV43ASLTON2WKOZSGQ4TAMBXHE2TINA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>