something is iffy about the data responses in master

[Dieterbe]

examples:

1) grafana sometimes says data out of range

this was probably due to a bug in normalization, which should be fixed in #1674

2) metrictank versions dashboard

our internal dashboard metrictank-versions?orgId=1&from=now-24h&to=now
has a query like so:
groupByNode(metrictank.stats.*.*.version.*.gauge1, 5, 'sum')

what do we know?

• it does not happen consistently (reminds of mergeSeries() needs runtime normalizing #1673 )
• seems to be caused by some series not having the final point whereas others do
• sometimes when i switch to points mode (not stacked) i can also see that some data is very high res, whereas others is only a few points.

this could possibly be fixed by #1676
I also suspect the current logic which tries to honor max-points-per-req-soft is to blame:
it coarsens certain series but not others, this is possibly why they don't stack well in stacked charts.
=> multiple requests that follow the same retention rule should probably be coarsened together.
the "support different intervals" makes sense when data has different retention rules, not so much when trying to accommodate max-points-per-req-soft on same-retention-rule data.

update: problem 2 is not fixed by #1674 (tested v0.13.1-314-ga3706f6)

[Dieterbe]

We have done some quick tests in Grafana to confirm issue 2 is very likely caused by different series having different intervals, which would be caused by max-points-per-req-soft only effecting some, but not all requests.
Here's a quick test with a high-res, and a low-res series.

When you stack them, the artifact appears when the series with "missing points" / lower resolution is at the bottom.....

but looks fine if the series have a different order.

(looks the exact same with null -> null as zero, connected, or null, options)

this is likely why we've had trouble reproducing. also #1682 will fix this.

also...

dieter Today at 11:17 AM
anyone have experience with grafana stacked charts when series don't have timestamps that line up exactly? e.g. one series has a point every 10s, another every 60s.

davkal 4 hours ago
Yup that’s a known issue with flot, our graphing library

davkal 4 hours ago
there are a handful of issues around this

davkal 4 hours ago
let me find one

davkal 4 hours ago
grafana/grafana#9294