when adding new point to aggregate, always see if we can flush #1425
Conversation
e.g. when flushing every 10s and we reach a timestamp like 12:34:10 we can always flush, because any next point will always go into the next bucket. We already did this but only in the case of already having data in the bucket. In the case of an unfortunate mismatch between aggergation and series (e.g. aggregate across 10s but the data itself comes every 10s or even less frequent), we can also just flush in this case, rather than waiting.
|
It does not pass the unit tests |
I admit I am a little lost after 10 minutes of going through it with various scenarii in mind and thinking I understood it but I don't. |
|
how does it look now? |
Much better. Is it possible that |
|
it's possible. we (you actually IIRC) recently made a change to how we deal we deal with duplicate points in general. if we decided on first write wins, this should not be an issue. if we decided last write wins, than we have a problem with the approach here. |
It was me indeed, and we had subsequent writes with identical timestamps to fail, so it's first write wins. So yes we have code guarding us against it. |
e.g. when flushing every 10s and we reach a timestamp like 12:34:10
we can always flush, because any next point will always go into
the next bucket. We already did this but only in the case of already
having data in the bucket. In the case of an unfortunate mismatch
between aggergation and series (e.g. aggregate across 10s but the
data itself comes every 10s or even less frequent), we can also
just flush in this case, rather than waiting.