-
Notifications
You must be signed in to change notification settings - Fork 107
use pointslicepool for /getdata cluster-fan out requests #1921
Conversation
NOTE: i'm still working on some tests to confirm improvements |
1242387
to
de35bcf
Compare
i spun up docker-cluster-query with 4 query pods. I ran
I ran it for more than an hour. dashboard is here |
this should help reduce memory usage in query nodes when they decode /getdata requests coming back
de35bcf
to
ec83e22
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we need to manually maintain Series msgp encoding / decoding etc... ?
because we use custom code to use the slice pool. perhaps it can be done cleaner with the msgp extension system but i didn't want to spend too much time on it. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Note: this builds on top of #1923, review/merge that first.
when handling a user query, fanning out said query, and reading back the /getdata responses from peers, this will use the pointslicepool to decode incoming point slices into and should reduce memory usage. see #962
for review purposes, the generated code is identical except for the calls to pointslicepool.GetMin instead of
make