This repository has been archived by the owner on Aug 13, 2019. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 179
postings compression exploration #629
Draft
naivewong
wants to merge
18
commits into
prometheus-junkyard:master
Choose a base branch
from
naivewong:postings-compression
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
postings compression exploration #629
naivewong
wants to merge
18
commits into
prometheus-junkyard:master
from
naivewong:postings-compression
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
naivewong
commented
Jun 12, 2019
- Store the first posting, then store the delta to that (not the delta to the previous element, the delta to the first number only). Find the minimum number of bits required to store the deltas. The bitpack using those many bits. Also be able to do binary search here.
- Use blocks of postings, like 4KB block, store deltas with the previous number. Binary search to the right block and then iterate.
Signed-off-by: naivewong <867245430@qq.com>
|
||
startOff := it.offset / (deltaBlockSize * 8) * deltaBlockSize | ||
num := len(it.bs.bstream) / deltaBlockSize - it.offset / (deltaBlockSize * 8) | ||
// Do binary search between current position and end. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Try checking if the next block is too big before doing the search.
|
codesome
reviewed
Jun 13, 2019
Signed-off-by: naivewong <867245430@qq.com>
naivewong
force-pushed
the
postings-compression
branch
2 times, most recently
from
June 17, 2019 16:52
ef22dcd
to
11f2b1b
Compare
Signed-off-by: naivewong <867245430@qq.com>
naivewong
force-pushed
the
postings-compression
branch
from
June 17, 2019 17:00
11f2b1b
to
7cfcf3d
Compare
Signed-off-by: naivewong <867245430@qq.com>
naivewong
force-pushed
the
postings-compression
branch
from
June 18, 2019 16:16
e11da7c
to
99ce72a
Compare
Signed-off-by: naivewong <867245430@qq.com>
Signed-off-by: naivewong <867245430@qq.com>
Signed-off-by: naivewong <867245430@qq.com>
Signed-off-by: naivewong <867245430@qq.com>
Signed-off-by: naivewong <867245430@qq.com>
Signed-off-by: naivewong <867245430@qq.com>
naivewong
force-pushed
the
postings-compression
branch
from
June 28, 2019 07:46
d130757
to
b3f2b5e
Compare
Signed-off-by: naivewong <867245430@qq.com>
Signed-off-by: naivewong <867245430@qq.com>
naivewong
force-pushed
the
postings-compression
branch
2 times, most recently
from
July 4, 2019 14:49
c4b6f0f
to
c7505ed
Compare
Signed-off-by: naivewong <867245430@qq.com>
naivewong
force-pushed
the
postings-compression
branch
from
July 5, 2019 02:52
c7505ed
to
95f3c79
Compare
…comments) Signed-off-by: naivewong <867245430@qq.com>
Signed-off-by: naivewong <867245430@qq.com>
Signed-off-by: naivewong <867245430@qq.com>
Signed-off-by: naivewong <867245430@qq.com>
Signed-off-by: naivewong <867245430@qq.com>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.