Skip to content

add compression support in GetMsgPackedValueAsString #327

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 20 commits into
base: master
Choose a base branch
from

Conversation

hendrikmuhs
Copy link
Contributor

adds support to retrieve a compressed version of the msgpacked value. If values are stored compressed directly uses the stored/compressed value.

@hendrikmuhs hendrikmuhs marked this pull request as ready for review March 28, 2025 09:24
@hendrikmuhs hendrikmuhs force-pushed the getmsgpackedvalue-compressed branch from 1e47f94 to f90462c Compare March 30, 2025 18:41
Copy link
Contributor

@gmossessian gmossessian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

didn't really check the c++ code very carefully but left some questions. feel free to ping on slack :)


let mut snap_decoder = Decoder::new();
let value_compressed =
m.get_msgpacked_value_compressed(keyvi::Compression::SNAPPY_COMPRESSION);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should there be tests for each of the compression algorithms available?

ctypedef enum CompressionAlgorithm:
NO_COMPRESSION,
ZLIB_COMPRESSION,
SNAPPY_COMPRESSION
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about ZSTD?

@@ -100,6 +100,18 @@ class NullValueStoreReader final : public IValueStoreReader {
attributes_t GetValueAsAttributeVector(uint64_t fsa_value) const override { return attributes_t(); }

std::string GetValueAsString(uint64_t fsa_value) const override { return ""; }

std::string GetRawValueAsString(uint64_t fsa_value) const override { return "\x00\xc0"; }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just curious, what is \x00\xc0? both here and \xc0 below.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants