Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(hyperloglog): add support of the Hyperloglog data structure #2142

Merged
merged 38 commits into from
Jul 31, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
3e92895
Support Hyperloglog
Mar 7, 2024
e8cdf69
add origin copyright
Mar 7, 2024
d4b174d
code tuning
Mar 8, 2024
c780c84
code format
Mar 8, 2024
5f86c40
HLL: modify subkeys storage format with dense encoding (#2)
tutububug Mar 22, 2024
cdc375b
fix error of lint and code check (#3)
tutububug Mar 28, 2024
145b703
remove debug code (#5)
tutububug Apr 3, 2024
ae5df70
trigger GitHub actions
Apr 11, 2024
85d286c
Merge branch 'unstable' into hyperloglog-dev-github
tutububug Apr 12, 2024
efa8984
fix check code (#7)
tutububug Apr 12, 2024
57cebd5
fix
Apr 15, 2024
62a943f
update
Apr 15, 2024
48eef8d
Merge branch 'unstable' into hyperloglog-dev-github
tutububug Apr 15, 2024
993dfb7
move copied functions to new file (#10)
tutububug Apr 19, 2024
6df6a69
Merge branch 'unstable' into hyperloglog-dev-github
git-hulk Apr 19, 2024
93f3bc4
correct register merge condition (#12)
tutububug Apr 20, 2024
78a57fc
fix (#13)
tutububug Apr 22, 2024
2cd439c
Merge branch 'unstable' into hyperloglog-dev-github
tutububug Apr 22, 2024
bbdbcb0
fix (#14)
tutububug Apr 23, 2024
559ebc9
fix (#15)
tutububug Apr 24, 2024
bc32fa0
Merge branch 'unstable' into hyperloglog-dev-github
tutububug Apr 24, 2024
98d77d3
Merge branch 'unstable' into hyperloglog-dev-github
tutububug Apr 28, 2024
08d07db
fix code check
Apr 28, 2024
c6d99e2
Merge branch 'unstable' into hyperloglog-dev-github
git-hulk May 10, 2024
64a7a44
Merge branch 'unstable' into hyperloglog-dev-github
tutububug May 10, 2024
7cf88ef
minor updates
mapleFU Jul 30, 2024
d3b2978
remove the code for merge
mapleFU Jul 30, 2024
38e99f0
resume code for bitmap
mapleFU Jul 30, 2024
7e72ca8
Keep cleanup the logic
mapleFU Jul 30, 2024
1af314e
Merge branch 'unstable' into hyperloglog-dev-github
mapleFU Jul 30, 2024
a8e84fd
basic skeleton finished
mapleFU Jul 30, 2024
7f6653c
Fix testing
mapleFU Jul 30, 2024
008ec3f
Update vendor lib
mapleFU Jul 30, 2024
afdb7a7
Trying to fix lint
mapleFU Jul 30, 2024
d8cc9a1
remove bad conflict resolve
mapleFU Jul 30, 2024
22923f9
update comments
mapleFU Jul 30, 2024
e4f3e77
Change HLL to 11
mapleFU Jul 30, 2024
b6fc4a1
trying to fix lint
mapleFU Jul 31, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion src/storage/redis_db.cc
Original file line number Diff line number Diff line change
Expand Up @@ -99,10 +99,18 @@ rocksdb::Status Database::GetMetadata(RedisTypes types, const Slice &ns_key, std
}

rocksdb::Status Database::GetRawMetadata(const Slice &ns_key, std::string *bytes) {
return getRaw(ns_key, bytes);
}

rocksdb::Status Database::GetRawData(const Slice &key, std::string *bytes) {
return getRaw(key, bytes);
}

rocksdb::Status Database::getRaw(const Slice &key, std::string *bytes) {
LatestSnapShot ss(storage_);
rocksdb::ReadOptions read_options;
read_options.snapshot = ss.GetSnapShot();
return storage_->Get(read_options, metadata_cf_handle_, ns_key, bytes);
return storage_->Get(read_options, metadata_cf_handle_, key, bytes);
}

rocksdb::Status Database::Expire(const Slice &user_key, uint64_t timestamp) {
Expand Down
4 changes: 4 additions & 0 deletions src/storage/redis_db.h
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ class Database {
[[nodiscard]] rocksdb::Status GetMetadata(RedisTypes types, const Slice &ns_key, std::string *raw_value,
Metadata *metadata, Slice *rest);
[[nodiscard]] rocksdb::Status GetRawMetadata(const Slice &ns_key, std::string *bytes);
[[nodiscard]] rocksdb::Status GetRawData(const Slice &ns_key, std::string *bytes);
[[nodiscard]] rocksdb::Status Expire(const Slice &user_key, uint64_t timestamp);
[[nodiscard]] rocksdb::Status Del(const Slice &user_key);
[[nodiscard]] rocksdb::Status MDel(const std::vector<Slice> &keys, uint64_t *deleted_cnt);
Expand All @@ -63,6 +64,9 @@ class Database {
[[nodiscard]] rocksdb::Status KeyExist(const std::string &key);
[[nodiscard]] rocksdb::Status Rename(const std::string &key, const std::string &new_key, bool nx, bool *ret);

private:
rocksdb::Status getRaw(const Slice &ns_key, std::string *bytes);

protected:
engine::Storage *storage_;
rocksdb::ColumnFamilyHandle *metadata_cf_handle_;
Expand Down
6 changes: 6 additions & 0 deletions src/storage/redis_metadata.h
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ enum RedisType : uint8_t {
kRedisBloomFilter = 9,
kRedisJson = 10,
kRedisSearch = 11,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need to delete

kRedisHyperloglog = 12,
};

struct RedisTypes {
Expand Down Expand Up @@ -329,3 +330,8 @@ class SearchMetadata : public Metadata {
void Encode(std::string *dst) const override;
rocksdb::Status Decode(Slice *input) override;
};

class HyperloglogMetadata: public Metadata {
public:
explicit HyperloglogMetadata(bool generate_version = true) : Metadata(kRedisHyperloglog, generate_version) {}
};
Loading
Loading