Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for the BITFIELD command #1901

Merged
merged 8 commits into from
Dec 3, 2023

Conversation

julic20s
Copy link
Contributor

@julic20s julic20s commented Nov 20, 2023

@julic20s julic20s force-pushed the bitfield-command branch 2 times, most recently from f4bf982 to 5598089 Compare November 21, 2023 04:24
@julic20s julic20s marked this pull request as ready for review November 21, 2023 04:54
Copy link
Member

@mapleFU mapleFU left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ooops seems this patch is extremly huge...I need take time for review, just some tiny comments first.

src/common/bitfield_util.h Outdated Show resolved Hide resolved
src/common/bitfield_util.h Outdated Show resolved Hide resolved
src/types/redis_bitmap.cc Outdated Show resolved Hide resolved
@git-hulk
Copy link
Member

@julic20s Can use ./x.py format to format your code before pushing.

@julic20s
Copy link
Contributor Author

@julic20s Can use ./x.py format to format your code before pushing.

Sorry, I didn't see it at first, I just used shortcut formatting in the ide

@git-hulk
Copy link
Member

@julic20s Can use ./x.py format to format your code before pushing.

Sorry, I didn't see it at first, I just used shortcut formatting in the ide

ooh, that's fine. You can also configure clang-format in IDE if supported.

@julic20s julic20s force-pushed the bitfield-command branch 2 times, most recently from 9b3fb4a to fea8910 Compare November 21, 2023 12:48
src/common/bitfield_util.h Outdated Show resolved Hide resolved
src/common/bitfield_util.h Outdated Show resolved Hide resolved
src/common/bitfield_util.h Outdated Show resolved Hide resolved
src/common/bitfield_util.h Outdated Show resolved Hide resolved
src/common/bitfield_util.h Outdated Show resolved Hide resolved
src/common/bitfield_util.h Outdated Show resolved Hide resolved
src/storage/batch_extractor.cc Show resolved Hide resolved
src/types/redis_bitmap.cc Outdated Show resolved Hide resolved
src/commands/cmd_bit.cc Outdated Show resolved Hide resolved
@mapleFU
Copy link
Member

mapleFU commented Nov 21, 2023

Thanks @julic20s . This patch is huge, and I need some time to review the whole patch. I just submit some small comments.

Copy link
Member

@PragmaTwice PragmaTwice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution!

Just some random thoughts without going into the details.

Besides, I believe it would be beneficial to create golang test cases for the recently added command.

src/types/redis_bitmap.cc Outdated Show resolved Hide resolved
src/types/redis_bitmap_string.cc Outdated Show resolved Hide resolved
src/types/redis_bitmap_string.cc Outdated Show resolved Hide resolved
@julic20s
Copy link
Contributor Author

julic20s commented Nov 22, 2023

The mutex pool in lock manager may be reallocated by MultiGet(). Then the old mutex pointers in LockGuard all be invalid...I guess that's why I failed the test. Should I use a std::deque to replace the vector?

@git-hulk
Copy link
Member

git-hulk commented Nov 22, 2023

@julic20s Guess not, you should use std::optional<LockGuard> instead of LockGuard directly since the LockGuard constructor will lock the mutex, and if you use LockGuard = default, it won't lock the mutex at all.

refer:

std::optional<LockGuard> lock_guard;

@julic20s
Copy link
Contributor Author

julic20s commented Nov 22, 2023 via email

@julic20s
Copy link
Contributor Author

 LockGuard(LockGuard &&guard) : lock_(guard.lock_) { guard.lock_ = nullptr; } // correct constructor
 LockGuard &operator=(LockGuard &&other) = default; // uncorrect assignment.

😭

@git-hulk
Copy link
Member

@julic20s BTW, would you like to take anonymous functions(get/set/get_segment) from Bitmap::bitfield into named functions. This function is a bit hard to read for now.

@julic20s julic20s marked this pull request as draft November 22, 2023 10:20
src/common/lock_manager.h Outdated Show resolved Hide resolved
src/common/lock_manager.h Outdated Show resolved Hide resolved
src/common/parse_util.h Outdated Show resolved Hide resolved
src/commands/command_parser.h Outdated Show resolved Hide resolved
src/commands/command_parser.h Outdated Show resolved Hide resolved
src/commands/command_parser.h Outdated Show resolved Hide resolved
src/commands/command_parser.h Outdated Show resolved Hide resolved
src/types/redis_bitmap.cc Outdated Show resolved Hide resolved
@mapleFU
Copy link
Member

mapleFU commented Nov 24, 2023

I'm a bit busy on workdays, I'll go through a pass this weekend. If other thinks it's ok we can review an merge it first

@julic20s julic20s force-pushed the bitfield-command branch 2 times, most recently from 791478f to cf1921f Compare November 24, 2023 16:30
@julic20s julic20s marked this pull request as ready for review November 24, 2023 16:31
@julic20s
Copy link
Contributor Author

julic20s commented Nov 25, 2023

@git-hulk Could you help me approval the workflow? I fix some problems that I found. But I can't reproduction the unittest failed on my local machine. I tried both gcc and clang.

@mapleFU
Copy link
Member

mapleFU commented Nov 25, 2023

done

src/common/bitfield_util.h Outdated Show resolved Hide resolved
src/common/bitfield_util.h Outdated Show resolved Hide resolved
src/common/bitfield_util.h Outdated Show resolved Hide resolved
src/common/bitfield_util.h Outdated Show resolved Hide resolved
src/common/bitfield_util.cc Outdated Show resolved Hide resolved
src/common/bitfield_util.cc Show resolved Hide resolved
src/common/bitfield_util.h Outdated Show resolved Hide resolved
src/common/bitfield_util.h Outdated Show resolved Hide resolved
src/types/redis_bitmap.cc Outdated Show resolved Hide resolved
src/types/redis_bitmap.cc Show resolved Hide resolved
src/types/redis_bitmap.cc Outdated Show resolved Hide resolved
src/types/redis_bitmap.cc Outdated Show resolved Hide resolved
@julic20s
Copy link
Contributor Author

I am trying to use rocksdb::PinnableSlice to improve Bitmap::SegmentCacheStore. But I failed because of some strange error. When I use std::unordered_map<uint32_t, rocksdb::PinnableSlice> cache_ and get its iterator, the iterator compare function is failed to instantiate (such as: it != cache_.end()), which makes me can not travel cache_ in BatchForFlush(). rocksdb::Cleanable has a bug in current rocksdb version(It's copy constructor deleted, but the param of it is not marked const).

@julic20s julic20s requested a review from mapleFU November 26, 2023 14:56
@PragmaTwice
Copy link
Member

PragmaTwice commented Nov 26, 2023

I am trying to use rocksdb::PinnableSlice to improve Bitmap::SegmentCacheStore. But I failed because of some strange error. When I use std::unordered_map<uint32_t, rocksdb::PinnableSlice> cache_ and get its iterator, the iterator compare function is failed to instantiate (such as: it != cache_.end()), which makes me can not travel cache_ in BatchForFlush(). rocksdb::Cleanable has a bug in current rocksdb version(It's copy constructor deleted, but the param of it is not marked const).

I don't think using PinnableSlice is necessary for this PR to be merged.

@julic20s
Copy link
Contributor Author

@mapleFU Hi, I finished all changes you pointed, could you review them again? Thanks.

@mapleFU
Copy link
Member

mapleFU commented Nov 28, 2023

Sure, I'm now becoming very busy on workdays, I'll finish review these few days.

@git-hulk
Copy link
Member

git-hulk commented Dec 3, 2023

@PragmaTwice @mapleFU I think we can go ahead to merge this PR if overall looks good.

Copy link

sonarcloud bot commented Dec 3, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 13 Code Smells

No Coverage information No Coverage information
0.5% 0.5% Duplication

@PragmaTwice PragmaTwice merged commit 901c771 into apache:unstable Dec 3, 2023
30 checks passed
@PragmaTwice
Copy link
Member

Thank you for your contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants