Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(keys, scan): Support arbitrary glob patterns #2608

Merged
merged 18 commits into from
Oct 29, 2024

Conversation

nathanlo-hrt
Copy link
Contributor

Kvrocks currently only supports prefix matching (glob patterns like ab*).
This change implements arbitrary glob patterns for KEYS and SCAN MATCH [pattern].

src/common/glob.h Outdated Show resolved Hide resolved

if (!util::StringMatch(suffix_glob, user_key.substr(prefix.size()))) {
continue;
}
keys->emplace_back(user_key);
cnt++;
Copy link
Member

@PragmaTwice PragmaTwice Oct 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering that, if nothing got matched in this limit, e.g. the limit is 10, and in this 10 keys no key is matched, but the 12th key is matched, will a valid cursor be returned so that users can use it to continue the scan?

Also could we add a test case to ensure that?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it seems like cnt is only incremented when we actually add a key to the result, so the loop will continue until no more keys match the prefix, or limit keys are inserted into the result vector

Copy link
Member

@PragmaTwice PragmaTwice Oct 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm I think we'd better refactor the logic, so that the SCAN can be quick, instead of a long-time blocking scan. E.g. we can scan a fixed maximum numbers of keys even if the limit/cnt is not reach. It's fine to return even zero matched key to users.
Or maybe we can confirm the logic in Redis?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if we can just merge this first and then plan to refactor to a "quick" scan.

WDYT? cc @mapleFU @git-hulk

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for missing this comment. This sounds good to me after taking a rough review.

Hmmm I think we'd better refactor the logic, so that the SCAN can be quick, instead of a long-time blocking scan. E.g. we can scan a fixed maximum numbers of keys even if the limit/cnt is not reach. It's fine to return even zero matched key to users.
Or maybe we can confirm the logic in Redis?

Yes, Redis will set a max iteration number(10*count) to avoid blocking too long.

Refer: https://github.com/redis/redis/blob/611c950293ae34dcef148ec62c9dd9626d7dc9e3/src/db.c#L1212

@nathanlo99
Copy link

just to say it, it would be nice to have the typos script run as a precommit hook or lint: it's a bit unideal to get the workflow approved by a reviewer just to find out you've introduced a typo in your commit somewhere

@git-hulk
Copy link
Member

Hi @nathanlo-hrt Seems the Go test is broken, would you mind fixing it to make it mergeable?

Copy link

@PragmaTwice PragmaTwice merged commit 4aa36ec into apache:unstable Oct 29, 2024
32 checks passed
@nathanlo-hrt nathanlo-hrt deleted the globs branch October 29, 2024 14:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants