-
Notifications
You must be signed in to change notification settings - Fork 472
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix MSETNX not allow overriding the same key #1631
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Didn't review it carefully, just a tiny round.
After a round of review I get to understand this patch. There is a problem: class MultiLockGuard {
public:
explicit MultiLockGuard(LockManager *lock_mgr, const std::vector<std::string> &keys) : lock_mgr_(lock_mgr) {
locks_ = lock_mgr_->MultiGet(keys);
for (const auto &iter : locks_) {
iter->lock();
}
}
~MultiLockGuard() {
// Lock with order `A B C` and unlock should be `C B A`
for (auto iter = locks_.rbegin(); iter != locks_.rend(); ++iter) {
(*iter)->unlock();
}
}
MultiLockGuard(const MultiLockGuard &) = delete;
MultiLockGuard &operator=(const MultiLockGuard &) = delete;
private:
LockManager *lock_mgr_ = nullptr;
std::vector<std::mutex *> locks_;
}; Previously, the lock happens per-key. It was a bit wierd but would not introduce problem. Now, A proposal is that using std::lock( https://en.cppreference.com/w/cpp/thread/lock ) instead of loop in @PragmaTwice would that be possible? |
@mapleFU I think it won't have the deadlock for the multi-keys lock since it will sort keys first: https://github.com/apache/kvrocks/blob/unstable/src/storage/lock_manager.cc#L43 |
Nice I didn't find out that. |
The previous implementation which wrote the batch per key, we should change it to a single batch. Also use the multi-key lock before the write operations instead of inside the loop. This is mention in #1631, found by hulk. Co-authored-by: mwish <1506118561@qq.com>
The changes in apache#337 causing this issue: ``` 127.0.0.1:6379> flushall OK 127.0.0.1:6379> msetnx k v1 k v2 (integer) 1 127.0.0.1:6379> get k "v2 127.0.0.1:6666> flushall OK 127.0.0.1:6666> msetnx k v1 k v2 (integer) 0 127.0.0.1:6666> get k "v1" ``` Redis allow we overriding the same key but kvrocks will fail in this case. The way to handle it is to revert the changes in apache#337. So this PR is based on that, after reverting the changes of apache#337, we can reuse the logic of MSet. And hulk mentions that MSETNX is an exclusive command and we better to lock those keys and then remove the exclusive flag from the command. So we use MultiLockGuard before Exists call to lock multi keys. And change MSet to support non-lock calls. And also remove the exclusive flag from MSETNX command.
The previous implementation which wrote the batch per key, we should change it to a single batch. Also use the multi-key lock before the write operations instead of inside the loop. This is mention in apache#1631, found by hulk. Co-authored-by: mwish <1506118561@qq.com>
The changes in #337 causing this issue:
Redis allow we overriding the same key but kvrocks will
fail in this case. The way to handle it is to revert the
changes in #337.
So this PR is based on that, after reverting the changes
of #337, we can reuse the logic of MSet. And hulk mentions
that MSETNX is an exclusive command and we better to lock
those keys and then remove the exclusive flag from the command.
So we use MultiLockGuard before Exists call to lock multi
keys. And change MSet to support non-lock calls. And also
remove the exclusive flag from MSETNX command.
Co-authored-by: hulk hulk.website@gmail.com