Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement WAL iterator for migrating by WAL logs #2007

Closed
Tracked by #1223
git-hulk opened this issue Jan 12, 2024 · 2 comments
Closed
Tracked by #1223

Implement WAL iterator for migrating by WAL logs #2007

git-hulk opened this issue Jan 12, 2024 · 2 comments
Assignees

Comments

@git-hulk
Copy link
Member

No description provided.

@caipengbo
Copy link
Contributor

In #2004, we implement the unified key-value iterators, I want to change data to provide a unified iterators.

For incremental data, we need to call rocksdb::WALIter and then implement the rocksdb::WriteBatch::Handler interface.

I want to provide an interface that behaves similarly to engine::DBIterator:

class DBIterator {
public:
explicit DBIterator(Storage *storage, rocksdb::ReadOptions read_options, int slot = -1);
~DBIterator() = default;
bool Valid() const;
void Seek(const std::string &target = "");
void Next();
// return the raw key in rocksdb
Slice Key() const;
// return the namespace and user key without prefix
std::tuple<Slice, Slice> UserKey() const;
Slice Value() const;
RedisType Type() const;
void Reset();
std::unique_ptr<SubKeyIterator> GetSubKeyIterator() const;
private:
void nextUntilValid();
Storage *storage_;
rocksdb::ReadOptions read_options_;
int slot_ = -1;
Metadata metadata_ = Metadata(kRedisNone, false);
rocksdb::ColumnFamilyHandle *metadata_cf_handle_ = nullptr;
std::unique_ptr<rocksdb::Iterator> metadata_iter_;
std::unique_ptr<SubKeyIterator> subkey_iter_;
};
.

It will wrap rocksdb::WALIter and can return different value types. It is possible to implement their processing logic depending on the concrete type.

HDYT @git-hulk

@git-hulk
Copy link
Member Author

@caipengbo Sounds good to me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants