Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert store iterators to variants rather than using polymorphism. #4759

Merged
merged 2 commits into from
Oct 27, 2024

Conversation

clemahieu
Copy link
Contributor

This converts the store iterators from version that use polymorphism to using std::variant.

The primary drivers for this change is to simplify and allow more complex iterators to be created where this was previously fairly difficult to do.

The iterators are separated in the 3 parts:

  • The base database iterators LMDB/RocksDB that use the library's native data type, MDB_val/Slice.
  • A variant database iterator that can hold any of the raw database iterators and adapts the native database type to std::span<uint8_t>
  • A strong-typed database iterator that mirrors existing iterator behavior with strong typing for key/value pairs.

The iterators are bi-directional with a sentinel value to signify end of iteration. The iterators are also circular so after reaching the end-sentinel value, further incrementing will wrap around to the beginning, or decrementing will position the iterator at the last key/value.

Using std::variant also eliminates heap allocations when constructing iterators which can improve performance in some circumstances.

@gr0vity-dev-bot
Copy link

gr0vity-dev-bot commented Oct 20, 2024

Test Results for Commit 37f4f5c

Pull Request 4759: Results
Overall Status:

Test Case Results

  • 5n4pr_conf_10k_bintree: PASS (Duration: 107s)
  • 5n4pr_conf_10k_change: PASS (Duration: 170s)
  • 5n4pr_conf_change_dependant: PASS (Duration: 143s)
  • 5n4pr_conf_change_independant: PASS (Duration: 165s)
  • 5n4pr_conf_send_dependant: PASS (Duration: 136s)
  • 5n4pr_conf_send_independant: PASS (Duration: 151s)
  • 5n4pr_rocks_10k_bintree: PASS (Duration: 112s)
  • 5n4pr_rocks_10k_change: PASS (Duration: 206s)

Last updated: 2024-10-27 03:00:50 UTC

nano/store/iterator.cpp Outdated Show resolved Hide resolved
@clemahieu clemahieu added this to the V28 milestone Oct 21, 2024
…transaction->GetHandle()

Returning either Transaction * or ReadOptions * seems dubious, the intent of this commit is to retain 1-to-1 compatibility while adding strong typing.
@clemahieu clemahieu marked this pull request as ready for review October 26, 2024 14:47
@clemahieu
Copy link
Contributor Author

I checked on using std::reverse_iterator rather than implementing our own though it looks like it only supports copyable iterators. https://en.cppreference.com/w/cpp/iterator/reverse_iterator/reverse_iterator

pwojcikdev
pwojcikdev previously approved these changes Oct 26, 2024
Copy link
Contributor

@pwojcikdev pwojcikdev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really, really good. Already makes working with database backends simpler.

nano/store/rocksdb/iterator.cpp Outdated Show resolved Hide resolved
{
if (dynamic_cast<nano::store::read_transaction const *> (&transaction_a) != nullptr)
{
return static_cast<::rocksdb::ReadOptions *> (transaction_a.get_handle ());
Copy link
Contributor

@pwojcikdev pwojcikdev Oct 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible for the handle to store the pointer as std::any and use type safe cast std::any_cast? Or is that not necessary here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think applying a similar change to transactions would be another good improvement. I'll leave that to another PR though.

{
// FIXME Don't convert via lmdb::db_val, this is just a placeholder
auto const & data = *iter;
lmdb::db_val key_val{ MDB_val{ data.first.size (), const_cast<void *> (reinterpret_cast<void const *> (data.first.data ())) } };
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, if it works it works 😆

@clemahieu clemahieu merged commit 09b635f into nanocurrency:develop Oct 27, 2024
27 of 28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Merged / V28.0
Development

Successfully merging this pull request may close these issues.

3 participants