Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reuse Wasm-side buffer while reading iterators #435

Merged
merged 2 commits into from
Oct 16, 2023

Conversation

RReverser
Copy link
Member

Description of Changes

Reuse buffer when reading new chunks from the iterator, instead of freeing and allocating a new one each time.

This will be mainly useful on large iterations that span over our 64KiB chunks.

API and ABI

  • This is a breaking change to the module ABI
  • This is a breaking change to the module API
  • This is a breaking change to the ClientAPI
  • This is a breaking change to the SDK API

If the API is breaking, please state below what will break

Expected complexity level and risk

How complicated do you think these changes are? Grade on a scale from 1 to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex change.

This complexity rating applies not only to the complexity apparent in the diff,
but also to its interactions with existing and future code.

If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning ways.

This will be mainly useful on large iterations that span over
our 64KiB chunks.
@RReverser
Copy link
Member Author

Woops, gonna fix in a sec.

@kulakowski
Copy link
Contributor

Conceptually makes sense. This doesn't change the buffer sharing protocol for all the iteration stuff, correct, so it's not an ABI break? The wasm side was and is responsible.

@RReverser
Copy link
Member Author

This doesn't change the buffer sharing protocol for all the iteration stuff, correct, so it's not an ABI break? The wasm side was and is responsible.

Correct, this is purely Wasm-side optimisation with no ABI change.

@@ -263,7 +263,11 @@ fn buffer_table_iter(
let mut iter = sys::iter(table_id, filter.as_deref())?;

// First item is an encoded schema.
let schema_raw = iter.next().expect("Missing schema").expect("Failed to get schema");
let schema_raw = iter
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW this will go away if/when #420 is merged, so will also become simpler.

@RReverser RReverser enabled auto-merge (squash) October 16, 2023 18:04
@RReverser RReverser merged commit 7478e78 into master Oct 16, 2023
@RReverser RReverser deleted the wasm-iter-reuse-buffer branch October 16, 2023 18:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants