Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ranges cache: add ownership check #6004

Merged
merged 2 commits into from
Jun 4, 2023

Conversation

Jonathan-Rosenberg
Copy link
Contributor

@Jonathan-Rosenberg Jonathan-Rosenberg commented Jun 1, 2023

Since we pass nil when calling for Value, the initialized buffer isn't owned by the calling function. This makes the behavior of the data inside the buffer unexpected. You can read about it here some more ([P1], [P2]).

Closes 1578

@github-actions
Copy link

github-actions bot commented Jun 1, 2023

No linked issues found. Please add the corresponding issues in the pull request description.
Use GitHub automation to close the issue when a PR is merged

@Jonathan-Rosenberg Jonathan-Rosenberg added the exclude-changelog PR description should not be included in next release changelog label Jun 1, 2023
@Jonathan-Rosenberg Jonathan-Rosenberg marked this pull request as ready for review June 1, 2023 15:16
@Jonathan-Rosenberg Jonathan-Rosenberg added the minor-change Used for PRs that don't require issue attached label Jun 1, 2023
Copy link
Contributor

@arielshaqed arielshaqed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow!

@@ -92,7 +92,13 @@ func (m *RangeManager) GetValueGE(ctx context.Context, ns committed.Namespace, i
}
return nil, ErrKeyNotFound
}
vBytes, _, err := value.Value(nil)
vBytes, owned, err := value.Value(nil)
if !owned {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, great catch!

Is there any way for us to change this bad ownership status? IIUC when !owned, iterator it shares underlying state with some other iterator. I would expect this state to persist, and now very many calls (including future getValue()s) will keep on copying every value. It might be considerably more efficient if we could convince Pebble to unshare the iterator -- say by creating future iterators on a new iterator hierarchy. (It goes without saying that this is no reason to delay this PR!)

Copy link
Contributor

@arielshaqed arielshaqed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

StillLGTM

LGTM

@Jonathan-Rosenberg Jonathan-Rosenberg merged commit 03068a7 into master Jun 4, 2023
@Jonathan-Rosenberg Jonathan-Rosenberg deleted the fix/cache-value-ownership branch June 4, 2023 11:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
exclude-changelog PR description should not be included in next release changelog minor-change Used for PRs that don't require issue attached
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants