Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce Checksumming #299

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from
Draft

Introduce Checksumming #299

wants to merge 3 commits into from

Conversation

Kerollmops
Copy link
Member

@Kerollmops Kerollmops commented Dec 7, 2024

This PR introduces the support for checksumming the pages and data in LMDB. It takes most of the work from this other PR.

@Kerollmops Kerollmops force-pushed the checksumming branch 20 times, most recently from db3e839 to 5496b9b Compare December 29, 2024 17:35
@Kerollmops Kerollmops force-pushed the checksumming branch 3 times, most recently from 9baf6b6 to eae2d8e Compare December 29, 2024 18:07
@Kerollmops
Copy link
Member Author

Hey @hyc 👋 I hope you're well.

While implementing checksumming support into my LMDB wrapper, I realized that I probably didn't understand some of the details about the checksumming feature.

I have a test that opens an LMDB environment from mdb.master3 and sets up checksumming by using mdb_env_set_checksum along with a CRC algorithm. I write two entries, read them, commit, and close the environment. I modify the content on disk by also making sure not to change the length of the entries (even though it shouldn't be an issue) and reopen the database with the exact same settings. When I read the entries again, LMDB doesn't detect the corruption and returns the modified value.

So, I thought I needed to enable the encryption and checksumming features. I did that, and now LMDB is returning BadTxn when I try to read the entries just after the commit, not even after the on-disk on-purpose corruption, before I closed the environment.

I am thinking about a bug on LMDB's side and will create a reproducer if you confirm that this looks like one or if I forgot something. You can read the test that is triggering the issue there.

Have a nice day and end of the year, Howard ⛄

@Kerollmops Kerollmops added the breaking A change that is breaking the semver label Dec 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking A change that is breaking the semver
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant