Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make NoSync more effective #291

Closed
wants to merge 1 commit into from
Closed

make NoSync more effective #291

wants to merge 1 commit into from

Conversation

seebs
Copy link

@seebs seebs commented Sep 30, 2021

The database init and grow operations still used fsync even when
NoSync was set, on the grounds that these operations are comparatively
rare and it makes sense to want them to be more reliable even at
some marginal cost in speed.

But if you're doing tests on something which happens to use
boltdb lightly, so it creates databases but barely touches them, this
can result in hundreds and hundreds of fsyncs, possibly in parallel,
and that becomes expensive.

Worth noting: Yes, it's intentional that the grow() case is using
db.file.Sync() rather than fdatasync(). On some platforms, fdatasync()
may not cause filesystem metadata updates to get flushed, just raw
file data block updates. We optimistically hope that (File).Sync() will
do the right thing.

The database init and grow operations still used fsync even when
NoSync was set, on the grounds that these operations are comparatively
rare and it makes sense to want them to be more reliable even at
some marginal cost in speed.

But if you're doing tests on something which happens to *use*
boltdb, this can result in hundreds and hundreds of fsyncs, possibly
in parallel, and that becomes expensive.
@missinglink
Copy link
Contributor

semi-related: #290

@seebs
Copy link
Author

seebs commented Oct 1, 2021

If it matters, I did wave these changes at Ben Johnson before submitting them. I was originally going to change the db.file.Sync to fdatasync for consistency, but apparently the not-identical-on-some-platforms was itself intentional.

@ptabor
Copy link
Contributor

ptabor commented Jan 19, 2023

  1. I can imagine db.grow syncs can be expensive when operating on small (<AllocSize) databases.
    Are the init's worth the change as well ?

  2. Why db.NoGrowSync flag is not sufficient to cover the Initial grow syncs ?

  3. How about having the InitialAllocSize flag that determines the file size after the creation ?

@ahrtr ahrtr closed this Jul 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

4 participants