Use a NoSyncFileSystem to skip Sync/FSync to reduce test times #380
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ayulas
approved these changes
Feb 7, 2023
mrambacher
changed the title
mrambacher
changed the title
ayulas
pushed a commit
that referenced
this pull request
Feb 26, 2023
Use a NoSyncFileSystem to skip Sync/FSync to reduce test times
Yuval-Ariel
pushed a commit
that referenced
this pull request
May 1, 2023
Use a NoSyncFileSystem to skip Sync/FSync to reduce test times
Yuval-Ariel
pushed a commit
that referenced
this pull request
May 4, 2023
Use a NoSyncFileSystem to skip Sync/FSync to reduce test times
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR is based on RocksDB 9545. This change introduces an InjectionFS class that can intercept the calls to the FileSystem and its associated File classes, allowing different behavior to be injected into the system.
Two examples of this Injection are part of this PR. The CountedFileSystem class allows file operations (open, close, read, write, etc) to be counted. This class existed before but takes advantage of the new infrastructure.
The NoSyncFileSystem is another InjectionFS. This file system can be used to turn off the various sync and fsync operations. This class is meant to be used primarily for testing.
As documented elsewhere, the MacOS uses time-consuming sync operations, causing the unit tests to take a very long time. Below are the results of a make check on a Mac showing the performance improvement:
main: make -j 10 check 5320.78s user 1995.73s system 75% cpu 2:40:45.65 total
PR: make -j 10 check 4166.23s user 1556.93s system 119% cpu 1:19:54.49 total
PR+319: make -j 10 check 3367.97s user 1225.99s system 209% cpu 36:36.18 total