Fix rocksdb can't auto resume after no space error #229
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
related rocksdb issue: facebook/rocksdb#8217 |
git-hulk
changed the title
git-hulk
previously approved these changes
Apr 22, 2021
… only trigger by db_->Write without any other background action (compact/flush)
karelrooted
force-pushed
the
fix/can-not-auto-resume-after-no-space-error
branch
from
77a7b77 to
82a1419
Compare
git-hulk
approved these changes
Apr 22, 2021
ShooterIT
approved these changes
Apr 22, 2021
|
@Mergifyio backport 1.3 |
mergify bot
pushed a commit
that referenced
this pull request
Apr 23, 2021
Fix can't auto resume after no space error when the no space error is only trigger by db_->Write without any other background action (compact/flush) (cherry picked from commit 55c1753) # Conflicts: # src/server.cc # src/storage.h
|
Command
|
Merged
Merged
ShooterIT
pushed a commit
that referenced
this pull request
Jun 10, 2022
In #229, the issue where RocksDB could not recover from the no Space background error was fixed. This problem RocksDB at facebook/rocksdb#8376 has been repaired, but the issue has not been thoroughly solved, The same problem will still occur when an EDQUOT Disk Quota Exceeded error is encountered (see the detailed in facebook/rocksdb#10134). RocksDB cannot recover from this problem and must be restarted. This problem is more likely to occur when kvrocks is deployed in container. In order to handle all versions of RocksDB, we manually resume DB when we encounter two retryable io errors: No space left on device and Disk Quota Exceeded. For the Disk Quota Exceeded error, RocksDB did not expose a friendly interface, so we did a string match.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fix can't auto resume after no space error when the no space error is only trigger by db_->Write without any other background action (compact/flush)