Copy retention leases when trim unsafe commits #37995
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When a primary shard is recovered from its store, we trim the last commit (if it's unsafe). If a shard crashes before the recovery completes, we will lose the committed retention leases because they are baked in the last commit. With this change, we copy the retention leases from the last commit to the safe commit when trimming unsafe commits.
dnhatn
added
>enhancement
v7.0.0
:Distributed Indexing/Distributed
|
Pinging @elastic/es-distributed |
24 tasks
|
This is a getting a bit out of hand. I'm starting to wonder if we're not better off having a dedicated state file in the index/translog/shard folder that is dedicated for this. @dnhatn this is not meant specifically at this PR as you are just extending the "let's store this in a lucene commit" line of thought. I'm just starting to doubt if it is the right one... |
jasontedor
approved these changes
Feb 1, 2019
server/src/test/java/org/elasticsearch/index/shard/IndexShardTests.java
Outdated
Show resolved
Hide resolved
|
Thanks @dnhatn. |
|
I am pulling this in, and we will revert it when we have a new persistent layer. Thanks for looking @bleskes and @jasontedor. |
jasontedor
added a commit
to jasontedor/elasticsearch
that referenced
this pull request
Feb 12, 2019
* master: Enable removal of retention leases (elastic#38751) Make the 'get templates' types deprecation message consistent. (elastic#38533) Copy retention leases when trim unsafe commits (elastic#37995) Fix the version check for LegacyGeoShapeFieldMapper (elastic#38547)
jasontedor
added a commit
to jasontedor/elasticsearch
that referenced
this pull request
Feb 12, 2019
* master: Enable removal of retention leases (elastic#38751) Make the 'get templates' types deprecation message consistent. (elastic#38533) Copy retention leases when trim unsafe commits (elastic#37995) Fix the version check for LegacyGeoShapeFieldMapper (elastic#38547)
dnhatn
added a commit
that referenced
this pull request
Feb 12, 2019
When a primary shard is recovered from its store, we trim the last commit (when it's unsafe). If that primary crashes before the recovery completes, we will lose the committed retention leases because they are baked in the last commit. With this change, we copy the retention leases from the last commit to the safe commit when trimming unsafe commits.
dnhatn
added a commit
that referenced
this pull request
Feb 13, 2019
When a primary shard is recovered from its store, we trim the last commit (when it's unsafe). If that primary crashes before the recovery completes, we will lose the committed retention leases because they are baked in the last commit. With this change, we copy the retention leases from the last commit to the safe commit when trimming unsafe commits. Relates #37165
dnhatn
added a commit
that referenced
this pull request
Feb 13, 2019
When a primary shard is recovered from its store, we trim the last commit (when it's unsafe). If that primary crashes before the recovery completes, we will lose the committed retention leases because they are baked in the last commit. With this change, we copy the retention leases from the last commit to the safe commit when trimming unsafe commits. Relates #37165
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When a primary shard is recovered from its store, we trim the last
commit (when it's unsafe). If that primary crashes before the recovery
completes, we will lose the committed retention leases because they are
baked in the last commit. With this change, we copy the retention leases
from the last commit to the safe commit when trimming unsafe commits.
Relates #37165