Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce retention lease persistence #37375

Merged
merged 15 commits into from
Jan 12, 2019

Conversation

jasontedor
Copy link
Member

This commit introduces the persistence of retention leases by persisting them in index commits and recovering them when recovering a shard from store.

Relates #37165

This commit introduces the persistence of retention leases by persisting
them in index commits and recovering them when recovering a shard from
store.
@jasontedor jasontedor added >enhancement v7.0.0 :Distributed Indexing/Distributed A catch all label for anything in the Distributed Area. Please avoid if you can. v6.7.0 labels Jan 11, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed

@jasontedor jasontedor mentioned this pull request Jan 11, 2019
24 tasks
@jasontedor
Copy link
Member Author

@elasticmachine run gradle build tests 2

Copy link
Member

@dnhatn dnhatn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left two minor comments.

@jasontedor
Copy link
Member Author

Thanks @dnhatn, I have responded to your feedback.

…rsistence

* elastic/master:
  Fix PrimaryAllocationIT Race Condition (elastic#37355)
  SQL: Make `FULL` non-reserved keyword in the grammar (elastic#37377)
  SQL: [Tests] Fix and enable internalClusterTests (elastic#37300)
  ML: Fix testMigrateConfigs  (elastic#37373)
  Fix RollupDocumentation test to wait for job to stop
Copy link
Member

@dnhatn dnhatn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks @jasontedor.

@jasontedor
Copy link
Member Author

@elasticmachine run gradle build tests 1

@jasontedor jasontedor merged commit 03be4db into elastic:master Jan 12, 2019
jasontedor added a commit that referenced this pull request Jan 12, 2019
This commit introduces the persistence of retention leases by persisting
them in index commits and recovering them when recovering a shard from
store.
Objects.requireNonNull(retentionLease);
return String.format(
Locale.ROOT,
"id:%s;retaining_seq_no:%d;timestamp:%d;source:%s",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should use JSON, with the upside of being more extensible and will allow us to use our parsing infra (and the validation features it has). WDYT?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not particularly concerned about this needing to be extensible. I can be convinced that JSON is worth it but I am doubting.

@jasontedor jasontedor deleted the retention-lease-persistence branch January 15, 2019 03:51
dnhatn added a commit that referenced this pull request Jan 31, 2019
If a new retention lease is added while a primary's soft-deletes policy
is locked for peer-recovery, that lease won't be baked into the Lucene
commit.

Relates #37165
Relates #37375
dnhatn added a commit that referenced this pull request Jan 31, 2019
If a new retention lease is added while a primary's soft-deletes policy
is locked for peer-recovery, that lease won't be baked into the Lucene
commit.

Relates #37165
Relates #37375
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Distributed Indexing/Distributed A catch all label for anything in the Distributed Area. Please avoid if you can. >enhancement v6.7.0 v7.0.0-beta1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants