Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove synced-flush #50776

Closed
7 tasks done
dnhatn opened this issue Jan 9, 2020 · 2 comments
Closed
7 tasks done

Remove synced-flush #50776

dnhatn opened this issue Jan 9, 2020 · 2 comments
Assignees
@dnhatn
Labels
:Distributed Indexing/Recovery Anything around constructing a new shard, either from a local or a remote source. Meta

Comments

@dnhatn
Copy link
Member

dnhatn commented Jan 9, 2020 •
edited
Loading

To simplify rolling and full cluster restarts, we want to remove the need for a synced flush and allow quicker recoveries even if indexing is ongoing during the rolling restart. By moving everything to peer recovery retention leases, we can get rid of synced flush in 8.0, as a regular flush then has the same effect as a synced flush.
@dnhatn dnhatn added :Distributed Indexing/Recovery Anything around constructing a new shard, either from a local or a remote source. Meta labels Jan 9, 2020
@dnhatn dnhatn self-assigned this Jan 9, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (:Distributed/Recovery)

This was referenced Jan 10, 2020
Merged
Merged
dnhatn added a commit that referenced this issue Jan 13, 2020
@dnhatn
Deprecate synced flush (#50835)
1a32d71 
A normal flush has the same effect as a synced flush on Elasticsearch 
7.6 or later. It's deprecated in 7.6 and will be removed in 8.0.

Relates #50776
dnhatn added a commit that referenced this issue Jan 14, 2020
@dnhatn
Deprecate synced flush (#50835)
fb32a55 
A normal flush has the same effect as a synced flush on Elasticsearch 
7.6 or later. It's deprecated in 7.6 and will be removed in 8.0.

Relates #50776
dnhatn added a commit that referenced this issue Jan 14, 2020
@dnhatn
Deprecate synced flush (#50835)
08b8450 
A normal flush has the same effect as a synced flush on Elasticsearch
7.6 or later. It's deprecated in 7.6 and will be removed in 8.0.

Relates #50776
SivagurunathanV pushed a commit to SivagurunathanV/elasticsearch that referenced this issue Jan 23, 2020
@dnhatn @SivagurunathanV
Deprecate synced flush (elastic#50835)
276b0ed 
A normal flush has the same effect as a synced flush on Elasticsearch
7.6 or later. It's deprecated in 7.6 and will be removed in 8.0.

Relates elastic#50776
@dnhatn dnhatn mentioned this issue Jan 24, 2020
Merged
dnhatn added a commit that referenced this issue Jan 27, 2020
@dnhatn
Remove sync flush logic in Engine (#51450)
0c87892 
This change removes the sync-flush logic in the InternalEngine as we no 
longer issue or renew syncIds in 8.0.

Relates #50776
@dnhatn dnhatn mentioned this issue Jan 28, 2020
Merged
dnhatn added a commit that referenced this issue Jan 28, 2020
@dnhatn
Add test verify replica allocator with sync_id (#51512)
634a46c 
We no longer issue new sync_ids in 8.0, but we still need to make sure 
that the replica allocator prefers copies with matching sync_id. This
commit adds tests for that.

Relates #50776
albertzaharovits pushed a commit to albertzaharovits/elasticsearch that referenced this issue Jan 28, 2020
@dnhatn @albertzaharovits
Add test verify replica allocator with sync_id (elastic#51512)
b299978 
We no longer issue new sync_ids in 8.0, but we still need to make sure 
that the replica allocator prefers copies with matching sync_id. This
commit adds tests for that.

Relates elastic#50776
albertzaharovits added a commit that referenced this issue Jan 28, 2020
@albertzaharovits @jkakavas
@elasticmachine @droberts195 @dnhatn @pugnascotia
Deprecate timeout.tcp_read AD/LDAP realm setting (#47305)
f016b17 
* Done

* Update docs/reference/settings/security-settings.asciidoc

Co-Authored-By: Ioannis Kakavas <ikakavas@protonmail.com>

* Update docs/reference/settings/security-settings.asciidoc

Co-Authored-By: Ioannis Kakavas <ikakavas@protonmail.com>

* refactored ldap_search explanation

* Tim's review!

* [ML] Use CSV ingest processor in find_file_structure ingest pipeline (#51492)

Changes the find_file_structure response to include a CSV
ingest processor in the ingest pipeline it suggests.

Previously the Kibana file upload functionality parsed CSV
in the browser, but by parsing CSV in the ingest pipeline
it makes the Kibana file upload functionality more easily
interchangable with Filebeat such that the configurations
it creates can more easily be used to import data with the
same structure repeatedly in production.

* Add test verify replica allocator with sync_id (#51512)

We no longer issue new sync_ids in 8.0, but we still need to make sure 
that the replica allocator prefers copies with matching sync_id. This
commit adds tests for that.

Relates #50776

* Formatting: keep simple if / else on the same line (#51526)

Previous the formatter was breaking simple if/else statements (i.e.
without braces) onto separate lines, which could be fragile because the
formatter cannot also introduce braces. Instead, keep such expressions
on the same line.

* Nits

Co-authored-by: Ioannis Kakavas <ikakavas@protonmail.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: David Roberts <dave.roberts@elastic.co>
Co-authored-by: Nhat Nguyen <nhat.nguyen@elastic.co>
Co-authored-by: Rory Hunter <pugnascotia@users.noreply.github.com>
@dnhatn
Copy link
Member Author

dnhatn commented Jan 30, 2020

All sub tasks are completed.

@dnhatn dnhatn closed this as completed Jan 30, 2020
This was referenced Feb 3, 2020
Closed
Closed
@mkleen mkleen mentioned this issue Mar 25, 2021
Closed
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dnhatn dnhatn

Labels
:Distributed Indexing/Recovery Anything around constructing a new shard, either from a local or a remote source. Meta
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@dnhatn @elasticmachine