Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Detect noop of update index settings #61348

Merged
merged 3 commits into from
Aug 20, 2020
Merged

Conversation

dnhatn
Copy link
Member

@dnhatn dnhatn commented Aug 19, 2020

This optimization is more relevant in the context of CCR. When a node in the follower leaves, we reallocate the shard-follow tasks on that node to other nodes. The new tasks will overwhelm the follower cluster with many put-mapping, update-settings requests, although most of them are noop. This change detects and optimizes for noop update-settings requests.

@dnhatn dnhatn added >enhancement :Distributed Indexing/Distributed A catch all label for anything in the Distributed Area. Please avoid if you can. v8.0.0 v7.10.0 labels Aug 19, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (:Distributed/Distributed)

@elasticmachine elasticmachine added the Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. label Aug 19, 2020
@dnhatn dnhatn changed the title Detect noop of index setting update Detect noop of update index settings Aug 19, 2020
Copy link
Contributor

@henningandersen henningandersen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@dnhatn
Copy link
Member Author

dnhatn commented Aug 20, 2020

run elasticsearch-ci/packaging-sample-windows

@dnhatn dnhatn added :Distributed Indexing/CCR Issues around the Cross Cluster State Replication features and removed :Distributed Indexing/CCR Issues around the Cross Cluster State Replication features labels Aug 20, 2020
@dnhatn
Copy link
Member Author

dnhatn commented Aug 20, 2020

Thanks Henning!

@dnhatn dnhatn merged commit 30e0edb into elastic:master Aug 20, 2020
@dnhatn dnhatn deleted the noop-update-index branch August 20, 2020 19:56
dnhatn added a commit that referenced this pull request Aug 24, 2020
This optimization is more relevant in the context of CCR. When a node in
the follower cluster leaves, we reallocate the shard-follow tasks on 
that node to other nodes. The new tasks will overwhelm the follower
cluster with many put-mapping, update-settings requests, although most
of them are noop. This change detects and optimizes the noop
update-settings requests.
DaveCTurner added a commit to DaveCTurner/elasticsearch that referenced this pull request Apr 7, 2021
This test works by capturing the applied cluster state and verifying
that it does not change as no-op updates are applied. However the
captured cluster state might contain some ongoing relocations which
subsequently finish, causing an unexpected cluster state update. This
commit fixes that by waiting for any ongoing relocations too.

Relates elastic#61348
Closes elastic#70961
DaveCTurner added a commit that referenced this pull request Apr 7, 2021
This test works by capturing the applied cluster state and verifying
that it does not change as no-op updates are applied. However the
captured cluster state might contain some ongoing relocations which
subsequently finish, causing an unexpected cluster state update. This
commit fixes that by waiting for any ongoing relocations too.

Relates #61348
Closes #70961
DaveCTurner added a commit that referenced this pull request Apr 7, 2021
This test works by capturing the applied cluster state and verifying
that it does not change as no-op updates are applied. However the
captured cluster state might contain some ongoing relocations which
subsequently finish, causing an unexpected cluster state update. This
commit fixes that by waiting for any ongoing relocations too.

Relates #61348
Closes #70961
@jakelandis jakelandis removed the v8.0.0 label Jul 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Distributed Indexing/Distributed A catch all label for anything in the Distributed Area. Please avoid if you can. >enhancement Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. v7.10.0 v8.0.0-alpha1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants