Skip to content
This repository has been archived by the owner on Aug 2, 2022. It is now read-only.

Fix for upgrading mapping #309

Merged
merged 2 commits into from
Nov 10, 2020

Conversation

kaituo
Copy link
Member

@kaituo kaituo commented Nov 9, 2020

Issue #, if available:

Description of changes:

This PR fixes two bugs of upgrading mapping:
First, previously we only tried once when upgrading mapping. It is possible we need to try a few more times. The drawback is that if the code has a bug, we retry endlessly. I should fix it in the future.
Second, we enclose the upgrade mapping function call under a user context in a recent commit. Upgrade mapping will fail when security plugin is enabled since a regular user cannot upgrade mappings of system indices.

The PR also adds upper bounds to our dynamic settings if reasonable.
Testing done:

  1. Tested that upgrade mapping succeeds with new changes.
  2. Tested the upper bounds of the changed settings are in effect.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

This PR fixes two bugs of upgrading mapping:
First, previously we only tried once when upgrading mapping.  It is possible we need to try a few more times.  The drawback is that if the code has a bug, we retry endlessly.  I should fix it in the future.
Second, we enclose the upgrade mapping function call under a user context in a recent commit.  Upgrade mapping will fail when security plugin is enabled since a regular user cannot upgrade mappings of system indices.

The PR also adds upper bounds to our dynamic settings if reasonable.
Testing done:
1. Tested that upgrade mapping succeeds with new changes.
2. Tested the upper bounds of the changed settings are in effect.
@codecov
Copy link

codecov bot commented Nov 9, 2020

Codecov Report

Merging #309 (d762af5) into master (75d564b) will increase coverage by 0.07%.
The diff coverage is 84.84%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master     #309      +/-   ##
============================================
+ Coverage     72.06%   72.14%   +0.07%     
- Complexity     1969     1988      +19     
============================================
  Files           199      201       +2     
  Lines          9465     9553      +88     
  Branches        844      847       +3     
============================================
+ Hits           6821     6892      +71     
- Misses         2233     2248      +15     
- Partials        411      413       +2     
Flag Coverage Δ Complexity Δ
cli 79.27% <ø> (ø) 0.00 <ø> (ø)
plugin 71.56% <84.84%> (+0.09%) 0.00 <19.00> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ Complexity Δ
...endistroforelasticsearch/ad/cluster/DailyCron.java 100.00% <ø> (ø) 6.00 <0.00> (ø)
...sticsearch/ad/indices/AnomalyDetectionIndices.java 49.01% <0.00%> (-1.33%) 39.00 <0.00> (-1.00)
...relasticsearch/ad/cluster/MasterEventListener.java 94.59% <87.50%> (+0.15%) 8.00 <1.00> (ø)
...ter/diskcleanup/ModelCheckpointIndexRetention.java 89.47% <89.47%> (ø) 8.00 <8.00> (?)
...ticsearch/ad/cluster/diskcleanup/IndexCleanup.java 95.00% <95.00%> (ø) 10.00 <10.00> (?)
...oforelasticsearch/ad/AnomalyDetectorJobRunner.java 75.00% <100.00%> (ø) 36.00 <0.00> (ø)
...ticsearch/ad/settings/AnomalyDetectorSettings.java 100.00% <100.00%> (ø) 1.00 <0.00> (ø)
...port/SearchAnomalyDetectorInfoTransportAction.java 54.54% <0.00%> (-9.10%) 4.00% <0.00%> (ø%)
... and 2 more

Copy link
Contributor

@saratvemulapalli saratvemulapalli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the change, didnt realize the update mapping would fail.

ActionListener.wrap(r -> updateRunning.set(false), exception -> logger.error("Fail to updatea all mappings")),
ActionListener.wrap(r -> updateRunning.set(false), exception -> {
// TODO: don't retry endlessly. Can be annoying if there are too many exception logs.
updateRunning.set(false);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If remove this line, will not update index mapping if any exception happens?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if removing this line, we will only try to update mapping once per node.

ActionListener.wrap(r -> updateRunning.set(false), exception -> {
// TODO: don't retry endlessly. Can be annoying if there are too many exception logs.
updateRunning.set(false);
logger.error("Fail to updatea all mappings");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add ad to the error message.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added

@kaituo kaituo added the bug Something isn't working label Nov 9, 2020
Copy link
Contributor

@ylwu-amzn ylwu-amzn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for the change!

@kaituo kaituo merged commit 75db58b into opendistro-for-elasticsearch:master Nov 10, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants