Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backports 2.7 #1482

Merged
merged 15 commits into from
Mar 18, 2024
Merged

Backports 2.7 #1482

merged 15 commits into from
Mar 18, 2024

Conversation

eirsep and others added 10 commits March 18, 2024 12:33
…es or metadata creation fails (opensearch-project#900)

* log errors and clean up monitor when indexing doc level queries or metadata creation fails
* refactor delete monitor action to re-use delete methods
Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>
Signed-off-by: Chase Engelbrecht <engechas@amazon.com>
…oject#1097)

Signed-off-by: Subhobrata Dey <sbcd90@gmail.com>
Signed-off-by: Chase Engelbrecht <engechas@amazon.com>
…arch-project#1302)

* optimize doc-level monitor execution for datastreams

Signed-off-by: Subhobrata Dey <sbcd90@gmail.com>

* add more tests to address comments

Signed-off-by: Subhobrata Dey <sbcd90@gmail.com>

* add integTest for multiple datastreams inside a single index pattern

* add integTest for multiple datastreams inside a single index pattern

Signed-off-by: Subhobrata Dey <sbcd90@gmail.com>

---------

Signed-off-by: Subhobrata Dey <sbcd90@gmail.com>
Signed-off-by: Chase Engelbrecht <engechas@amazon.com>
…arch-project#1355)

* Bulk index findings and sequentially invoke auto-correlations

Signed-off-by: Megha Goyal <goyamegh@amazon.com>

* Bulk index findings in batches of 10000 and make it configurable

Signed-off-by: Megha Goyal <goyamegh@amazon.com>

* Addressing review comments

Signed-off-by: Megha Goyal <goyamegh@amazon.com>

* Add integ tests to test bulk index findings

Signed-off-by: Megha Goyal <goyamegh@amazon.com>

* Fix ktlint formatting

Signed-off-by: Megha Goyal <goyamegh@amazon.com>

---------

Signed-off-by: Megha Goyal <goyamegh@amazon.com>
Signed-off-by: Chase Engelbrecht <engechas@amazon.com>
… percolate queries (opensearch-project#1435)

* add jvm aware and max docs settings for batching docs for percolate queries

Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>

* fix stats logging

Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>

* add queryfieldnames field in findings mapping

Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>

---------

Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>
Signed-off-by: Chase Engelbrecht <engechas@amazon.com>
…vel monitor instead of entire _source for each doc (opensearch-project#1441)

* optimize to fetch only fields relevant to doc level queries in doc level monitor

Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>

* fix test for settings check

Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>

* fix ktlint

Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>

---------

Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>
Signed-off-by: Chase Engelbrecht <engechas@amazon.com>
…c level monitor execution (opensearch-project#1445)

* optimize sequence number calculation and reduce search requests by n where n is number of shards being queried in the executino

Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>

* fix tests

Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>

* optimize check indices and execute to query only write index of aliases and datastreams during monitor creation

Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>

* fix test

Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>

* add javadoc

Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>

* add tests to verify seq_no calculation

Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>

---------

Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>
Signed-off-by: Chase Engelbrecht <engechas@amazon.com>
Signed-off-by: Chase Engelbrecht <engechas@amazon.com>
Signed-off-by: Chase Engelbrecht <engechas@amazon.com>
Signed-off-by: Joanne Wang <jowg@amazon.com>
Signed-off-by: Chase Engelbrecht <engechas@amazon.com>
Signed-off-by: Chase Engelbrecht <engechas@amazon.com>
Signed-off-by: Chase Engelbrecht <engechas@amazon.com>
Signed-off-by: Chase Engelbrecht <engechas@amazon.com>
Signed-off-by: Chase Engelbrecht <engechas@amazon.com>
Signed-off-by: Chase Engelbrecht <engechas@amazon.com>
Copy link

codecov bot commented Mar 18, 2024

Codecov Report

Attention: Patch coverage is 84.45172% with 95 lines in your changes are missing coverage. Please review.

Project coverage is 77.22%. Comparing base (65c3d3a) to head (cb1e89e).
Report is 3 commits behind head on 2.7.

Files Patch % Lines
.../opensearch/alerting/DocumentLevelMonitorRunner.kt 83.67% 31 Missing and 25 partials ⚠️
...opensearch/alerting/util/DocLevelMonitorQueries.kt 84.75% 7 Missing and 18 partials ⚠️
.../org/opensearch/alerting/MonitorMetadataService.kt 37.50% 0 Missing and 5 partials ⚠️
.../kotlin/org/opensearch/alerting/util/IndexUtils.kt 72.22% 1 Missing and 4 partials ⚠️
...in/org/opensearch/alerting/MonitorRunnerService.kt 90.90% 2 Missing ⚠️
.../alerting/transport/TransportIndexMonitorAction.kt 66.66% 0 Missing and 2 partials ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##                2.7    #1482      +/-   ##
============================================
+ Coverage     76.65%   77.22%   +0.56%     
  Complexity      116      116              
============================================
  Files           125      125              
  Lines          7257     7684     +427     
  Branches       1050     1122      +72     
============================================
+ Hits           5563     5934     +371     
- Misses         1160     1179      +19     
- Partials        534      571      +37     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@engechas engechas merged commit 20bd240 into opensearch-project:2.7 Mar 18, 2024
14 of 16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants