-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bugfix] Fix flaky DefaultCacheStatsHolderTests #14462
[Bugfix] Fix flaky DefaultCacheStatsHolderTests #14462
Conversation
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
❌ Gradle check result for d8f283b: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Flaky test: #14292 |
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
server/src/test/java/org/opensearch/common/cache/stats/DefaultCacheStatsHolderTests.java
Outdated
Show resolved
Hide resolved
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
❌ Gradle check result for 406f8e2: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
❌ Gradle check result for 32e87e4: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
server/src/test/java/org/opensearch/common/cache/stats/DefaultCacheStatsHolderTests.java
Show resolved
Hide resolved
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #14462 +/- ##
============================================
+ Coverage 71.42% 71.61% +0.19%
- Complexity 59978 61965 +1987
============================================
Files 4985 5117 +132
Lines 282275 291712 +9437
Branches 40946 42168 +1222
============================================
+ Hits 201603 208898 +7295
- Misses 63999 65573 +1574
- Partials 16673 17241 +568 ☔ View full report in Codecov by Sentry. |
Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> (cherry picked from commit 0eb39ae) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
(cherry picked from commit 0eb39ae) Signed-off-by: Peter Alfonsi <petealft@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com>
* Fix flaky test in range aggregation yaml test (opensearch-project#14486) Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * Use CODECOV_TOKEN (opensearch-project#14536) Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * [Tiered Caching] Moving query recomputation logic outside of write lock (opensearch-project#14187) * Moving query recompute out of write lock Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * [Tiered Caching] Moving query recomputation logic outside of write lock Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding java doc for the completable map Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Changes to call future handler only once per key Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing spotless check Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Added changelog Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comments Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing gradle fail Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comments to refactor unit test Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * minor UT refactor Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com> * Fix Flaky Test ClusterRerouteIT.testDelayWithALargeAmountOfShards (opensearch-project#14510) Signed-off-by: kkewwei kkewwei@163.com Signed-off-by: kkewwei kkewwei@163.com Signed-off-by: kkewwei <kkewwei@163.com> * Add doc for debugging rest tests (opensearch-project#14491) * add doc for debugging rest tests Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * Update TESTING.md Co-authored-by: Marc Handalian <handalm@amazon.com> Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * Address comment Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> --------- Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> Co-authored-by: Marc Handalian <handalm@amazon.com> * Fix flaky DefaultCacheStatsHolderTests (opensearch-project#14462) Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> * Startreemapping (#24) * Star tree mapping changes with feature flag Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * Fixes and tests Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * addressing review comments and adding validations, integ tests Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * addressing review comments Signed-off-by: Bharathwaj G <bharath78910@gmail.com> --------- Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * OnHeap Star Tree Implementation Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * addressed nits Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * addressed major nits Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * includes Count Aggregator Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> --------- Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> Signed-off-by: kkewwei kkewwei@163.com Signed-off-by: kkewwei <kkewwei@163.com> Signed-off-by: Peter Alfonsi <petealft@amazon.com> Signed-off-by: Bharathwaj G <bharath78910@gmail.com> Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com> Co-authored-by: Prudhvi Godithi <pgodithi@amazon.com> Co-authored-by: Sagar <99425694+sgup432@users.noreply.github.com> Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com> Co-authored-by: kkewwei <kkewwei@163.com> Co-authored-by: Marc Handalian <handalm@amazon.com> Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Sarthak Aggarwal <sarthagg@amazon.com>
* Startreemapping (#24) * Star tree mapping changes with feature flag Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * Fixes and tests Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * addressing review comments and adding validations, integ tests Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * addressing review comments Signed-off-by: Bharathwaj G <bharath78910@gmail.com> --------- Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * Onheap impl startree (#25) * Fix flaky test in range aggregation yaml test (opensearch-project#14486) Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * Use CODECOV_TOKEN (opensearch-project#14536) Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * [Tiered Caching] Moving query recomputation logic outside of write lock (opensearch-project#14187) * Moving query recompute out of write lock Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * [Tiered Caching] Moving query recomputation logic outside of write lock Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding java doc for the completable map Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Changes to call future handler only once per key Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing spotless check Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Added changelog Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comments Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing gradle fail Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comments to refactor unit test Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * minor UT refactor Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com> * Fix Flaky Test ClusterRerouteIT.testDelayWithALargeAmountOfShards (opensearch-project#14510) Signed-off-by: kkewwei kkewwei@163.com Signed-off-by: kkewwei kkewwei@163.com Signed-off-by: kkewwei <kkewwei@163.com> * Add doc for debugging rest tests (opensearch-project#14491) * add doc for debugging rest tests Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * Update TESTING.md Co-authored-by: Marc Handalian <handalm@amazon.com> Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * Address comment Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> --------- Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> Co-authored-by: Marc Handalian <handalm@amazon.com> * Fix flaky DefaultCacheStatsHolderTests (opensearch-project#14462) Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> * Startreemapping (#24) * Star tree mapping changes with feature flag Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * Fixes and tests Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * addressing review comments and adding validations, integ tests Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * addressing review comments Signed-off-by: Bharathwaj G <bharath78910@gmail.com> --------- Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * OnHeap Star Tree Implementation Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * addressed nits Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * addressed major nits Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * includes Count Aggregator Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> --------- Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> Signed-off-by: kkewwei kkewwei@163.com Signed-off-by: kkewwei <kkewwei@163.com> Signed-off-by: Peter Alfonsi <petealft@amazon.com> Signed-off-by: Bharathwaj G <bharath78910@gmail.com> Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com> Co-authored-by: Prudhvi Godithi <pgodithi@amazon.com> Co-authored-by: Sagar <99425694+sgup432@users.noreply.github.com> Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com> Co-authored-by: kkewwei <kkewwei@163.com> Co-authored-by: Marc Handalian <handalm@amazon.com> Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Sarthak Aggarwal <sarthagg@amazon.com> --------- Signed-off-by: Bharathwaj G <bharath78910@gmail.com> Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> Signed-off-by: kkewwei kkewwei@163.com Signed-off-by: kkewwei <kkewwei@163.com> Signed-off-by: Peter Alfonsi <petealft@amazon.com> Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com> Co-authored-by: Prudhvi Godithi <pgodithi@amazon.com> Co-authored-by: Sagar <99425694+sgup432@users.noreply.github.com> Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com> Co-authored-by: kkewwei <kkewwei@163.com> Co-authored-by: Marc Handalian <handalm@amazon.com> Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Sarthak Aggarwal <sarthagg@amazon.com>
* Fix flaky test in range aggregation yaml test (opensearch-project#14486) Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * Use CODECOV_TOKEN (opensearch-project#14536) Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * [Tiered Caching] Moving query recomputation logic outside of write lock (opensearch-project#14187) * Moving query recompute out of write lock Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * [Tiered Caching] Moving query recomputation logic outside of write lock Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding java doc for the completable map Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Changes to call future handler only once per key Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing spotless check Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Added changelog Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comments Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing gradle fail Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comments to refactor unit test Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * minor UT refactor Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com> * Fix Flaky Test ClusterRerouteIT.testDelayWithALargeAmountOfShards (opensearch-project#14510) Signed-off-by: kkewwei kkewwei@163.com Signed-off-by: kkewwei kkewwei@163.com Signed-off-by: kkewwei <kkewwei@163.com> * Add doc for debugging rest tests (opensearch-project#14491) * add doc for debugging rest tests Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * Update TESTING.md Co-authored-by: Marc Handalian <handalm@amazon.com> Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * Address comment Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> --------- Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> Co-authored-by: Marc Handalian <handalm@amazon.com> * Fix flaky DefaultCacheStatsHolderTests (opensearch-project#14462) Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> * [AUTO] [main] Add bwc version 2.15.1. (opensearch-project#14549) * Add bwc version 2.15.1 Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Fix auto-generated version Signed-off-by: Andrew Ross <andrross@amazon.com> --------- Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Andrew Ross <andrross@amazon.com> Co-authored-by: opensearch-ci-bot <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: Andrew Ross <andrross@amazon.com> * Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently (opensearch-project#14550) * Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comment Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com> Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com> * Add allowlist setting for ingest-common processors (opensearch-project#14479) Add a new static setting that lets an operator choose specific ingest processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <andrross@amazon.com> * Fix file cache initialization (opensearch-project#14004) * fix file cache initialization Signed-off-by: panguixin <panguixin@bytedance.com> * changelog Signed-off-by: panguixin <panguixin@bytedance.com> * add test Signed-off-by: panguixin <panguixin@bytedance.com> --------- Signed-off-by: panguixin <panguixin@bytedance.com> * Add Ashish Singh as maintainer (opensearch-project#14567) Signed-off-by: Bukhtawar Khan <bukhtawa@amazon.com> * Allow @internalapi annotation on classes not meant to be constructed outside of the OpenSearch core (opensearch-project#14575) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Bump com.azure:azure-storage-common from 12.21.2 to 12.25.1 in /plugins/repository-azure (opensearch-project#14517) * Bump com.azure:azure-storage-common in /plugins/repository-azure Bumps [com.azure:azure-storage-common](https://github.com/Azure/azure-sdk-for-java) from 12.21.2 to 12.25.1. - [Release notes](https://github.com/Azure/azure-sdk-for-java/releases) - [Commits](Azure/azure-sdk-for-java@azure-storage-common_12.21.2...azure-storage-blob_12.25.1) --- updated-dependencies: - dependency-name: com.azure:azure-storage-common dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> --------- Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * Add allowlist setting for search-pipeline-common processors (opensearch-project#14562) Add a new static setting that lets an operator choose specific search pipeline processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. A distinct setting exists for each of request, response, and search phase results processors. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <andrross@amazon.com> * Bump Apache Lucene to 9.11.1 (opensearch-project#14576) (opensearch-project#14581) (cherry picked from commit 0095fd1) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Add unittests for RemoteClusterStateAttributesManager (opensearch-project#14427) * Add unittests for RemoteClusterStateAttributesManager Signed-off-by: Shivansh Arora <hishiv@amazon.com> * Add Ashish Singh to codeowners (opensearch-project#14592) Signed-off-by: Ashish Singh <ssashish@amazon.com> * Add batching processor base type AbstractBatchingProcessor (opensearch-project#14554) Signed-off-by: Liyun Xiu <xiliyun@amazon.com> * Add @internalapi annotation to japicmp exclusions (opensearch-project#14597) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Fix issue 14519:Parsing a GetResult returns NPE if found field is mis… (opensearch-project#14552) * Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Fix issue 14519:Fix spotless issues. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Fix issue 14519:update changelog Signed-off-by: vatsal <vatsal.v.anand@gmail.com> --------- Signed-off-by: vatsal <vatsal.v.anand@gmail.com> Signed-off-by: Daniel Widdis <widdis@gmail.com> Co-authored-by: Daniel Widdis <widdis@gmail.com> * Star tree mapping changes (opensearch-project#14605) * Star tree mapping changes with feature flag --------- Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (opensearch-project#14610) * Bump com.microsoft.azure:msal4j in /plugins/repository-azure Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0. - [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases) - [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt) - [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0) --- updated-dependencies: - dependency-name: com.microsoft.azure:msal4j dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * [Bugfix] Fix ICacheKeySerializerTests flakiness (opensearch-project#14564) * Fix testInvalidInput flakiness Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Addressed andrross's comment Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun security check Signed-off-by: Peter Alfonsi <petealft@amazon.com> --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> * Correct typo in method name (opensearch-project#14621) Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Refactoring FilterPath.parse by using an iterative approach instead of recursion. (opensearch-project#14200) * Refactor FilterPath parse function (opensearch-project#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Implement unit tests for FilterPathTests (opensearch-project#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Write warn log if Filter is empty; Add comments (opensearch-project#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Add changelog Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove unnecessary log statement Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove unused logger Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Spotless apply Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove incorrect changelog Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> --------- Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(opensearch-project#14612) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> * Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (opensearch-project#14513) * Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata * Correct the check for deciding upload of hashes of consistent settings Signed-off-by: Sooraj Sinha <soosinha@amazon.com> * Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (opensearch-project#14613) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> * Fix assertion failure while deleting remote backed index (opensearch-project#14601) Signed-off-by: Sachin Kale <kalsac@amazon.com> * OnHeap Star Tree Implementation Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * addressed nits Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * addressed major nits Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * includes Count Aggregator Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * handling for missing doc values Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * addressing review comments Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * rebasing with main Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * support for empty sequential doc values iterator Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * nits Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * min and max star tree aggregators Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * star tree file formats * Star tree codec changes Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * Adding tests Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * Addressing comments Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * addressing review comments Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * Star tree merge changes Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * fix annotations * star-tree file formats reader and javadoc fixes * read for composite index values Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * doc values file format Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> --------- Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> Signed-off-by: kkewwei kkewwei@163.com Signed-off-by: kkewwei <kkewwei@163.com> Signed-off-by: Peter Alfonsi <petealft@amazon.com> Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Andrew Ross <andrross@amazon.com> Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com> Signed-off-by: panguixin <panguixin@bytedance.com> Signed-off-by: Bukhtawar Khan <bukhtawa@amazon.com> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Shivansh Arora <hishiv@amazon.com> Signed-off-by: Ashish Singh <ssashish@amazon.com> Signed-off-by: Liyun Xiu <xiliyun@amazon.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> Signed-off-by: Daniel Widdis <widdis@gmail.com> Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Sooraj Sinha <soosinha@amazon.com> Signed-off-by: Sachin Kale <kalsac@amazon.com> Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> Signed-off-by: Bharathwaj G <bharath78910@gmail.com> Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com> Co-authored-by: Prudhvi Godithi <pgodithi@amazon.com> Co-authored-by: Sagar <99425694+sgup432@users.noreply.github.com> Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com> Co-authored-by: kkewwei <kkewwei@163.com> Co-authored-by: Marc Handalian <handalm@amazon.com> Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: Andrew Ross <andrross@amazon.com> Co-authored-by: panguixin <panguixin@bytedance.com> Co-authored-by: Bukhtawar Khan <bukhtawa@amazon.com> Co-authored-by: Andriy Redko <andriy.redko@aiven.io> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Co-authored-by: Shivansh Arora <hishiv@amazon.com> Co-authored-by: Ashish Singh <ssashish@amazon.com> Co-authored-by: Liyun Xiu <xiliyun@amazon.com> Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com> Co-authored-by: Daniel Widdis <widdis@gmail.com> Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com> Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com> Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com> Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com> Co-authored-by: Sachin Kale <sachinpkale@gmail.com> Co-authored-by: Sarthak Aggarwal <sarthagg@amazon.com>
* Fix flaky test in range aggregation yaml test (opensearch-project#14486) Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * Use CODECOV_TOKEN (opensearch-project#14536) Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * [Tiered Caching] Moving query recomputation logic outside of write lock (opensearch-project#14187) * Moving query recompute out of write lock Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * [Tiered Caching] Moving query recomputation logic outside of write lock Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding java doc for the completable map Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Changes to call future handler only once per key Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing spotless check Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Added changelog Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comments Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing gradle fail Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comments to refactor unit test Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * minor UT refactor Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com> * Fix Flaky Test ClusterRerouteIT.testDelayWithALargeAmountOfShards (opensearch-project#14510) Signed-off-by: kkewwei kkewwei@163.com Signed-off-by: kkewwei kkewwei@163.com Signed-off-by: kkewwei <kkewwei@163.com> * Add doc for debugging rest tests (opensearch-project#14491) * add doc for debugging rest tests Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * Update TESTING.md Co-authored-by: Marc Handalian <handalm@amazon.com> Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * Address comment Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> --------- Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> Co-authored-by: Marc Handalian <handalm@amazon.com> * Fix flaky DefaultCacheStatsHolderTests (opensearch-project#14462) Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> * [AUTO] [main] Add bwc version 2.15.1. (opensearch-project#14549) * Add bwc version 2.15.1 Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Fix auto-generated version Signed-off-by: Andrew Ross <andrross@amazon.com> --------- Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Andrew Ross <andrross@amazon.com> Co-authored-by: opensearch-ci-bot <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: Andrew Ross <andrross@amazon.com> * Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently (opensearch-project#14550) * Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comment Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com> Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com> * Add allowlist setting for ingest-common processors (opensearch-project#14479) Add a new static setting that lets an operator choose specific ingest processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <andrross@amazon.com> * Fix file cache initialization (opensearch-project#14004) * fix file cache initialization Signed-off-by: panguixin <panguixin@bytedance.com> * changelog Signed-off-by: panguixin <panguixin@bytedance.com> * add test Signed-off-by: panguixin <panguixin@bytedance.com> --------- Signed-off-by: panguixin <panguixin@bytedance.com> * Add Ashish Singh as maintainer (opensearch-project#14567) Signed-off-by: Bukhtawar Khan <bukhtawa@amazon.com> * Allow @internalapi annotation on classes not meant to be constructed outside of the OpenSearch core (opensearch-project#14575) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Bump com.azure:azure-storage-common from 12.21.2 to 12.25.1 in /plugins/repository-azure (opensearch-project#14517) * Bump com.azure:azure-storage-common in /plugins/repository-azure Bumps [com.azure:azure-storage-common](https://github.com/Azure/azure-sdk-for-java) from 12.21.2 to 12.25.1. - [Release notes](https://github.com/Azure/azure-sdk-for-java/releases) - [Commits](Azure/azure-sdk-for-java@azure-storage-common_12.21.2...azure-storage-blob_12.25.1) --- updated-dependencies: - dependency-name: com.azure:azure-storage-common dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> --------- Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * Add allowlist setting for search-pipeline-common processors (opensearch-project#14562) Add a new static setting that lets an operator choose specific search pipeline processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. A distinct setting exists for each of request, response, and search phase results processors. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <andrross@amazon.com> * Bump Apache Lucene to 9.11.1 (opensearch-project#14576) (opensearch-project#14581) (cherry picked from commit 0095fd1) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Add unittests for RemoteClusterStateAttributesManager (opensearch-project#14427) * Add unittests for RemoteClusterStateAttributesManager Signed-off-by: Shivansh Arora <hishiv@amazon.com> * Add Ashish Singh to codeowners (opensearch-project#14592) Signed-off-by: Ashish Singh <ssashish@amazon.com> * Add batching processor base type AbstractBatchingProcessor (opensearch-project#14554) Signed-off-by: Liyun Xiu <xiliyun@amazon.com> * Add @internalapi annotation to japicmp exclusions (opensearch-project#14597) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Fix issue 14519:Parsing a GetResult returns NPE if found field is mis… (opensearch-project#14552) * Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Fix issue 14519:Fix spotless issues. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Fix issue 14519:update changelog Signed-off-by: vatsal <vatsal.v.anand@gmail.com> --------- Signed-off-by: vatsal <vatsal.v.anand@gmail.com> Signed-off-by: Daniel Widdis <widdis@gmail.com> Co-authored-by: Daniel Widdis <widdis@gmail.com> * Star tree mapping changes (opensearch-project#14605) * Star tree mapping changes with feature flag --------- Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (opensearch-project#14610) * Bump com.microsoft.azure:msal4j in /plugins/repository-azure Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0. - [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases) - [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt) - [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0) --- updated-dependencies: - dependency-name: com.microsoft.azure:msal4j dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * [Bugfix] Fix ICacheKeySerializerTests flakiness (opensearch-project#14564) * Fix testInvalidInput flakiness Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Addressed andrross's comment Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun security check Signed-off-by: Peter Alfonsi <petealft@amazon.com> --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> * Correct typo in method name (opensearch-project#14621) Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Refactoring FilterPath.parse by using an iterative approach instead of recursion. (opensearch-project#14200) * Refactor FilterPath parse function (opensearch-project#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Implement unit tests for FilterPathTests (opensearch-project#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Write warn log if Filter is empty; Add comments (opensearch-project#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Add changelog Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove unnecessary log statement Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove unused logger Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Spotless apply Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove incorrect changelog Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> --------- Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(opensearch-project#14612) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> * Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (opensearch-project#14513) * Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata * Correct the check for deciding upload of hashes of consistent settings Signed-off-by: Sooraj Sinha <soosinha@amazon.com> * Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (opensearch-project#14613) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> * Fix assertion failure while deleting remote backed index (opensearch-project#14601) Signed-off-by: Sachin Kale <kalsac@amazon.com> * OnHeap Star Tree Implementation Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * addressed nits Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * addressed major nits Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * includes Count Aggregator Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * handling for missing doc values Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * addressing review comments Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * rebasing with main Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * support for empty sequential doc values iterator Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * nits Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * min and max star tree aggregators Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * star tree file formats * Star tree codec changes Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * Adding tests Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * Addressing comments Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * addressing review comments Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * Star tree merge changes Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * fix annotations * star-tree file formats reader and javadoc fixes * read for composite index values Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * doc values file format Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> --------- Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> Signed-off-by: kkewwei kkewwei@163.com Signed-off-by: kkewwei <kkewwei@163.com> Signed-off-by: Peter Alfonsi <petealft@amazon.com> Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Andrew Ross <andrross@amazon.com> Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com> Signed-off-by: panguixin <panguixin@bytedance.com> Signed-off-by: Bukhtawar Khan <bukhtawa@amazon.com> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Shivansh Arora <hishiv@amazon.com> Signed-off-by: Ashish Singh <ssashish@amazon.com> Signed-off-by: Liyun Xiu <xiliyun@amazon.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> Signed-off-by: Daniel Widdis <widdis@gmail.com> Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Sooraj Sinha <soosinha@amazon.com> Signed-off-by: Sachin Kale <kalsac@amazon.com> Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> Signed-off-by: Bharathwaj G <bharath78910@gmail.com> Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com> Co-authored-by: Prudhvi Godithi <pgodithi@amazon.com> Co-authored-by: Sagar <99425694+sgup432@users.noreply.github.com> Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com> Co-authored-by: kkewwei <kkewwei@163.com> Co-authored-by: Marc Handalian <handalm@amazon.com> Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: Andrew Ross <andrross@amazon.com> Co-authored-by: panguixin <panguixin@bytedance.com> Co-authored-by: Bukhtawar Khan <bukhtawa@amazon.com> Co-authored-by: Andriy Redko <andriy.redko@aiven.io> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Co-authored-by: Shivansh Arora <hishiv@amazon.com> Co-authored-by: Ashish Singh <ssashish@amazon.com> Co-authored-by: Liyun Xiu <xiliyun@amazon.com> Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com> Co-authored-by: Daniel Widdis <widdis@gmail.com> Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com> Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com> Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com> Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com> Co-authored-by: Sachin Kale <sachinpkale@gmail.com> Co-authored-by: Sarthak Aggarwal <sarthagg@amazon.com>
* Fix flaky test in range aggregation yaml test (opensearch-project#14486) Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * Use CODECOV_TOKEN (opensearch-project#14536) Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * [Tiered Caching] Moving query recomputation logic outside of write lock (opensearch-project#14187) * Moving query recompute out of write lock Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * [Tiered Caching] Moving query recomputation logic outside of write lock Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding java doc for the completable map Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Changes to call future handler only once per key Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing spotless check Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Added changelog Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comments Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing gradle fail Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comments to refactor unit test Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * minor UT refactor Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com> * Fix Flaky Test ClusterRerouteIT.testDelayWithALargeAmountOfShards (opensearch-project#14510) Signed-off-by: kkewwei kkewwei@163.com Signed-off-by: kkewwei kkewwei@163.com Signed-off-by: kkewwei <kkewwei@163.com> * Add doc for debugging rest tests (opensearch-project#14491) * add doc for debugging rest tests Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * Update TESTING.md Co-authored-by: Marc Handalian <handalm@amazon.com> Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * Address comment Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> --------- Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> Co-authored-by: Marc Handalian <handalm@amazon.com> * Fix flaky DefaultCacheStatsHolderTests (opensearch-project#14462) Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> * [AUTO] [main] Add bwc version 2.15.1. (opensearch-project#14549) * Add bwc version 2.15.1 Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Fix auto-generated version Signed-off-by: Andrew Ross <andrross@amazon.com> --------- Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Andrew Ross <andrross@amazon.com> Co-authored-by: opensearch-ci-bot <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: Andrew Ross <andrross@amazon.com> * Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently (opensearch-project#14550) * Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comment Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com> Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com> * Add allowlist setting for ingest-common processors (opensearch-project#14479) Add a new static setting that lets an operator choose specific ingest processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <andrross@amazon.com> * Fix file cache initialization (opensearch-project#14004) * fix file cache initialization Signed-off-by: panguixin <panguixin@bytedance.com> * changelog Signed-off-by: panguixin <panguixin@bytedance.com> * add test Signed-off-by: panguixin <panguixin@bytedance.com> --------- Signed-off-by: panguixin <panguixin@bytedance.com> * Add Ashish Singh as maintainer (opensearch-project#14567) Signed-off-by: Bukhtawar Khan <bukhtawa@amazon.com> * Allow @internalapi annotation on classes not meant to be constructed outside of the OpenSearch core (opensearch-project#14575) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Bump com.azure:azure-storage-common from 12.21.2 to 12.25.1 in /plugins/repository-azure (opensearch-project#14517) * Bump com.azure:azure-storage-common in /plugins/repository-azure Bumps [com.azure:azure-storage-common](https://github.com/Azure/azure-sdk-for-java) from 12.21.2 to 12.25.1. - [Release notes](https://github.com/Azure/azure-sdk-for-java/releases) - [Commits](Azure/azure-sdk-for-java@azure-storage-common_12.21.2...azure-storage-blob_12.25.1) --- updated-dependencies: - dependency-name: com.azure:azure-storage-common dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> --------- Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * Add allowlist setting for search-pipeline-common processors (opensearch-project#14562) Add a new static setting that lets an operator choose specific search pipeline processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. A distinct setting exists for each of request, response, and search phase results processors. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <andrross@amazon.com> * Bump Apache Lucene to 9.11.1 (opensearch-project#14576) (opensearch-project#14581) (cherry picked from commit 0095fd1) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Add unittests for RemoteClusterStateAttributesManager (opensearch-project#14427) * Add unittests for RemoteClusterStateAttributesManager Signed-off-by: Shivansh Arora <hishiv@amazon.com> * Add Ashish Singh to codeowners (opensearch-project#14592) Signed-off-by: Ashish Singh <ssashish@amazon.com> * Add batching processor base type AbstractBatchingProcessor (opensearch-project#14554) Signed-off-by: Liyun Xiu <xiliyun@amazon.com> * Add @internalapi annotation to japicmp exclusions (opensearch-project#14597) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Fix issue 14519:Parsing a GetResult returns NPE if found field is mis… (opensearch-project#14552) * Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Fix issue 14519:Fix spotless issues. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Fix issue 14519:update changelog Signed-off-by: vatsal <vatsal.v.anand@gmail.com> --------- Signed-off-by: vatsal <vatsal.v.anand@gmail.com> Signed-off-by: Daniel Widdis <widdis@gmail.com> Co-authored-by: Daniel Widdis <widdis@gmail.com> * Star tree mapping changes (opensearch-project#14605) * Star tree mapping changes with feature flag --------- Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (opensearch-project#14610) * Bump com.microsoft.azure:msal4j in /plugins/repository-azure Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0. - [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases) - [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt) - [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0) --- updated-dependencies: - dependency-name: com.microsoft.azure:msal4j dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * [Bugfix] Fix ICacheKeySerializerTests flakiness (opensearch-project#14564) * Fix testInvalidInput flakiness Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Addressed andrross's comment Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun security check Signed-off-by: Peter Alfonsi <petealft@amazon.com> --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> * Correct typo in method name (opensearch-project#14621) Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Refactoring FilterPath.parse by using an iterative approach instead of recursion. (opensearch-project#14200) * Refactor FilterPath parse function (opensearch-project#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Implement unit tests for FilterPathTests (opensearch-project#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Write warn log if Filter is empty; Add comments (opensearch-project#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Add changelog Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove unnecessary log statement Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove unused logger Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Spotless apply Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove incorrect changelog Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> --------- Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(opensearch-project#14612) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> * Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (opensearch-project#14513) * Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata * Correct the check for deciding upload of hashes of consistent settings Signed-off-by: Sooraj Sinha <soosinha@amazon.com> * Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (opensearch-project#14613) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> * Fix assertion failure while deleting remote backed index (opensearch-project#14601) Signed-off-by: Sachin Kale <kalsac@amazon.com> * OnHeap Star Tree Implementation Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * addressed nits Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * addressed major nits Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * includes Count Aggregator Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * handling for missing doc values Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * addressing review comments Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * rebasing with main Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * support for empty sequential doc values iterator Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * nits Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * min and max star tree aggregators Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * star tree file formats * Star tree codec changes Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * Adding tests Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * Addressing comments Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * addressing review comments Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * Star tree merge changes Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * fix annotations * star-tree file formats reader and javadoc fixes * read for composite index values Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * doc values file format Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> --------- Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> Signed-off-by: kkewwei kkewwei@163.com Signed-off-by: kkewwei <kkewwei@163.com> Signed-off-by: Peter Alfonsi <petealft@amazon.com> Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Andrew Ross <andrross@amazon.com> Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com> Signed-off-by: panguixin <panguixin@bytedance.com> Signed-off-by: Bukhtawar Khan <bukhtawa@amazon.com> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Shivansh Arora <hishiv@amazon.com> Signed-off-by: Ashish Singh <ssashish@amazon.com> Signed-off-by: Liyun Xiu <xiliyun@amazon.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> Signed-off-by: Daniel Widdis <widdis@gmail.com> Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Sooraj Sinha <soosinha@amazon.com> Signed-off-by: Sachin Kale <kalsac@amazon.com> Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> Signed-off-by: Bharathwaj G <bharath78910@gmail.com> Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com> Co-authored-by: Prudhvi Godithi <pgodithi@amazon.com> Co-authored-by: Sagar <99425694+sgup432@users.noreply.github.com> Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com> Co-authored-by: kkewwei <kkewwei@163.com> Co-authored-by: Marc Handalian <handalm@amazon.com> Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: Andrew Ross <andrross@amazon.com> Co-authored-by: panguixin <panguixin@bytedance.com> Co-authored-by: Bukhtawar Khan <bukhtawa@amazon.com> Co-authored-by: Andriy Redko <andriy.redko@aiven.io> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Co-authored-by: Shivansh Arora <hishiv@amazon.com> Co-authored-by: Ashish Singh <ssashish@amazon.com> Co-authored-by: Liyun Xiu <xiliyun@amazon.com> Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com> Co-authored-by: Daniel Widdis <widdis@gmail.com> Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com> Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com> Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com> Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com> Co-authored-by: Sachin Kale <sachinpkale@gmail.com> Co-authored-by: Sarthak Aggarwal <sarthagg@amazon.com>
Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com>
…ensearch-project#14542) (cherry picked from commit 0eb39ae) Signed-off-by: Peter Alfonsi <petealft@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> Signed-off-by: kkewwei <kkewwei@163.com>
Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com>
commit 93a3132 Author: Peter Alfonsi <petealft@amazon.com> Date: Wed Sep 4 11:44:03 2024 -0700 fix broken ITs Signed-off-by: Peter Alfonsi <petealft@amazon.com> commit fd92c5a Author: Marc Handalian <handalm@amazon.com> Date: Thu Jan 11 06:41:55 2024 -0800 Update runTask to optionally install plugins (opensearch-project#11844) Signed-off-by: Marc Handalian <marc.handalian@gmail.com> commit 4d78661 Author: Peter Alfonsi <petealft@amazon.com> Date: Tue Sep 3 16:02:31 2024 -0700 Misc test fixes Signed-off-by: Peter Alfonsi <petealft@amazon.com> commit 70a6935 Author: Peter Alfonsi <peter.alfonsi@gmail.com> Date: Fri Aug 23 10:42:05 2024 -0700 [Bugfix] Fixes IRC NPE bug for timed-out cacheable queries (opensearch-project#15327) * Fix IRC timeout bug Signed-off-by: Peter Alfonsi <petealft@amazon.com> * addressed Sagar's comments Signed-off-by: Peter Alfonsi <petealft@amazon.com> * addressed Ankit's comments Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Add UT for test coverage Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun gradle Signed-off-by: Peter Alfonsi <petealft@amazon.com> * tweak imports in new UT Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun gradle Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun gradle Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun gradle Signed-off-by: Peter Alfonsi <petealft@amazon.com> --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> commit f9943f8 Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Wed Jul 17 19:02:05 2024 -0700 Clear ehcache disk cache files during initialization (opensearch-project#14738) * Clear ehcache disk cache files during initialization Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding UT to fix line coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comment Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding more Uts for better line coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Throwing exception in case we fail to clear cache files during startup Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding more UTs Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding a UT for more coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing gradle build Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Update ehcache disk cache close() logic Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> commit fc2007e Author: Peter Alfonsi <peter.alfonsi@gmail.com> Date: Mon Jul 1 15:54:39 2024 -0700 [Bugfix] Fix ICacheKeySerializerTests flakiness (opensearch-project#14564) * Fix testInvalidInput flakiness Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Addressed andrross's comment Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun security check Signed-off-by: Peter Alfonsi <petealft@amazon.com> --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> commit 1ef62e2 Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Wed Jun 26 11:14:42 2024 -0700 Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently (opensearch-project#14550) * Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comment Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com> Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com> commit 29ec9d6 Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Tue Jun 25 12:04:17 2024 -0700 [Tiered Caching] Moving query recomputation logic outside of write lock (opensearch-project#14187) * Moving query recompute out of write lock Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * [Tiered Caching] Moving query recomputation logic outside of write lock Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding java doc for the completable map Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Changes to call future handler only once per key Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing spotless check Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Added changelog Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comments Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing gradle fail Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comments to refactor unit test Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * minor UT refactor Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com> commit 8347779 Author: Peter Alfonsi <peter.alfonsi@gmail.com> Date: Tue Jun 25 13:26:54 2024 -0700 Fix flaky DefaultCacheStatsHolderTests (opensearch-project#14462) Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> commit 40f868d Author: Kiran Prakash <awskiran@amazon.com> Date: Thu Jun 20 14:17:23 2024 -0700 Update IndicesRequestCacheCleanupIT.java (opensearch-project#14478) Signed-off-by: Kiran Prakash <awskiran@amazon.com> commit 094a9f2 Author: Kiran Prakash <awskiran@amazon.com> Date: Wed Jun 19 15:36:59 2024 -0700 Fix Indices Request Cache ITs from flaking and timing out (opensearch-project#14369) Signed-off-by: Kiran Prakash <awskiran@amazon.com> commit 3a110a8 Author: Kiran Prakash <awskiran@amazon.com> Date: Tue Jun 18 10:18:25 2024 -0700 [Tiered Cache] Use ConcurrentHashMap explicitly in IndicesRequestCache (opensearch-project#14409) Signed-off-by: Kiran Prakash <awskiran@amazon.com> commit bcf0f59 Author: Peter Alfonsi <peter.alfonsi@gmail.com> Date: Fri Jun 14 13:20:51 2024 -0700 [Bugfix] Fix TieredSpilloverCache flaky tests (opensearch-project#14333) * Fix flaky TSC stats tests Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Addressed andrross's comment Signed-off-by: Peter Alfonsi <petealft@amazon.com> * fix forbidden API Signed-off-by: Peter Alfonsi <petealft@amazon.com> --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> commit a222a03 Author: Kiran Prakash <awskiran@amazon.com> Date: Wed Jun 12 18:30:06 2024 -0700 [Tiered Caching] [Bug Fix] Use concurrentMap instead of HashMap to fix Concurrent Modification Exception (opensearch-project#14221) * use concurrentmap Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update CHANGELOG.md Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCache.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * revert feature flags Signed-off-by: Kiran Prakash <awskiran@amazon.com> * changelog to releaselog Signed-off-by: Kiran Prakash <awskiran@amazon.com> * use concurrentmap Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update CHANGELOG.md Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCache.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * revert feature flags Signed-off-by: Kiran Prakash <awskiran@amazon.com> * changelog to releaselog Signed-off-by: Kiran Prakash <awskiran@amazon.com> * revert the test removal Signed-off-by: Kiran Prakash <awskiran@amazon.com> * revert the conflict resolutions Signed-off-by: Kiran Prakash <awskiran@amazon.com> --------- Signed-off-by: Kiran Prakash <awskiran@amazon.com> commit f85cd00 Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Wed Jun 12 08:22:38 2024 -0700 Fix ShardNotFoundException during request cache clean up (opensearch-project#14219) * Fix for ShardNotFoundException during request cache clean up Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Added changelog Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fix forbidden gradle check Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> commit 406ae78 Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Thu Jun 6 15:02:00 2024 -0700 [Caching] Move cache removal notifications outside lru lock (opensearch-project#14017) --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> commit c5fd7fe Author: Peter Alfonsi <petealft@amazon.com> Date: Tue Sep 3 14:38:26 2024 -0700 Change IRC UT to correctly create shard Signed-off-by: Peter Alfonsi <petealft@amazon.com> commit 9c4d4af Author: Peter Alfonsi <peter.alfonsi@gmail.com> Date: Tue Jun 4 16:09:18 2024 -0700 [Tiered Caching] Additional ITs for cache stats (opensearch-project#13655) * Adds cache clear IT Signed-off-by: Peter Alfonsi <petealft@amazon.com> Cleaned up logic for cache stats ITs Signed-off-by: Peter Alfonsi <petealft@amazon.com> Adds more tests around tiered spillover cache Signed-off-by: Peter Alfonsi <petealft@amazon.com> Fixed cache stats behavior for overall /_nodes/stats call Signed-off-by: Peter Alfonsi <petealft@amazon.com> cleanup Signed-off-by: Peter Alfonsi <petealft@amazon.com> Fixed folder structure Signed-off-by: Peter Alfonsi <petealft@amazon.com> Addressed Sagar's comments Signed-off-by: Peter Alfonsi <petealft@amazon.com> Addressed Ankit's comments Signed-off-by: Peter Alfonsi <petealft@amazon.com> Break horrifyingly long test case into many shorter cases Signed-off-by: Peter Alfonsi <petealft@amazon.com> Added unsupported operation exception to TSC stats holder incrementEvictions() Signed-off-by: Peter Alfonsi <petealft@amazon.com> Addressed Sorabh's comments Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun assemble Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun gradle Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun gradle Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun gradle Signed-off-by: Peter Alfonsi <petealft@amazon.com> --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> commit be97e90 Author: Peter Alfonsi <peter.alfonsi@gmail.com> Date: Thu May 23 12:02:22 2024 -0700 [Bugfix] [Tiered Caching] Fixes issues when integrating tiered cache with disk cache (opensearch-project#13784) --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> commit 60ee715 Author: Kiran Prakash <awskiran@amazon.com> Date: Mon May 20 12:52:03 2024 -0700 Update IndicesRequestCacheIT.java (opensearch-project#13678) Signed-off-by: Kiran Prakash <awskiran@amazon.com> commit 51f6c1b Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Wed May 8 09:15:36 2024 -0700 Fix IndicesRequestCache clean up logic (opensearch-project#13597) Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com> Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com> commit 49e2701 Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Mon May 6 17:25:45 2024 -0700 Fix negative requestStats memory_size issue (opensearch-project#13553) This solves the bug where RequestStats memory_size metric was going negative in certain scenarios as reported in the issue. It turns out that the issue occurs when an indexShard is deleted and then reallocated on the same node. So whenever stale entries from older shard are deleted, those are accounted for the new shard which has the same shardId. --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> commit a5c5675 Author: Peter Alfonsi <peter.alfonsi@gmail.com> Date: Thu May 2 17:29:07 2024 -0700 [Tiered Caching] Adds stats implementation for TieredSpilloverCache (opensearch-project#13236) Stats rework part 4 of 4 --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> commit 46944cb Author: peteralfonsi <peter.alfonsi@gmail.com> Date: Tue Apr 30 10:59:03 2024 -0700 Fix flaky test CacheStatsAPIIndicesRequestCacheIT.testNullLevels() (opensearch-project#13457) * Fix flaky test Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Initialize CommonStatsFlags with empty array for levels Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Fixes tests using incorrect null levels Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun Signed-off-by: Peter Alfonsi <petealft@amazon.com> --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> commit 71fcad4 Author: Kiran Prakash <awskiran@amazon.com> Date: Mon Apr 29 18:41:33 2024 -0700 Fix Flaky test IndicesRequestCacheIT.testStaleKeysCleanupWithMultipleIndices (opensearch-project#13453) * Update IndicesRequestCacheIT.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheIT.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> --------- Signed-off-by: Kiran Prakash <awskiran@amazon.com> commit f60339f Author: peteralfonsi <peter.alfonsi@gmail.com> Date: Mon Apr 29 22:37:05 2024 -0700 [Tiered Caching] Bump versions for serialization in new cache stats API (opensearch-project#13460) --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> commit 88718a2 Author: peteralfonsi <peter.alfonsi@gmail.com> Date: Mon Apr 29 14:47:52 2024 -0700 [Tiered Caching] Expose new cache stats API (opensearch-project#13237) Step 3 out of 4 --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> commit 5e97082 Author: peteralfonsi <peter.alfonsi@gmail.com> Date: Sun Apr 28 08:15:18 2024 -0700 [Tiered Caching] Gate CacheStatsHolder logic behind FeatureFlags.PLUGGABLE_CACHE setting (opensearch-project#13238) Stats rework step 2 of 4 --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> commit ed881f7 Author: Kiran Prakash <awskiran@amazon.com> Date: Fri Apr 26 11:52:31 2024 -0700 [Tiered Caching] Make Indices Request Cache Stale Key Mgmt Threshold setting dynamic (opensearch-project#12941) * Update IndicesRequestCache.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update ClusterSettings.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheIT.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * spotless Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheIT.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * some refactoring Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCache.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * address existing tests Signed-off-by: Kiran Prakash <awskiran@amazon.com> * UTs Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update CHANGELOG.md Signed-off-by: Kiran Prakash <awskiran@amazon.com> * ITs Signed-off-by: Kiran Prakash <awskiran@amazon.com> * spotless Signed-off-by: Kiran Prakash <awskiran@amazon.com> * refactor Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheIT.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheIT.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheIT.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheIT.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update CHANGELOG.md Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update CHANGELOG.md Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheIT.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCache.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheIT.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * resolve conflicts Signed-off-by: Kiran Prakash <awskiran@amazon.com> * address code comments Signed-off-by: Kiran Prakash <awskiran@amazon.com> * address code comments Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheIT.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * rename tests Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheIT.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheIT.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * resolve conflicts Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCache.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * code comments Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheIT.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> --------- Signed-off-by: Kiran Prakash <awskiran@amazon.com> commit 424ccc9 Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Thu Apr 25 22:09:42 2024 -0700 [Tiered Caching] Expose a dynamic setting to disable/enable disk cache (opensearch-project#13373) * [Tiered Caching] Expose a dynamic setting to disable/enable disk cache Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Putting tiered cache settings behind feature flag Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding a changelog Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing Sorabh's comments Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Putting new setting behind feature flag Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> commit acbbb2f Author: Kiran Prakash <awskiran@amazon.com> Date: Wed Apr 24 14:50:39 2024 -0700 [Tiered Caching] Bug fix for IndicesRequestCache StaleKey management (opensearch-project#13070) * Update IndicesRequestCache.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update CHANGELOG.md Signed-off-by: Kiran Prakash <awskiran@amazon.com> * revert Signed-off-by: Kiran Prakash <awskiran@amazon.com> * revert Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCache.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCache.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCache.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCache.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCache.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCache.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * code comments only Signed-off-by: Kiran Prakash <awskiran@amazon.com> * docs changes Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update CHANGELOG.md Signed-off-by: Kiran Prakash <awskiran@amazon.com> * revert catching AlreadyClosedException Signed-off-by: Kiran Prakash <awskiran@amazon.com> * assert Signed-off-by: Kiran Prakash <awskiran@amazon.com> * conflicts Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCache.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * address comments Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCache.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCache.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * address conflicts Signed-off-by: Kiran Prakash <awskiran@amazon.com> * spotless apply Signed-off-by: Kiran Prakash <awskiran@amazon.com> * address comments Signed-off-by: Kiran Prakash <awskiran@amazon.com> * update code comments Signed-off-by: Kiran Prakash <awskiran@amazon.com> * address bug & add tests Signed-off-by: Kiran Prakash <awskiran@amazon.com> --------- Signed-off-by: Kiran Prakash <awskiran@amazon.com> commit f1d2e72 Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Mon Apr 15 10:43:22 2024 -0800 [Tiered Caching] Ehcache Disk cache IT (opensearch-project#12904) * Ehcache IT tests Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding some logs to print key/value size Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Add ehcache related invalidation IT Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Remvoing unnecessary IndicesRequestCache IT Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Indentation fix Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Added tests around expiration time and invalidation Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> commit b0de406 Author: peteralfonsi <peter.alfonsi@gmail.com> Date: Fri Apr 12 16:09:06 2024 -0700 [Tiered Caching] Stats rework (1/3): Interfaces and implementations for individual tiers (opensearch-project#12531) As part of tiered caching stats, changes the common ICache interface to use ICacheKey as its key. This key contains dimensions (for example, shard ID, index name, or tier) that can be used to aggregate stats. Also changes the CacheStats interface to store the necessary cache stats, and to support getting stats either as a total or aggregated by these dimensions. Integrates these changes with OpenSearchOnHeapCache and EhcacheDiskCache. The stats implementation for the TieredSpilloverCache will be in a followup PR. --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> commit 40a8d34 Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Thu Apr 11 14:49:03 2024 -0800 [Tiered Caching] Make took time policy dynamic and add additional integ tests (opensearch-project#13063) --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> commit 6f88777 Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Tue Mar 19 19:41:59 2024 -0700 Fixing ehcache flaky test (opensearch-project#12764) * Fixing ehcache flaky test Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding a ehcache issue reference for thread leak issue Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Updating comment Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> commit f69c089 Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Mon Mar 18 22:20:52 2024 -0700 [Tiered Caching] Clear up disk cache(ehcache) files during node shutdown (opensearch-project#12734) * Adding logic to clear up the disk cache files during close() * Adding logic to update entries count after invalidateAll() * Removing unneeded system log statement * Added comment in test for readability * Fixing issue where we were sending compacted byte[] array to ehcache but calculating size with padded byte[] --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> commit bf6488d Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Mon Mar 18 22:17:14 2024 -0700 [Tiered Caching] Fix test testComputeIfAbsentWithFactoryBasedCacheCreation (opensearch-project#12700) --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> commit 0fc1f40 Author: peteralfonsi <peter.alfonsi@gmail.com> Date: Mon Mar 18 13:39:35 2024 -0700 [Tiered Caching] Serializers for ehcache (opensearch-project#12709) Adds serializers and integrates them into ehcache disk cache --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> commit a45be0f Author: Kiran Prakash <awskiran@amazon.com> Date: Sat Mar 16 17:12:27 2024 -0700 [Tiered Caching] Indices Request cache stalekey management (opensearch-project#12625) * Introduce IndicesRequestCacheCleanupManager Signed-off-by: Kiran Prakash <awskiran@amazon.com> * using cleanup mgr to enqueue cleanups Signed-off-by: Kiran Prakash <awskiran@amazon.com> * readability improvements Signed-off-by: Kiran Prakash <awskiran@amazon.com> * update updateCleanupKeyToCountMap on new cache entry Signed-off-by: Kiran Prakash <awskiran@amazon.com> * create IndicesRequestCacheCleanupManager & settings and validators Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Add IRC specific cache cleaner and remove from IndicesService Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Unit Tests Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update CHANGELOG.md Signed-off-by: Kiran Prakash <awskiran@amazon.com> * move cachecleaner inside mgr Signed-off-by: Kiran Prakash <awskiran@amazon.com> * remove processCleanupKeys Signed-off-by: Kiran Prakash <awskiran@amazon.com> * minor cleanups Signed-off-by: Kiran Prakash <awskiran@amazon.com> * add updateCleanupKeyToCountMapOnCacheEviction Signed-off-by: Kiran Prakash <awskiran@amazon.com> * remove locks and make all methods synchronized Signed-off-by: Kiran Prakash <awskiran@amazon.com> * spotless Signed-off-by: Kiran Prakash <awskiran@amazon.com> * updateCleanupKeyToCountMapOnCacheEviction Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Testing Signed-off-by: Kiran Prakash <awskiran@amazon.com> * add Reschedule back to indices service Signed-off-by: Kiran Prakash <awskiran@amazon.com> * rename updateStaleKeysCount to incrementStaleKeysCount Signed-off-by: Kiran Prakash <awskiran@amazon.com> * rename getStaleKeysCountForTesting to getStaleKeysCount Signed-off-by: Kiran Prakash <awskiran@amazon.com> * rename threshold to stalenessThreshold Signed-off-by: Kiran Prakash <awskiran@amazon.com> * check for cleanupKey.entity == null Signed-off-by: Kiran Prakash <awskiran@amazon.com> * use computeIfPresent with keycountmap Signed-off-by: Kiran Prakash <awskiran@amazon.com> * log both staleKeysInCache & Staleness in debug logs Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Use HashMap instead of ConcurrentMap Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Address b/w compatibility Signed-off-by: Kiran Prakash <awskiran@amazon.com> * remove synchronized for updateCleanupKeyToCountMapOnCacheEviction Signed-off-by: Kiran Prakash <awskiran@amazon.com> * make cleanCache synchronized Signed-off-by: Kiran Prakash <awskiran@amazon.com> * remove shouldRemoveKey Signed-off-by: Kiran Prakash <awskiran@amazon.com> * spotlessApply Signed-off-by: Kiran Prakash <awskiran@amazon.com> --------- Signed-off-by: Kiran Prakash <awskiran@amazon.com> commit bad412c Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Fri Mar 15 16:53:54 2024 -0700 [Tiered caching] Supporting removal function on EhcacheDiskCache iterator (opensearch-project#12653) * [Tiered caching] Supporting removal function on EhcacheDiskCache iterator Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Minor refactoring in unit test Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> commit 15c4afa Author: peteralfonsi <peter.alfonsi@gmail.com> Date: Fri Mar 15 15:36:22 2024 -0700 [Tiered Caching] Cache tier policies (opensearch-project#12542) * Adds policy interface and took time policy impl Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Changes IndicesService to write a CachePolicyInfoWrapper before the QSR Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Moved took time logic from QSR to IndicesService Signed-off-by: Peter Alfonsi <petealft@amazon.com> * spotlessApply Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Addressed ansjcy's comments Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Partial rebase on most recent changes Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Integrated policies with new TSC changes Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Reverted unintended change to idea/vcs.xml Signed-off-by: Peter Alfonsi <petealft@amazon.com> * javadocs Signed-off-by: Peter Alfonsi <petealft@amazon.com> * github actions Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Set default threshold value to 10 ms Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Addressed Sorabh's comments Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Addressed Sorabh's second round of comments Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Set cachedQueryParser in IRC Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Addressed Sorabh's comments besides dynamic setting Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Removed dynamic setting, misc comments Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Added changelog entry Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Added missing javadoc Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Fixed failed gradle run Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Added setting validation test Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun gradle for flaky IT Signed-off-by: Peter Alfonsi <petealft@amazon.com> * javadocs Signed-off-by: Peter Alfonsi <petealft@amazon.com> --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> commit 45fce15 Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Thu Mar 14 13:00:24 2024 -0700 [Tiered Caching] Fixing flaky tiered cache test (opensearch-project#12650) * Fixing flaky tiered cache test Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Removing unnecessary comment Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Removing unused variable Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> commit bb66699 Author: Peter Alfonsi <petealft@amazon.com> Date: Fri Aug 30 11:36:05 2024 -0700 Attempt to fix broken ITs Signed-off-by: Peter Alfonsi <petealft@amazon.com> commit 58170bf Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Mon Mar 11 14:12:55 2024 -0700 [Tiered caching] Integrating IndicesRequestCache with CacheService controlled by a feature flag (opensearch-project#12533) * Adding changelog * Fixing gradle build issue * Fixing CacheService test * Adding UT in IndicesRequestCache with feature flag for more coverage * Updating changelog and renaming feature flag setting * Moving feature flag setting handling logic to CacheService by maintaining backward compatibility * Fixing broken UTs --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> commit 80d2130 Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Fri Mar 1 11:10:28 2024 -0800 [Tiered caching] Introducing cache plugins and exposing Ehcache as one of the pluggable disk cache option (opensearch-project#11874) Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> commit f372c2c Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Wed Jan 10 17:06:29 2024 -0800 [Tiered Caching] Enable serialization of IndicesRequestCache.Key (opensearch-project#10275) --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> Co-authored-by: Kiran Prakash <awskiran@amazon.com> commit b880c0b Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Tue Jan 9 10:57:52 2024 -0800 [Tiered caching] Framework changes (opensearch-project#10753) * [Tiered caching] Framework changes Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Added javadoc for new files/packages Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Added changelog Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing javadoc warnings Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comments Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing additional minor comments Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Moving non null check to builder for OS onHeapCache Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding package-info for new packages Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Removing service and adding different cache interfaces along with event listener support Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing gradle missingDoc issue Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Changing listener logic, removing tiered cache integration with IRC Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding opensearch.internal tag for LoadAwareCacheLoader Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing thread safety issue Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Remove compute function and event listener logic change for TieredCache Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Making Cache.compute function private Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding javadoc and more test for cache.put Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding write locks to refresh API as well Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Removing unwanted EventType class and refactoring one UT Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Removing TieredCache interface Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Description
Fixes flakiness in DefaultCacheStatsHolderTests.testConcurrentRemoval(). The test previously waited 1 ms after starting the removal of a set of dimensions, then added a set of dimensions back to the stats holder. If the removal took >1 ms, the adding could happen before the removal and an assertion would fail. On my machine I was sometimes able to reproduce the failure when the sleep interval was 0, confirming this was the issue.
Reworked the test to not rely on Thread.Sleep(), and also to do more concurrent removals. Re-adding the set of dimensions after the removals is not necessary. The test now passes when run 10,000 times on my machine.
Related Issues
Resolves #14301
Check List
- [N/A] API changes companion pull request created, if applicable.- [N/A] Public documentation issue/PR created, if applicable.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.