Clear ehcache disk cache files during initialization #14738
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
sgup432
requested review from
anasalkouz,
andrross,
ashking94,
Bukhtawar,
CEHENKLE,
dblock,
dbwiddis,
gbbafna,
kotwanikunal,
mch2,
msfroh,
nknize,
owaiskazi19,
reta,
Rishikesh1159,
sachinpkale,
saratvemulapalli,
shwetathareja,
sohami and
VachaShah
as code owners
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #14738 +/- ##
============================================
+ Coverage 71.74% 71.75% +0.01%
+ Complexity 62413 62325 -88
============================================
Files 5149 5131 -18
Lines 293396 292827 -569
Branches 42392 42352 -40
============================================
- Hits 210484 210130 -354
+ Misses 65590 65424 -166
+ Partials 17322 17273 -49 ☔ View full report in Codecov by Sentry. |
Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
sohami
reviewed
Jul 12, 2024
plugins/cache-ehcache/src/test/java/org/opensearch/cache/store/disk/EhCacheDiskCacheTests.java
Outdated
Show resolved
Hide resolved
plugins/cache-ehcache/src/main/java/org/opensearch/cache/store/disk/EhcacheDiskCache.java
Outdated
Show resolved
Hide resolved
Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
sohami
approved these changes
Jul 16, 2024
Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
|
❕ Gradle check result for a611976: UNSTABLE Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure. |
Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
|
❌ Gradle check result for 5e179dc: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
sohami
approved these changes
Jul 18, 2024
sohami
reviewed
Jul 18, 2024
plugins/cache-ehcache/src/main/java/org/opensearch/cache/store/disk/EhcacheDiskCache.java
Show resolved
Hide resolved
opensearch-trigger-bot bot
pushed a commit
that referenced
this pull request
Jul 18, 2024
* Clear ehcache disk cache files during initialization Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding UT to fix line coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comment Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding more Uts for better line coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Throwing exception in case we fail to clear cache files during startup Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding more UTs Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding a UT for more coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing gradle build Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Update ehcache disk cache close() logic Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> (cherry picked from commit b3b743d) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
sohami
pushed a commit
that referenced
this pull request
Jul 19, 2024
* Clear ehcache disk cache files during initialization * Adding UT to fix line coverage * Addressing comment * Adding more Uts for better line coverage * Throwing exception in case we fail to clear cache files during startup * Adding more UTs * Adding a UT for more coverage * Fixing gradle build * Update ehcache disk cache close() logic --------- (cherry picked from commit b3b743d) Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
kaushalmahi12
pushed a commit
to kaushalmahi12/OpenSearch
that referenced
this pull request
Jul 23, 2024
…ect#14738) * Clear ehcache disk cache files during initialization Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding UT to fix line coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comment Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding more Uts for better line coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Throwing exception in case we fail to clear cache files during startup Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding more UTs Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding a UT for more coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing gradle build Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Update ehcache disk cache close() logic Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
kaushalmahi12
pushed a commit
to kaushalmahi12/OpenSearch
that referenced
this pull request
Jul 23, 2024
…ect#14738) * Clear ehcache disk cache files during initialization Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding UT to fix line coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comment Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding more Uts for better line coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Throwing exception in case we fail to clear cache files during startup Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding more UTs Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding a UT for more coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing gradle build Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Update ehcache disk cache close() logic Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>
kkewwei
pushed a commit
to kkewwei/OpenSearch
that referenced
this pull request
Jul 24, 2024
…ect#14738) (opensearch-project#14806) * Clear ehcache disk cache files during initialization * Adding UT to fix line coverage * Addressing comment * Adding more Uts for better line coverage * Throwing exception in case we fail to clear cache files during startup * Adding more UTs * Adding a UT for more coverage * Fixing gradle build * Update ehcache disk cache close() logic --------- (cherry picked from commit b3b743d) Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Signed-off-by: kkewwei <kkewwei@163.com>
mch2
pushed a commit
that referenced
this pull request
Jul 30, 2024
…de level (#14708) * add logic to add headers to Task Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add logic to add queryGroupId to task headers Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * remove redundant code Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add changelog entry Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * address comments Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * fix precommit Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add UTs for RemoteIndexMetadataManager (#14660) Signed-off-by: Shivansh Arora <hishiv@amazon.com> Co-authored-by: Arpit-Bandejiya <abandeji@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix match_phrase_prefix_query not working on text field with multiple values and index_prefixes (#10959) * Fix match_phrase_prefix_query not working on text field with multiple values and index_prefixes Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Add more test Signed-off-by: Gao Binlong <gbinlong@amazon.com> * modify change log Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Fix test failure Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Change the indexAnalyzer used by prefix field Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Skip old version for yaml test Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Optimize some code Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Fix test failure Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Modify yaml test description Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Remove the name parameter for setAnalyzer() Signed-off-by: Gao Binlong <gbinlong@amazon.com> --------- Signed-off-by: Gao Binlong <gbinlong@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Offline calculation of total shard per node and caching it for weight calculation inside LocalShardBalancer (#14675) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * [bug fix] validate lower bound for top n size (#14587) Signed-off-by: Chenyang Ji <cyji@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Create SystemIndexRegistry with helper method matchesSystemIndex (#14415) * Create new extension point in SystemIndexPlugin for a single plugin to get registered system indices Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add to CHANGELOG Signed-off-by: Craig Perkins <cwperx@amazon.com> * WIP on system indices from IndexNameExpressionResolver Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add test in IndexNameExpressionResolverTests Signed-off-by: Craig Perkins <cwperx@amazon.com> * Remove changes in SystemIndexPlugin Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add method in IndexNameExpressionResolver to get matching system indices Signed-off-by: Craig Perkins <cwperx@amazon.com> * Show how resolver can be chained to get system indices Signed-off-by: Craig Perkins <cwperx@amazon.com> * Fix forbiddenApis check Signed-off-by: Craig Perkins <cwperx@amazon.com> * Update CHANGELOG Signed-off-by: Craig Perkins <cwperx@amazon.com> * Make SystemIndices internal Signed-off-by: Craig Perkins <cwperx@amazon.com> * Remove unneeded changes Signed-off-by: Craig Perkins <cwperx@amazon.com> * Fix CI failures Signed-off-by: Craig Perkins <cwperx@amazon.com> * Fix precommit errors Signed-off-by: Craig Perkins <cwperx@amazon.com> * Use Regex instead of WildcardMatcher Signed-off-by: Craig Perkins <cwperx@amazon.com> * Address code review feedback Signed-off-by: Craig Perkins <cwperx@amazon.com> * Allow caller to pass index expressions Signed-off-by: Craig Perkins <cwperx@amazon.com> * Create SystemIndexRegistry Signed-off-by: Craig Perkins <cwperx@amazon.com> * Update CHANGELOG Signed-off-by: Craig Perkins <cwperx@amazon.com> * Remove singleton limitation Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add javadoc Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add @experimentalapi annotation Signed-off-by: Craig Perkins <cwperx@amazon.com> --------- Signed-off-by: Craig Perkins <cwperx@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Refactor Grok validate pattern to iterative approach (#14206) * grok validate patterns recusrion to iterative Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> * Add max depth in resolving a pattern to avoid OOM Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> * change path from deque to arraylist Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> * rename queue to stack Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> * Change max depth to 500 Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> * typo originPatternName fix Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> * spotless Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> --------- Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump opentelemetry from 1.39.0 to 1.40.0 (#14674) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump jackson from 2.17.1 to 2.17.2 (#14687) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add release notes for release 1.3.18 (#14699) Signed-off-by: Zelin Hao <zelinhao@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump reactor from 3.5.19 to 3.5.20 (#14697) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add unit tests for read flow of RemoteClusterStateService and bug fix for transient settings (#14476) Signed-off-by: Shivansh Arora <hishiv@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Update version check for the bug fix of match_phrase_prefix_query not working on text field with multiple values and index_prefixes (#14703) Signed-off-by: Gao Binlong <gbinlong@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Remove unnecessary cast to int from test (#14696) Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * print reason why parent task was cancelled (#14604) Signed-off-by: kkewwei <kkewwei@163.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Use set of shard routing for shard in unassigned shard batch check. (#14533) Signed-off-by: Swetha Guptha <gupthasg@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add versioning for UploadedIndexMetadata (#14677) * Add versioning for UploadedIndexMetadata * Handle componentPrefix for backward compatibility Signed-off-by: Sooraj Sinha <soosinha@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix: update help output for _cat (#14722) * fixed help output for _cat Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io> * updated changelog Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io> * updated changelog Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io> --------- Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix hdfs-fixture kerb-admin & hadoop-minicluster dependencies are not being updated / false positive reports on CVEs (#14729) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Update to Gradle 8.9 (#14574) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix hdfs-fixture hadoop-minicluster dependencies are not being updated / false positive reports on CVEs (#14732) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add `strict_allow_templates` dynamic mapping option (#14555) * The dynamic mapping parameter supports strict_allow_templates Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Modify change log Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Modify skip version in yml test file Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Refactor some code Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Keep the old methods Signed-off-by: Gao Binlong <gbinlong@amazon.com> * change public to private Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Optimize some code Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Do not override toString method for Dynamic Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Optimize some code and modify the changelog Signed-off-by: Gao Binlong <gbinlong@amazon.com> --------- Signed-off-by: Gao Binlong <gbinlong@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump net.minidev:json-smart from 2.5.0 to 2.5.1 in /plugins/repository-azure (#14748) * Bump net.minidev:json-smart in /plugins/repository-azure Bumps [net.minidev:json-smart](https://github.com/netplex/json-smart-v2) from 2.5.0 to 2.5.1. - [Release notes](https://github.com/netplex/json-smart-v2/releases) - [Commits](netplex/json-smart-v2@2.5.0...2.5.1) --- updated-dependencies: - dependency-name: net.minidev:json-smart dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * remove query insights plugin from core (#14743) Signed-off-by: Chenyang Ji <cyji@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add `strict_allow_templates` dynamic mapping option (#14555) (#14737) (#14742) * The dynamic mapping parameter supports strict_allow_templates * Modify change log * Modify skip version in yml test file * Refactor some code * Keep the old methods * change public to private * Optimize some code * Do not override toString method for Dynamic * Optimize some code and modify the changelog --------- (cherry picked from commit 6b8b3ef) Signed-off-by: Gao Binlong <gbinlong@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix create or update alias API doesn't throw exception for unsupported parameters (#14719) * Fix create or update alias API doesn't throw exception for unsupported parameters Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Update version check in yml test Signed-off-by: Gao Binlong <gbinlong@amazon.com> * modify change log Signed-off-by: Gao Binlong <gbinlong@amazon.com> --------- Signed-off-by: Gao Binlong <gbinlong@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Remove query categorization from core (#14759) * Remove query categorization from core Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Add changelog Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Trigger Build Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> --------- Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add changes to propagate queryGroupId across child requests and nodes (#14614) * add query group header propagator Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * apply spotless check Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add new propagator in ThreadContext Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * spotlessApply Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * address comments Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (#14610) * Bump com.microsoft.azure:msal4j in /plugins/repository-azure Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0. - [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases) - [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt) - [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0) --- updated-dependencies: - dependency-name: com.microsoft.azure:msal4j dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * [Bugfix] Fix ICacheKeySerializerTests flakiness (#14564) * Fix testInvalidInput flakiness Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Addressed andrross's comment Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun security check Signed-off-by: Peter Alfonsi <petealft@amazon.com> --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Correct typo in method name (#14621) Signed-off-by: vatsal <vatsal.v.anand@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Refactoring FilterPath.parse by using an iterative approach instead of recursion. (#14200) * Refactor FilterPath parse function (#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Implement unit tests for FilterPathTests (#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Write warn log if Filter is empty; Add comments (#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Add changelog Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove unnecessary log statement Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove unused logger Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Spotless apply Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove incorrect changelog Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> --------- Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(#14612) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (#14513) * Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata * Correct the check for deciding upload of hashes of consistent settings Signed-off-by: Sooraj Sinha <soosinha@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add changelog Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add PR link changelog Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (#14613) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix assertion failure while deleting remote backed index (#14601) Signed-off-by: Sachin Kale <kalsac@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Allow system index warning in OpenSearchRestTestCase.refreshAllIndices (#14635) * Allow system index warning Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add to CHANGELOG Signed-off-by: Craig Perkins <cwperx@amazon.com> * Address code review comments Signed-off-by: Craig Perkins <cwperx@amazon.com> --------- Signed-off-by: Craig Perkins <cwperx@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Star tree codec changes (#14514) --------- Signed-off-by: Bharathwaj G <bharath78910@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump com.github.spullara.mustache.java:compiler from 0.9.13 to 0.9.14 in /modules/lang-mustache (#14672) * Bump com.github.spullara.mustache.java:compiler Bumps [com.github.spullara.mustache.java:compiler](https://github.com/spullara/mustache.java) from 0.9.13 to 0.9.14. - [Commits](spullara/mustache.java@mustache.java-0.9.13...mustache.java-0.9.14) --- updated-dependencies: - dependency-name: com.github.spullara.mustache.java:compiler dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump net.minidev:accessors-smart from 2.5.0 to 2.5.1 in /plugins/repository-azure (#14673) * Bump net.minidev:accessors-smart in /plugins/repository-azure Bumps [net.minidev:accessors-smart](https://github.com/netplex/json-smart-v2) from 2.5.0 to 2.5.1. - [Release notes](https://github.com/netplex/json-smart-v2/releases) - [Commits](netplex/json-smart-v2@2.5.0...2.5.1) --- updated-dependencies: - dependency-name: net.minidev:accessors-smart dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * move query group thread context propagator out of ThreadContext Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> --------- Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Peter Alfonsi <petealft@amazon.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Sooraj Sinha <soosinha@amazon.com> Signed-off-by: Sachin Kale <kalsac@amazon.com> Signed-off-by: Craig Perkins <cwperx@amazon.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com> Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com> Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com> Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com> Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com> Co-authored-by: Sachin Kale <sachinpkale@gmail.com> Co-authored-by: Craig Perkins <cwperx@amazon.com> Co-authored-by: Bharathwaj G <bharath78910@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add consumers to remote store based index settings (#14764) Signed-off-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add matchesPluginSystemIndexPattern to SystemIndexRegistry (#14750) * Add matchesPluginSystemIndexPattern to SystemIndexRegistry Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add to CHANGELOG Signed-off-by: Craig Perkins <cwperx@amazon.com> * Use single data structure to keep track of system indices Signed-off-by: Craig Perkins <cwperx@amazon.com> * Address code review comments Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add test for getAllDescriptors Signed-off-by: Craig Perkins <cwperx@amazon.com> * Update server/src/main/java/org/opensearch/indices/SystemIndexRegistry.java Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Craig Perkins <craig5008@gmail.com> --------- Signed-off-by: Craig Perkins <cwperx@amazon.com> Signed-off-by: Craig Perkins <craig5008@gmail.com> Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * SPI for loading ABC templates (#14659) * SPI for loading ABC templates Signed-off-by: mgodwan <mgodwan@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix bulk upsert ignores the default_pipeline and final_pipeline when the auto-created index matches the index template (#12891) * Fix bulk upsert ignores the default_pipeline and final_pipeline when auto-created index matches with the index template Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Modify changelog & comment Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Use new approach Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Fix test failure Signed-off-by: Gao Binlong <gbinlong@amazon.com> --------- Signed-off-by: Gao Binlong <gbinlong@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix flaky test due to node being used across all tests (#14787) Signed-off-by: Mohit Godwani <mgodwan@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Star Tree Implementation [OnHeap] (#14512) --------- Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add Gao Binlong as maintainer (#14796) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Clear ehcache disk cache files during initialization (#14738) * Clear ehcache disk cache files during initialization Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding UT to fix line coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comment Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding more Uts for better line coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Throwing exception in case we fail to clear cache files during startup Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding more UTs Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding a UT for more coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing gradle build Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Update ehcache disk cache close() logic Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Refactor remote-routing-table service inline with remote state interfaces (#14668) --------- Signed-off-by: Arpit Bandejiya <abandeji@amazon.com> Signed-off-by: Arpit-Bandejiya <abandeji@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Set version to 2.15 for determining metadata during migration to remote store Signed-off-by: Sandeep Kumawat <skumwt@amazon.com> Co-authored-by: Sandeep Kumawat <skumwt@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix bulk upsert ignores the default_pipeline and final_pipeline when the auto-created index matches the index template (#14793) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix create or update alias API doesn't throw exception for unsupported parameters (#14769) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Change RCSS info logs to debug (#14814) Signed-off-by: Shivansh Arora <hishiv@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * [Bugfix] Fix NPE in ReplicaShardAllocator (#13993) (#14385) * [Bugfix] Fix NPE in ReplicaShardAllocator (#13993) Signed-off-by: Daniil Roman <daniilroman.cv@gmail.com> * Add fix info to CHANGELOG.md Signed-off-by: Daniil Roman <danroman17397@gmail.com> --------- Signed-off-by: Daniil Roman <daniilroman.cv@gmail.com> Signed-off-by: Daniil Roman <danroman17397@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Run performance benchmark on pull requests (#14760) * add performance benchmark workflow for pull requests Signed-off-by: Rishabh Singh <sngri@amazon.com> * Update PERFORMANCE_BENCHMARKS.md Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> * Update PERFORMANCE_BENCHMARKS.md Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> * Update .github/workflows/benchmark-pull-request.yml Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> * Update .github/workflows/benchmark-pull-request.yml Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> * Update .github/workflows/benchmark-pull-request.yml Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> * Update .github/workflows/benchmark-pull-request.yml Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> --------- Signed-off-by: Rishabh Singh <sngri@amazon.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * fix constant_keyword field type (#14807) Signed-off-by: kkewwei <kkewwei@163.com> test Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * [Remote Store Migration] Reconcile remote store based index settings during STRICT mode switch (#14792) Signed-off-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add prefix mode verification setting for repository verification (#14790) * Add prefix mode verification setting for repository verification Signed-off-by: Ashish Singh <ssashish@amazon.com> * Add UTs and randomise prefix mode repository verification Signed-off-by: Ashish Singh <ssashish@amazon.com> * Incorporate PR review feedback Signed-off-by: Ashish Singh <ssashish@amazon.com> --------- Signed-off-by: Ashish Singh <ssashish@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add length check on comment body for benchmark workflow (#14834) Signed-off-by: Rishabh Singh <sngri@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add restore-from-snapshot test procedure for snapshot run benchmark config (#14842) Signed-off-by: Rishabh Singh <sngri@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix env variable name typo (#14843) Signed-off-by: Rishabh Singh <sngri@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Use circuit breaker in InternalHistogram when adding empty buckets (#14754) * introduce circuit breaker in InternalHistogram Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * use circuit breaker from reduce context Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * add test Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * revert use_real_memory change in OpenSearchNode Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * add change log Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> --------- Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * [Remote State] Create interface RemoteEntitiesManager (#14671) * Create interface RemoteEntitiesManager Signed-off-by: Shivansh Arora <hishiv@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Optimise TransportNodesAction to not send DiscoveryNodes for NodeStat… (#14749) * Optimize TransportNodesAction to not send DiscoveryNodes for NodeStats, NodesInfo and ClusterStats call Signed-off-by: Pranshu Shukla <pranshushukla06@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Enabling term version check on local state for all ClusterManager Read Transport Actions (#14273) * enabling term version check on local state for all admin read actions Signed-off-by: Rajiv Kumar Vaidyanathan <rajivkv@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Reduce logging in DEBUG for MasterService:run (#14795) * Reduce logging in DEBUG for MasteService:run by introducing short and long summary in Taskbatcher Signed-off-by: Sumit Bansal <sumitsb@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add SplitResponseProcessor to Search Pipelines (#14800) * Add SplitResponseProcessor for search pipelines Signed-off-by: Daniel Widdis <widdis@gmail.com> * Register the split processor factory Signed-off-by: Daniel Widdis <widdis@gmail.com> * Address code review comments Signed-off-by: Daniel Widdis <widdis@gmail.com> * Avoid list copy by casting array Signed-off-by: Daniel Widdis <widdis@gmail.com> --------- Signed-off-by: Daniel Widdis <widdis@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add integration tests for RemoteRoutingTable Service. (#14631) Signed-off-by: Shailendra Singh <singhlhs@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add SortResponseProcessor to Search Pipelines (#14785) * Add SortResponseProcessor for search pipelines Signed-off-by: Daniel Widdis <widdis@gmail.com> * Add stupid and unnecessary javadocs to satisfy overly strict CI Signed-off-by: Daniel Widdis <widdis@gmail.com> * Split casting and sorting methods for readability Signed-off-by: Daniel Widdis <widdis@gmail.com> * Register the sort processor factory Signed-off-by: Daniel Widdis <widdis@gmail.com> * Address code review comments Signed-off-by: Daniel Widdis <widdis@gmail.com> * Cast individual list elements to avoid creating two lists Signed-off-by: Daniel Widdis <widdis@gmail.com> * Add yamlRestTests Signed-off-by: Daniel Widdis <widdis@gmail.com> * Clarify why there's unusual sorting Signed-off-by: Daniel Widdis <widdis@gmail.com> * Use instanceof instead of isAssignableFrom Signed-off-by: Daniel Widdis <widdis@gmail.com> --------- Signed-off-by: Daniel Widdis <widdis@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix allowUnmappedFields, mapUnmappedFieldAsString settings to be applied when parsing query string query (#13957) * Modify to invoke QueryShardContext.fieldMapper() method to apply allowUnmappedFields and mapUnmappedFieldAsString settings Signed-off-by: imyp92 <pyw5420@gmail.com> * Add test cases to verify returning 400 responses if unmapped fields are included for some types of query Signed-off-by: imyp92 <pyw5420@gmail.com> * Add changelog Signed-off-by: imyp92 <pyw5420@gmail.com> --------- Signed-off-by: imyp92 <pyw5420@gmail.com> Signed-off-by: gaobinlong <gbinlong@amazon.com> Co-authored-by: gaobinlong <gbinlong@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump com.microsoft.azure:msal4j from 1.16.0 to 1.16.1 in /plugins/repository-azure (#14857) * Bump com.microsoft.azure:msal4j in /plugins/repository-azure Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.16.0 to 1.16.1. - [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases) - [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt) - [Commits](AzureAD/microsoft-authentication-library-for-java@v1.16.0...v1.16.1) --- updated-dependencies: - dependency-name: com.microsoft.azure:msal4j dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump com.gradle.develocity from 3.17.5 to 3.17.6 (#14856) * Bump com.gradle.develocity from 3.17.5 to 3.17.6 Bumps com.gradle.develocity from 3.17.5 to 3.17.6. --- updated-dependencies: - dependency-name: com.gradle.develocity dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump org.jline:jline in /test/fixtures/hdfs-fixture (#14859) Bumps [org.jline:jline](https://github.com/jline/jline3) from 3.26.2 to 3.26.3. - [Release notes](https://github.com/jline/jline3/releases) - [Changelog](https://github.com/jline/jline3/blob/master/changelog.md) - [Commits](jline/jline3@jline-parent-3.26.2...jline-parent-3.26.3) --- updated-dependencies: - dependency-name: org.jline:jline dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Use Lucene provided Persian stem (#14847) Lucene provided Persian stem apparently isn't hooked yet and this change is doing that based on what is done for Arabic stem support. Signed-off-by: Ebrahim Byagowi <ebrahim@gnu.org> Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump actions/checkout from 2 to 4 (#14858) * Bump actions/checkout from 2 to 4 Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 4. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v2...v4) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Deprecate batch_size parameter on bulk API (#14725) By default the full _bulk payload will be passed to ingest processors as a batch, with any sub batching logic to be implemented by each processor if necessary. Signed-off-by: Liyun Xiu <xiliyun@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add perms for remote snapshot cache eviction on scripted query (#14411) Signed-off-by: Finn Carroll <carrofin@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add transport interceptor to populate queryGroupId in task headers Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add rest, transport layer changes for Hot to warm tiering - dedicated setup (#13980) Signed-off-by: Neetika Singhal <neetiks@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Create listener to refresh search thread resource usage (#14832) * [bug fix] fix incorrect coordinator node search resource usages Signed-off-by: Chenyang Ji <cyji@amazon.com> * fix bug on serialization when passing task resource usage to coordinator Signed-off-by: Chenyang Ji <cyji@amazon.com> * add more unit tests Signed-off-by: Chenyang Ji <cyji@amazon.com> * remove query insights plugin related code Signed-off-by: Chenyang Ji <cyji@amazon.com> * create per request listener to refresh task resource usage Signed-off-by: Chenyang Ji <cyji@amazon.com> * Make new listener API public Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Add changelog Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove wrong files added Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Address review comments Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Build fix Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Make singleton Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Address review comments Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Make sure listener runs before plugin listeners Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Spotless Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Minor fix Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> --------- Signed-off-by: Chenyang Ji <cyji@amazon.com> Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Signed-off-by: Jay Deng <jayd0104@gmail.com> Co-authored-by: Chenyang Ji <cyji@amazon.com> Co-authored-by: Jay Deng <jayd0104@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Caching avg total bytes and avg free bytes inside ClusterInfo (#14851) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Use default value when index.number_of_replicas is null (#14812) * Use default value when index.number_of_replicas is null Signed-off-by: Liyun Xiu <xiliyun@amazon.com> * Add integration test Signed-off-by: Liyun Xiu <xiliyun@amazon.com> * Add changelog Signed-off-by: Liyun Xiu <xiliyun@amazon.com> --------- Signed-off-by: Liyun Xiu <xiliyun@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * [Remote Routing Table] Implement write and read flow for shard diff file. (#14684) * Implement write and read flow to upload/download shard diff file. Signed-off-by: Shailendra Singh <singhlhs@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Optimized ClusterStatsIndices to precomute shard stats (#14426) * Optimize Cluster Stats Indices to precomute node level stats Signed-off-by: Pranshu Shukla <pranshushukla06@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix constraint bug which allows more primary shards than average primary shards per index (#14908) Signed-off-by: Gaurav Bafna <gbbafna@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Optmising AwarenessAllocationDecider for hashmap.get call (#14761) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * update comment Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix IngestServiceTests.testBulkRequestExecutionWithFailures (#14918) The test would previously fail if the randomness led to only a single indexing request being included in the bulk payload. This change guarantees multiple indexing requests in order to ensure the batch logic kicks in. Also replace some unneeded mocks with real classes. Signed-off-by: Andrew Ross <andrross@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add queryGroupTask Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * remove unnecessary imports Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add QueryGroupTask tests Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * rename WLM transport request handler Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add CHANGELOG entry Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * fix ut Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * address comments Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * fix UT to remove the verify for final method Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * apply spotless Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> --------- Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> Signed-off-by: Shivansh Arora <hishiv@amazon.com> Signed-off-by: Gao Binlong <gbinlong@amazon.com> Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Chenyang Ji <cyji@amazon.com> Signed-off-by: Craig Perkins <cwperx@amazon.com> Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Zelin Hao <zelinhao@amazon.com> Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io> Signed-off-by: kkewwei <kkewwei@163.com> Signed-off-by: Swetha Guptha <gupthasg@amazon.com> Signed-off-by: Sooraj Sinha <soosinha@amazon.com> Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Signed-off-by: Peter Alfonsi <petealft@amazon.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> Signed-off-by: Sachin Kale <kalsac@amazon.com> Signed-off-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com> Signed-off-by: Craig Perkins <craig5008@gmail.com> Signed-off-by: mgodwan <mgodwan@amazon.com> Signed-off-by: Mohit Godwani <mgodwan@amazon.com> Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sandeep Kumawat <skumwt@amazon.com> Signed-off-by: Daniil Roman <daniilroman.cv@gmail.com> Signed-off-by: Daniil Roman <danroman17397@gmail.com> Signed-off-by: Rishabh Singh <sngri@amazon.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> Signed-off-by: Ashish Singh <ssashish@amazon.com> Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> Signed-off-by: Pranshu Shukla <pranshushukla06@gmail.com> Signed-off-by: Rajiv Kumar Vaidyanathan <rajivkv@amazon.com> Signed-off-by: Sumit Bansal <sumitsb@amazon.com> Signed-off-by: Daniel Widdis <widdis@gmail.com> Signed-off-by: Shailendra Singh <singhlhs@amazon.com> Signed-off-by: imyp92 <pyw5420@gmail.com> Signed-off-by: gaobinlong <gbinlong@amazon.com> Signed-off-by: Ebrahim Byagowi <ebrahim@gnu.org> Signed-off-by: Liyun Xiu <xiliyun@amazon.com> Signed-off-by: Finn Carroll <carrofin@amazon.com> Signed-off-by: Neetika Singhal <neetiks@amazon.com> Signed-off-by: Jay Deng <jayd0104@gmail.com> Signed-off-by: Gaurav Bafna <gbbafna@amazon.com> Signed-off-by: Andrew Ross <andrross@amazon.com> Co-authored-by: Shivansh Arora <hishiv@amazon.com> Co-authored-by: Arpit-Bandejiya <abandeji@amazon.com> Co-authored-by: gaobinlong <gbinlong@amazon.com> Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com> Co-authored-by: Chenyang Ji <cyji@amazon.com> Co-authored-by: Craig Perkins <cwperx@amazon.com> Co-authored-by: Sandesh Kumar <sandeshkr419@gmail.com> Co-authored-by: Andriy Redko <andriy.redko@aiven.io> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Lukáš Vlček <lukas.vlcek@aiven.io> Co-authored-by: kkewwei <kkewwei@163.com> Co-authored-by: SwethaGuptha <156877431+SwethaGuptha@users.noreply.github.com> Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com> Co-authored-by: Ahmed Sobeh <ahmed.sobeh@aiven.io> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com> Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com> Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com> Co-authored-by: Sachin Kale <sachinpkale@gmail.com> Co-authored-by: Bharathwaj G <bharath78910@gmail.com> Co-authored-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com> Co-authored-by: Craig Perkins <craig5008@gmail.com> Co-authored-by: Andriy Redko <drreta@gmail.com> Co-authored-by: Mohit Godwani <81609427+mgodwan@users.noreply.github.com> Co-authored-by: Sarthak Aggarwal <sarthagg@amazon.com> Co-authored-by: Sagar <99425694+sgup432@users.noreply.github.com> Co-authored-by: Sandeep Kumawat <2025sandeepkumawat@gmail.com> Co-authored-by: Sandeep Kumawat <skumwt@amazon.com> Co-authored-by: Daniil Roman <danroman17397@gmail.com> Co-authored-by: Rishabh Singh <rishabhksingh@gmail.com> Co-authored-by: kkewwei <kewei.11@bytedance.com> Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> Co-authored-by: Ashish Singh <ssashish@amazon.com> Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com> Co-authored-by: Pranshu Shukla <55992439+Pranshu-S@users.noreply.github.com> Co-authored-by: rajiv-kv <157019998+rajiv-kv@users.noreply.github.com> Co-authored-by: Sumit Bansal <sumit.asr@gmail.com> Co-authored-by: Daniel Widdis <widdis@gmail.com> Co-authored-by: shailendra0811 <167273922+shailendra0811@users.noreply.github.com> Co-authored-by: Park, Yeongwu <pyw5420@gmail.com> Co-authored-by: ebraminio <ebraminio@gmail.com> Co-authored-by: Liyun Xiu <xiliyun@amazon.com> Co-authored-by: Finn <carrofin@amazon.com> Co-authored-by: Neetika Singhal <neetiks@amazon.com> Co-authored-by: Jay Deng <jayd0104@gmail.com> Co-authored-by: Gaurav Bafna <85113518+gbbafna@users.noreply.github.com> Co-authored-by: Andrew Ross <andrross@amazon.com>
opensearch-trigger-bot bot
added a commit
that referenced
this pull request
Jul 30, 2024
…de level (#14708) * add logic to add headers to Task Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add logic to add queryGroupId to task headers Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * remove redundant code Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add changelog entry Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * address comments Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * fix precommit Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add UTs for RemoteIndexMetadataManager (#14660) Signed-off-by: Shivansh Arora <hishiv@amazon.com> Co-authored-by: Arpit-Bandejiya <abandeji@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix match_phrase_prefix_query not working on text field with multiple values and index_prefixes (#10959) * Fix match_phrase_prefix_query not working on text field with multiple values and index_prefixes Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Add more test Signed-off-by: Gao Binlong <gbinlong@amazon.com> * modify change log Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Fix test failure Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Change the indexAnalyzer used by prefix field Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Skip old version for yaml test Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Optimize some code Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Fix test failure Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Modify yaml test description Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Remove the name parameter for setAnalyzer() Signed-off-by: Gao Binlong <gbinlong@amazon.com> --------- Signed-off-by: Gao Binlong <gbinlong@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Offline calculation of total shard per node and caching it for weight calculation inside LocalShardBalancer (#14675) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * [bug fix] validate lower bound for top n size (#14587) Signed-off-by: Chenyang Ji <cyji@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Create SystemIndexRegistry with helper method matchesSystemIndex (#14415) * Create new extension point in SystemIndexPlugin for a single plugin to get registered system indices Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add to CHANGELOG Signed-off-by: Craig Perkins <cwperx@amazon.com> * WIP on system indices from IndexNameExpressionResolver Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add test in IndexNameExpressionResolverTests Signed-off-by: Craig Perkins <cwperx@amazon.com> * Remove changes in SystemIndexPlugin Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add method in IndexNameExpressionResolver to get matching system indices Signed-off-by: Craig Perkins <cwperx@amazon.com> * Show how resolver can be chained to get system indices Signed-off-by: Craig Perkins <cwperx@amazon.com> * Fix forbiddenApis check Signed-off-by: Craig Perkins <cwperx@amazon.com> * Update CHANGELOG Signed-off-by: Craig Perkins <cwperx@amazon.com> * Make SystemIndices internal Signed-off-by: Craig Perkins <cwperx@amazon.com> * Remove unneeded changes Signed-off-by: Craig Perkins <cwperx@amazon.com> * Fix CI failures Signed-off-by: Craig Perkins <cwperx@amazon.com> * Fix precommit errors Signed-off-by: Craig Perkins <cwperx@amazon.com> * Use Regex instead of WildcardMatcher Signed-off-by: Craig Perkins <cwperx@amazon.com> * Address code review feedback Signed-off-by: Craig Perkins <cwperx@amazon.com> * Allow caller to pass index expressions Signed-off-by: Craig Perkins <cwperx@amazon.com> * Create SystemIndexRegistry Signed-off-by: Craig Perkins <cwperx@amazon.com> * Update CHANGELOG Signed-off-by: Craig Perkins <cwperx@amazon.com> * Remove singleton limitation Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add javadoc Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add @experimentalapi annotation Signed-off-by: Craig Perkins <cwperx@amazon.com> --------- Signed-off-by: Craig Perkins <cwperx@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Refactor Grok validate pattern to iterative approach (#14206) * grok validate patterns recusrion to iterative Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> * Add max depth in resolving a pattern to avoid OOM Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> * change path from deque to arraylist Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> * rename queue to stack Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> * Change max depth to 500 Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> * typo originPatternName fix Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> * spotless Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> --------- Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump opentelemetry from 1.39.0 to 1.40.0 (#14674) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump jackson from 2.17.1 to 2.17.2 (#14687) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add release notes for release 1.3.18 (#14699) Signed-off-by: Zelin Hao <zelinhao@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump reactor from 3.5.19 to 3.5.20 (#14697) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add unit tests for read flow of RemoteClusterStateService and bug fix for transient settings (#14476) Signed-off-by: Shivansh Arora <hishiv@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Update version check for the bug fix of match_phrase_prefix_query not working on text field with multiple values and index_prefixes (#14703) Signed-off-by: Gao Binlong <gbinlong@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Remove unnecessary cast to int from test (#14696) Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * print reason why parent task was cancelled (#14604) Signed-off-by: kkewwei <kkewwei@163.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Use set of shard routing for shard in unassigned shard batch check. (#14533) Signed-off-by: Swetha Guptha <gupthasg@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add versioning for UploadedIndexMetadata (#14677) * Add versioning for UploadedIndexMetadata * Handle componentPrefix for backward compatibility Signed-off-by: Sooraj Sinha <soosinha@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix: update help output for _cat (#14722) * fixed help output for _cat Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io> * updated changelog Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io> * updated changelog Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io> --------- Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix hdfs-fixture kerb-admin & hadoop-minicluster dependencies are not being updated / false positive reports on CVEs (#14729) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Update to Gradle 8.9 (#14574) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix hdfs-fixture hadoop-minicluster dependencies are not being updated / false positive reports on CVEs (#14732) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add `strict_allow_templates` dynamic mapping option (#14555) * The dynamic mapping parameter supports strict_allow_templates Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Modify change log Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Modify skip version in yml test file Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Refactor some code Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Keep the old methods Signed-off-by: Gao Binlong <gbinlong@amazon.com> * change public to private Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Optimize some code Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Do not override toString method for Dynamic Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Optimize some code and modify the changelog Signed-off-by: Gao Binlong <gbinlong@amazon.com> --------- Signed-off-by: Gao Binlong <gbinlong@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump net.minidev:json-smart from 2.5.0 to 2.5.1 in /plugins/repository-azure (#14748) * Bump net.minidev:json-smart in /plugins/repository-azure Bumps [net.minidev:json-smart](https://github.com/netplex/json-smart-v2) from 2.5.0 to 2.5.1. - [Release notes](https://github.com/netplex/json-smart-v2/releases) - [Commits](netplex/json-smart-v2@2.5.0...2.5.1) --- updated-dependencies: - dependency-name: net.minidev:json-smart dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * remove query insights plugin from core (#14743) Signed-off-by: Chenyang Ji <cyji@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add `strict_allow_templates` dynamic mapping option (#14555) (#14737) (#14742) * The dynamic mapping parameter supports strict_allow_templates * Modify change log * Modify skip version in yml test file * Refactor some code * Keep the old methods * change public to private * Optimize some code * Do not override toString method for Dynamic * Optimize some code and modify the changelog --------- (cherry picked from commit 6b8b3ef) Signed-off-by: Gao Binlong <gbinlong@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix create or update alias API doesn't throw exception for unsupported parameters (#14719) * Fix create or update alias API doesn't throw exception for unsupported parameters Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Update version check in yml test Signed-off-by: Gao Binlong <gbinlong@amazon.com> * modify change log Signed-off-by: Gao Binlong <gbinlong@amazon.com> --------- Signed-off-by: Gao Binlong <gbinlong@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Remove query categorization from core (#14759) * Remove query categorization from core Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Add changelog Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Trigger Build Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> --------- Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add changes to propagate queryGroupId across child requests and nodes (#14614) * add query group header propagator Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * apply spotless check Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add new propagator in ThreadContext Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * spotlessApply Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * address comments Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (#14610) * Bump com.microsoft.azure:msal4j in /plugins/repository-azure Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0. - [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases) - [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt) - [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0) --- updated-dependencies: - dependency-name: com.microsoft.azure:msal4j dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * [Bugfix] Fix ICacheKeySerializerTests flakiness (#14564) * Fix testInvalidInput flakiness Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Addressed andrross's comment Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun security check Signed-off-by: Peter Alfonsi <petealft@amazon.com> --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Correct typo in method name (#14621) Signed-off-by: vatsal <vatsal.v.anand@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Refactoring FilterPath.parse by using an iterative approach instead of recursion. (#14200) * Refactor FilterPath parse function (#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Implement unit tests for FilterPathTests (#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Write warn log if Filter is empty; Add comments (#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Add changelog Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove unnecessary log statement Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove unused logger Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Spotless apply Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove incorrect changelog Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> --------- Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(#14612) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (#14513) * Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata * Correct the check for deciding upload of hashes of consistent settings Signed-off-by: Sooraj Sinha <soosinha@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add changelog Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add PR link changelog Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (#14613) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix assertion failure while deleting remote backed index (#14601) Signed-off-by: Sachin Kale <kalsac@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Allow system index warning in OpenSearchRestTestCase.refreshAllIndices (#14635) * Allow system index warning Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add to CHANGELOG Signed-off-by: Craig Perkins <cwperx@amazon.com> * Address code review comments Signed-off-by: Craig Perkins <cwperx@amazon.com> --------- Signed-off-by: Craig Perkins <cwperx@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Star tree codec changes (#14514) --------- Signed-off-by: Bharathwaj G <bharath78910@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump com.github.spullara.mustache.java:compiler from 0.9.13 to 0.9.14 in /modules/lang-mustache (#14672) * Bump com.github.spullara.mustache.java:compiler Bumps [com.github.spullara.mustache.java:compiler](https://github.com/spullara/mustache.java) from 0.9.13 to 0.9.14. - [Commits](spullara/mustache.java@mustache.java-0.9.13...mustache.java-0.9.14) --- updated-dependencies: - dependency-name: com.github.spullara.mustache.java:compiler dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump net.minidev:accessors-smart from 2.5.0 to 2.5.1 in /plugins/repository-azure (#14673) * Bump net.minidev:accessors-smart in /plugins/repository-azure Bumps [net.minidev:accessors-smart](https://github.com/netplex/json-smart-v2) from 2.5.0 to 2.5.1. - [Release notes](https://github.com/netplex/json-smart-v2/releases) - [Commits](netplex/json-smart-v2@2.5.0...2.5.1) --- updated-dependencies: - dependency-name: net.minidev:accessors-smart dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * move query group thread context propagator out of ThreadContext Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> --------- Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Peter Alfonsi <petealft@amazon.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Sooraj Sinha <soosinha@amazon.com> Signed-off-by: Sachin Kale <kalsac@amazon.com> Signed-off-by: Craig Perkins <cwperx@amazon.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com> Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com> Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com> Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com> Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com> Co-authored-by: Sachin Kale <sachinpkale@gmail.com> Co-authored-by: Craig Perkins <cwperx@amazon.com> Co-authored-by: Bharathwaj G <bharath78910@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add consumers to remote store based index settings (#14764) Signed-off-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add matchesPluginSystemIndexPattern to SystemIndexRegistry (#14750) * Add matchesPluginSystemIndexPattern to SystemIndexRegistry Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add to CHANGELOG Signed-off-by: Craig Perkins <cwperx@amazon.com> * Use single data structure to keep track of system indices Signed-off-by: Craig Perkins <cwperx@amazon.com> * Address code review comments Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add test for getAllDescriptors Signed-off-by: Craig Perkins <cwperx@amazon.com> * Update server/src/main/java/org/opensearch/indices/SystemIndexRegistry.java Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Craig Perkins <craig5008@gmail.com> --------- Signed-off-by: Craig Perkins <cwperx@amazon.com> Signed-off-by: Craig Perkins <craig5008@gmail.com> Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * SPI for loading ABC templates (#14659) * SPI for loading ABC templates Signed-off-by: mgodwan <mgodwan@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix bulk upsert ignores the default_pipeline and final_pipeline when the auto-created index matches the index template (#12891) * Fix bulk upsert ignores the default_pipeline and final_pipeline when auto-created index matches with the index template Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Modify changelog & comment Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Use new approach Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Fix test failure Signed-off-by: Gao Binlong <gbinlong@amazon.com> --------- Signed-off-by: Gao Binlong <gbinlong@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix flaky test due to node being used across all tests (#14787) Signed-off-by: Mohit Godwani <mgodwan@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Star Tree Implementation [OnHeap] (#14512) --------- Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add Gao Binlong as maintainer (#14796) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Clear ehcache disk cache files during initialization (#14738) * Clear ehcache disk cache files during initialization Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding UT to fix line coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comment Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding more Uts for better line coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Throwing exception in case we fail to clear cache files during startup Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding more UTs Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding a UT for more coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing gradle build Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Update ehcache disk cache close() logic Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Refactor remote-routing-table service inline with remote state interfaces (#14668) --------- Signed-off-by: Arpit Bandejiya <abandeji@amazon.com> Signed-off-by: Arpit-Bandejiya <abandeji@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Set version to 2.15 for determining metadata during migration to remote store Signed-off-by: Sandeep Kumawat <skumwt@amazon.com> Co-authored-by: Sandeep Kumawat <skumwt@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix bulk upsert ignores the default_pipeline and final_pipeline when the auto-created index matches the index template (#14793) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix create or update alias API doesn't throw exception for unsupported parameters (#14769) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Change RCSS info logs to debug (#14814) Signed-off-by: Shivansh Arora <hishiv@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * [Bugfix] Fix NPE in ReplicaShardAllocator (#13993) (#14385) * [Bugfix] Fix NPE in ReplicaShardAllocator (#13993) Signed-off-by: Daniil Roman <daniilroman.cv@gmail.com> * Add fix info to CHANGELOG.md Signed-off-by: Daniil Roman <danroman17397@gmail.com> --------- Signed-off-by: Daniil Roman <daniilroman.cv@gmail.com> Signed-off-by: Daniil Roman <danroman17397@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Run performance benchmark on pull requests (#14760) * add performance benchmark workflow for pull requests Signed-off-by: Rishabh Singh <sngri@amazon.com> * Update PERFORMANCE_BENCHMARKS.md Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> * Update PERFORMANCE_BENCHMARKS.md Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> * Update .github/workflows/benchmark-pull-request.yml Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> * Update .github/workflows/benchmark-pull-request.yml Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> * Update .github/workflows/benchmark-pull-request.yml Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> * Update .github/workflows/benchmark-pull-request.yml Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> --------- Signed-off-by: Rishabh Singh <sngri@amazon.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * fix constant_keyword field type (#14807) Signed-off-by: kkewwei <kkewwei@163.com> test Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * [Remote Store Migration] Reconcile remote store based index settings during STRICT mode switch (#14792) Signed-off-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add prefix mode verification setting for repository verification (#14790) * Add prefix mode verification setting for repository verification Signed-off-by: Ashish Singh <ssashish@amazon.com> * Add UTs and randomise prefix mode repository verification Signed-off-by: Ashish Singh <ssashish@amazon.com> * Incorporate PR review feedback Signed-off-by: Ashish Singh <ssashish@amazon.com> --------- Signed-off-by: Ashish Singh <ssashish@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add length check on comment body for benchmark workflow (#14834) Signed-off-by: Rishabh Singh <sngri@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add restore-from-snapshot test procedure for snapshot run benchmark config (#14842) Signed-off-by: Rishabh Singh <sngri@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix env variable name typo (#14843) Signed-off-by: Rishabh Singh <sngri@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Use circuit breaker in InternalHistogram when adding empty buckets (#14754) * introduce circuit breaker in InternalHistogram Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * use circuit breaker from reduce context Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * add test Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * revert use_real_memory change in OpenSearchNode Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * add change log Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> --------- Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * [Remote State] Create interface RemoteEntitiesManager (#14671) * Create interface RemoteEntitiesManager Signed-off-by: Shivansh Arora <hishiv@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Optimise TransportNodesAction to not send DiscoveryNodes for NodeStat… (#14749) * Optimize TransportNodesAction to not send DiscoveryNodes for NodeStats, NodesInfo and ClusterStats call Signed-off-by: Pranshu Shukla <pranshushukla06@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Enabling term version check on local state for all ClusterManager Read Transport Actions (#14273) * enabling term version check on local state for all admin read actions Signed-off-by: Rajiv Kumar Vaidyanathan <rajivkv@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Reduce logging in DEBUG for MasterService:run (#14795) * Reduce logging in DEBUG for MasteService:run by introducing short and long summary in Taskbatcher Signed-off-by: Sumit Bansal <sumitsb@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add SplitResponseProcessor to Search Pipelines (#14800) * Add SplitResponseProcessor for search pipelines Signed-off-by: Daniel Widdis <widdis@gmail.com> * Register the split processor factory Signed-off-by: Daniel Widdis <widdis@gmail.com> * Address code review comments Signed-off-by: Daniel Widdis <widdis@gmail.com> * Avoid list copy by casting array Signed-off-by: Daniel Widdis <widdis@gmail.com> --------- Signed-off-by: Daniel Widdis <widdis@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add integration tests for RemoteRoutingTable Service. (#14631) Signed-off-by: Shailendra Singh <singhlhs@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add SortResponseProcessor to Search Pipelines (#14785) * Add SortResponseProcessor for search pipelines Signed-off-by: Daniel Widdis <widdis@gmail.com> * Add stupid and unnecessary javadocs to satisfy overly strict CI Signed-off-by: Daniel Widdis <widdis@gmail.com> * Split casting and sorting methods for readability Signed-off-by: Daniel Widdis <widdis@gmail.com> * Register the sort processor factory Signed-off-by: Daniel Widdis <widdis@gmail.com> * Address code review comments Signed-off-by: Daniel Widdis <widdis@gmail.com> * Cast individual list elements to avoid creating two lists Signed-off-by: Daniel Widdis <widdis@gmail.com> * Add yamlRestTests Signed-off-by: Daniel Widdis <widdis@gmail.com> * Clarify why there's unusual sorting Signed-off-by: Daniel Widdis <widdis@gmail.com> * Use instanceof instead of isAssignableFrom Signed-off-by: Daniel Widdis <widdis@gmail.com> --------- Signed-off-by: Daniel Widdis <widdis@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix allowUnmappedFields, mapUnmappedFieldAsString settings to be applied when parsing query string query (#13957) * Modify to invoke QueryShardContext.fieldMapper() method to apply allowUnmappedFields and mapUnmappedFieldAsString settings Signed-off-by: imyp92 <pyw5420@gmail.com> * Add test cases to verify returning 400 responses if unmapped fields are included for some types of query Signed-off-by: imyp92 <pyw5420@gmail.com> * Add changelog Signed-off-by: imyp92 <pyw5420@gmail.com> --------- Signed-off-by: imyp92 <pyw5420@gmail.com> Signed-off-by: gaobinlong <gbinlong@amazon.com> Co-authored-by: gaobinlong <gbinlong@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump com.microsoft.azure:msal4j from 1.16.0 to 1.16.1 in /plugins/repository-azure (#14857) * Bump com.microsoft.azure:msal4j in /plugins/repository-azure Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.16.0 to 1.16.1. - [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases) - [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt) - [Commits](AzureAD/microsoft-authentication-library-for-java@v1.16.0...v1.16.1) --- updated-dependencies: - dependency-name: com.microsoft.azure:msal4j dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump com.gradle.develocity from 3.17.5 to 3.17.6 (#14856) * Bump com.gradle.develocity from 3.17.5 to 3.17.6 Bumps com.gradle.develocity from 3.17.5 to 3.17.6. --- updated-dependencies: - dependency-name: com.gradle.develocity dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump org.jline:jline in /test/fixtures/hdfs-fixture (#14859) Bumps [org.jline:jline](https://github.com/jline/jline3) from 3.26.2 to 3.26.3. - [Release notes](https://github.com/jline/jline3/releases) - [Changelog](https://github.com/jline/jline3/blob/master/changelog.md) - [Commits](jline/jline3@jline-parent-3.26.2...jline-parent-3.26.3) --- updated-dependencies: - dependency-name: org.jline:jline dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Use Lucene provided Persian stem (#14847) Lucene provided Persian stem apparently isn't hooked yet and this change is doing that based on what is done for Arabic stem support. Signed-off-by: Ebrahim Byagowi <ebrahim@gnu.org> Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump actions/checkout from 2 to 4 (#14858) * Bump actions/checkout from 2 to 4 Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 4. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v2...v4) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Deprecate batch_size parameter on bulk API (#14725) By default the full _bulk payload will be passed to ingest processors as a batch, with any sub batching logic to be implemented by each processor if necessary. Signed-off-by: Liyun Xiu <xiliyun@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add perms for remote snapshot cache eviction on scripted query (#14411) Signed-off-by: Finn Carroll <carrofin@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add transport interceptor to populate queryGroupId in task headers Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add rest, transport layer changes for Hot to warm tiering - dedicated setup (#13980) Signed-off-by: Neetika Singhal <neetiks@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Create listener to refresh search thread resource usage (#14832) * [bug fix] fix incorrect coordinator node search resource usages Signed-off-by: Chenyang Ji <cyji@amazon.com> * fix bug on serialization when passing task resource usage to coordinator Signed-off-by: Chenyang Ji <cyji@amazon.com> * add more unit tests Signed-off-by: Chenyang Ji <cyji@amazon.com> * remove query insights plugin related code Signed-off-by: Chenyang Ji <cyji@amazon.com> * create per request listener to refresh task resource usage Signed-off-by: Chenyang Ji <cyji@amazon.com> * Make new listener API public Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Add changelog Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove wrong files added Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Address review comments Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Build fix Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Make singleton Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Address review comments Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Make sure listener runs before plugin listeners Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Spotless Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Minor fix Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> --------- Signed-off-by: Chenyang Ji <cyji@amazon.com> Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Signed-off-by: Jay Deng <jayd0104@gmail.com> Co-authored-by: Chenyang Ji <cyji@amazon.com> Co-authored-by: Jay Deng <jayd0104@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Caching avg total bytes and avg free bytes inside ClusterInfo (#14851) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Use default value when index.number_of_replicas is null (#14812) * Use default value when index.number_of_replicas is null Signed-off-by: Liyun Xiu <xiliyun@amazon.com> * Add integration test Signed-off-by: Liyun Xiu <xiliyun@amazon.com> * Add changelog Signed-off-by: Liyun Xiu <xiliyun@amazon.com> --------- Signed-off-by: Liyun Xiu <xiliyun@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * [Remote Routing Table] Implement write and read flow for shard diff file. (#14684) * Implement write and read flow to upload/download shard diff file. Signed-off-by: Shailendra Singh <singhlhs@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Optimized ClusterStatsIndices to precomute shard stats (#14426) * Optimize Cluster Stats Indices to precomute node level stats Signed-off-by: Pranshu Shukla <pranshushukla06@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix constraint bug which allows more primary shards than average primary shards per index (#14908) Signed-off-by: Gaurav Bafna <gbbafna@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Optmising AwarenessAllocationDecider for hashmap.get call (#14761) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * update comment Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix IngestServiceTests.testBulkRequestExecutionWithFailures (#14918) The test would previously fail if the randomness led to only a single indexing request being included in the bulk payload. This change guarantees multiple indexing requests in order to ensure the batch logic kicks in. Also replace some unneeded mocks with real classes. Signed-off-by: Andrew Ross <andrross@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add queryGroupTask Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * remove unnecessary imports Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add QueryGroupTask tests Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * rename WLM transport request handler Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add CHANGELOG entry Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * fix ut Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * address comments Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * fix UT to remove the verify for final method Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * apply spotless Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> --------- Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> Signed-off-by: Shivansh Arora <hishiv@amazon.com> Signed-off-by: Gao Binlong <gbinlong@amazon.com> Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Chenyang Ji <cyji@amazon.com> Signed-off-by: Craig Perkins <cwperx@amazon.com> Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Zelin Hao <zelinhao@amazon.com> Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io> Signed-off-by: kkewwei <kkewwei@163.com> Signed-off-by: Swetha Guptha <gupthasg@amazon.com> Signed-off-by: Sooraj Sinha <soosinha@amazon.com> Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Signed-off-by: Peter Alfonsi <petealft@amazon.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> Signed-off-by: Sachin Kale <kalsac@amazon.com> Signed-off-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com> Signed-off-by: Craig Perkins <craig5008@gmail.com> Signed-off-by: mgodwan <mgodwan@amazon.com> Signed-off-by: Mohit Godwani <mgodwan@amazon.com> Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sandeep Kumawat <skumwt@amazon.com> Signed-off-by: Daniil Roman <daniilroman.cv@gmail.com> Signed-off-by: Daniil Roman <danroman17397@gmail.com> Signed-off-by: Rishabh Singh <sngri@amazon.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> Signed-off-by: Ashish Singh <ssashish@amazon.com> Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> Signed-off-by: Pranshu Shukla <pranshushukla06@gmail.com> Signed-off-by: Rajiv Kumar Vaidyanathan <rajivkv@amazon.com> Signed-off-by: Sumit Bansal <sumitsb@amazon.com> Signed-off-by: Daniel Widdis <widdis@gmail.com> Signed-off-by: Shailendra Singh <singhlhs@amazon.com> Signed-off-by: imyp92 <pyw5420@gmail.com> Signed-off-by: gaobinlong <gbinlong@amazon.com> Signed-off-by: Ebrahim Byagowi <ebrahim@gnu.org> Signed-off-by: Liyun Xiu <xiliyun@amazon.com> Signed-off-by: Finn Carroll <carrofin@amazon.com> Signed-off-by: Neetika Singhal <neetiks@amazon.com> Signed-off-by: Jay Deng <jayd0104@gmail.com> Signed-off-by: Gaurav Bafna <gbbafna@amazon.com> Signed-off-by: Andrew Ross <andrross@amazon.com> Co-authored-by: Shivansh Arora <hishiv@amazon.com> Co-authored-by: Arpit-Bandejiya <abandeji@amazon.com> Co-authored-by: gaobinlong <gbinlong@amazon.com> Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com> Co-authored-by: Chenyang Ji <cyji@amazon.com> Co-authored-by: Craig Perkins <cwperx@amazon.com> Co-authored-by: Sandesh Kumar <sandeshkr419@gmail.com> Co-authored-by: Andriy Redko <andriy.redko@aiven.io> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Lukáš Vlček <lukas.vlcek@aiven.io> Co-authored-by: kkewwei <kkewwei@163.com> Co-authored-by: SwethaGuptha <156877431+SwethaGuptha@users.noreply.github.com> Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com> Co-authored-by: Ahmed Sobeh <ahmed.sobeh@aiven.io> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com> Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com> Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com> Co-authored-by: Sachin Kale <sachinpkale@gmail.com> Co-authored-by: Bharathwaj G <bharath78910@gmail.com> Co-authored-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com> Co-authored-by: Craig Perkins <craig5008@gmail.com> Co-authored-by: Andriy Redko <drreta@gmail.com> Co-authored-by: Mohit Godwani <81609427+mgodwan@users.noreply.github.com> Co-authored-by: Sarthak Aggarwal <sarthagg@amazon.com> Co-authored-by: Sagar <99425694+sgup432@users.noreply.github.com> Co-authored-by: Sandeep Kumawat <2025sandeepkumawat@gmail.com> Co-authored-by: Sandeep Kumawat <skumwt@amazon.com> Co-authored-by: Daniil Roman <danroman17397@gmail.com> Co-authored-by: Rishabh Singh <rishabhksingh@gmail.com> Co-authored-by: kkewwei <kewei.11@bytedance.com> Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> Co-authored-by: Ashish Singh <ssashish@amazon.com> Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com> Co-authored-by: Pranshu Shukla <55992439+Pranshu-S@users.noreply.github.com> Co-authored-by: rajiv-kv <157019998+rajiv-kv@users.noreply.github.com> Co-authored-by: Sumit Bansal <sumit.asr@gmail.com> Co-authored-by: Daniel Widdis <widdis@gmail.com> Co-authored-by: shailendra0811 <167273922+shailendra0811@users.noreply.github.com> Co-authored-by: Park, Yeongwu <pyw5420@gmail.com> Co-authored-by: ebraminio <ebraminio@gmail.com> Co-authored-by: Liyun Xiu <xiliyun@amazon.com> Co-authored-by: Finn <carrofin@amazon.com> Co-authored-by: Neetika Singhal <neetiks@amazon.com> Co-authored-by: Jay Deng <jayd0104@gmail.com> Co-authored-by: Gaurav Bafna <85113518+gbbafna@users.noreply.github.com> Co-authored-by: Andrew Ross <andrross@amazon.com> (cherry picked from commit eb306d2) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
jed326
added a commit
that referenced
this pull request
Aug 7, 2024
…de level (#14708) (#15029) * add logic to add headers to Task * add logic to add queryGroupId to task headers * remove redundant code * add changelog entry * address comments * fix precommit * Add UTs for RemoteIndexMetadataManager (#14660) * Fix match_phrase_prefix_query not working on text field with multiple values and index_prefixes (#10959) * Fix match_phrase_prefix_query not working on text field with multiple values and index_prefixes * Add more test * modify change log * Fix test failure * Change the indexAnalyzer used by prefix field * Skip old version for yaml test * Optimize some code * Fix test failure * Modify yaml test description * Remove the name parameter for setAnalyzer() --------- * Offline calculation of total shard per node and caching it for weight calculation inside LocalShardBalancer (#14675) * [bug fix] validate lower bound for top n size (#14587) * Create SystemIndexRegistry with helper method matchesSystemIndex (#14415) * Create new extension point in SystemIndexPlugin for a single plugin to get registered system indices * Add to CHANGELOG * WIP on system indices from IndexNameExpressionResolver * Add test in IndexNameExpressionResolverTests * Remove changes in SystemIndexPlugin * Add method in IndexNameExpressionResolver to get matching system indices * Show how resolver can be chained to get system indices * Fix forbiddenApis check * Update CHANGELOG * Make SystemIndices internal * Remove unneeded changes * Fix CI failures * Fix precommit errors * Use Regex instead of WildcardMatcher * Address code review feedback * Allow caller to pass index expressions * Create SystemIndexRegistry * Update CHANGELOG * Remove singleton limitation * Add javadoc * Add @experimentalapi annotation --------- * Refactor Grok validate pattern to iterative approach (#14206) * grok validate patterns recusrion to iterative * Add max depth in resolving a pattern to avoid OOM * change path from deque to arraylist * rename queue to stack * Change max depth to 500 * typo originPatternName fix * spotless --------- * Bump opentelemetry from 1.39.0 to 1.40.0 (#14674) * Bump jackson from 2.17.1 to 2.17.2 (#14687) * Add release notes for release 1.3.18 (#14699) * Bump reactor from 3.5.19 to 3.5.20 (#14697) * Add unit tests for read flow of RemoteClusterStateService and bug fix for transient settings (#14476) * Update version check for the bug fix of match_phrase_prefix_query not working on text field with multiple values and index_prefixes (#14703) * Remove unnecessary cast to int from test (#14696) * print reason why parent task was cancelled (#14604) * Use set of shard routing for shard in unassigned shard batch check. (#14533) * Add versioning for UploadedIndexMetadata (#14677) * Add versioning for UploadedIndexMetadata * Handle componentPrefix for backward compatibility * Fix: update help output for _cat (#14722) * fixed help output for _cat * updated changelog * updated changelog --------- * Fix hdfs-fixture kerb-admin & hadoop-minicluster dependencies are not being updated / false positive reports on CVEs (#14729) * Update to Gradle 8.9 (#14574) * Fix hdfs-fixture hadoop-minicluster dependencies are not being updated / false positive reports on CVEs (#14732) * Add `strict_allow_templates` dynamic mapping option (#14555) * The dynamic mapping parameter supports strict_allow_templates * Modify change log * Modify skip version in yml test file * Refactor some code * Keep the old methods * change public to private * Optimize some code * Do not override toString method for Dynamic * Optimize some code and modify the changelog --------- * Bump net.minidev:json-smart from 2.5.0 to 2.5.1 in /plugins/repository-azure (#14748) * Bump net.minidev:json-smart in /plugins/repository-azure Bumps [net.minidev:json-smart](https://github.com/netplex/json-smart-v2) from 2.5.0 to 2.5.1. - [Release notes](https://github.com/netplex/json-smart-v2/releases) - [Commits](netplex/json-smart-v2@2.5.0...2.5.1) --- updated-dependencies: - dependency-name: net.minidev:json-smart dependency-type: direct:production update-type: version-update:semver-patch ... * Updating SHAs * Update changelog --------- * remove query insights plugin from core (#14743) * Add `strict_allow_templates` dynamic mapping option (#14555) (#14737) (#14742) * The dynamic mapping parameter supports strict_allow_templates * Modify change log * Modify skip version in yml test file * Refactor some code * Keep the old methods * change public to private * Optimize some code * Do not override toString method for Dynamic * Optimize some code and modify the changelog --------- (cherry picked from commit 6b8b3ef) * Fix create or update alias API doesn't throw exception for unsupported parameters (#14719) * Fix create or update alias API doesn't throw exception for unsupported parameters * Update version check in yml test * modify change log --------- * Remove query categorization from core (#14759) * Remove query categorization from core * Add changelog * Trigger Build --------- * Add changes to propagate queryGroupId across child requests and nodes (#14614) * add query group header propagator * apply spotless check * add new propagator in ThreadContext * spotlessApply * address comments * Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (#14610) * Bump com.microsoft.azure:msal4j in /plugins/repository-azure Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0. - [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases) - [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt) - [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0) --- updated-dependencies: - dependency-name: com.microsoft.azure:msal4j dependency-type: direct:production update-type: version-update:semver-minor ... * Updating SHAs * Update changelog --------- * [Bugfix] Fix ICacheKeySerializerTests flakiness (#14564) * Fix testInvalidInput flakiness * Addressed andrross's comment * rerun security check --------- * Correct typo in method name (#14621) * Refactoring FilterPath.parse by using an iterative approach instead of recursion. (#14200) * Refactor FilterPath parse function (#12067) * Implement unit tests for FilterPathTests (#12067) * Write warn log if Filter is empty; Add comments (#12067) * Add changelog * Remove unnecessary log statement * Remove unused logger * Spotless apply * Remove incorrect changelog --------- * Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(#14612) * Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (#14513) * Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata * Correct the check for deciding upload of hashes of consistent settings * add changelog * add PR link changelog * Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (#14613) * Fix assertion failure while deleting remote backed index (#14601) * Allow system index warning in OpenSearchRestTestCase.refreshAllIndices (#14635) * Allow system index warning * Add to CHANGELOG * Address code review comments --------- * Star tree codec changes (#14514) --------- * Bump com.github.spullara.mustache.java:compiler from 0.9.13 to 0.9.14 in /modules/lang-mustache (#14672) * Bump com.github.spullara.mustache.java:compiler Bumps [com.github.spullara.mustache.java:compiler](https://github.com/spullara/mustache.java) from 0.9.13 to 0.9.14. - [Commits](spullara/mustache.java@mustache.java-0.9.13...mustache.java-0.9.14) --- updated-dependencies: - dependency-name: com.github.spullara.mustache.java:compiler dependency-type: direct:production update-type: version-update:semver-patch ... * Updating SHAs * Update changelog --------- * Bump net.minidev:accessors-smart from 2.5.0 to 2.5.1 in /plugins/repository-azure (#14673) * Bump net.minidev:accessors-smart in /plugins/repository-azure Bumps [net.minidev:accessors-smart](https://github.com/netplex/json-smart-v2) from 2.5.0 to 2.5.1. - [Release notes](https://github.com/netplex/json-smart-v2/releases) - [Commits](netplex/json-smart-v2@2.5.0...2.5.1) --- updated-dependencies: - dependency-name: net.minidev:accessors-smart dependency-type: direct:production update-type: version-update:semver-patch ... * Updating SHAs * Update changelog --------- * move query group thread context propagator out of ThreadContext --------- * Add consumers to remote store based index settings (#14764) * Add matchesPluginSystemIndexPattern to SystemIndexRegistry (#14750) * Add matchesPluginSystemIndexPattern to SystemIndexRegistry * Add to CHANGELOG * Use single data structure to keep track of system indices * Address code review comments * Add test for getAllDescriptors * Update server/src/main/java/org/opensearch/indices/SystemIndexRegistry.java --------- * SPI for loading ABC templates (#14659) * SPI for loading ABC templates * Fix bulk upsert ignores the default_pipeline and final_pipeline when the auto-created index matches the index template (#12891) * Fix bulk upsert ignores the default_pipeline and final_pipeline when auto-created index matches with the index template * Modify changelog & comment * Use new approach * Fix test failure --------- * Fix flaky test due to node being used across all tests (#14787) * Star Tree Implementation [OnHeap] (#14512) --------- * Add Gao Binlong as maintainer (#14796) * Clear ehcache disk cache files during initialization (#14738) * Clear ehcache disk cache files during initialization * Adding UT to fix line coverage * Addressing comment * Adding more Uts for better line coverage * Throwing exception in case we fail to clear cache files during startup * Adding more UTs * Adding a UT for more coverage * Fixing gradle build * Update ehcache disk cache close() logic --------- * Refactor remote-routing-table service inline with remote state interfaces (#14668) --------- * Set version to 2.15 for determining metadata during migration to remote store * Fix bulk upsert ignores the default_pipeline and final_pipeline when the auto-created index matches the index template (#14793) * Fix create or update alias API doesn't throw exception for unsupported parameters (#14769) * Change RCSS info logs to debug (#14814) * [Bugfix] Fix NPE in ReplicaShardAllocator (#13993) (#14385) * [Bugfix] Fix NPE in ReplicaShardAllocator (#13993) * Add fix info to CHANGELOG.md --------- * Run performance benchmark on pull requests (#14760) * add performance benchmark workflow for pull requests * Update PERFORMANCE_BENCHMARKS.md * Update PERFORMANCE_BENCHMARKS.md * Update .github/workflows/benchmark-pull-request.yml * Update .github/workflows/benchmark-pull-request.yml * Update .github/workflows/benchmark-pull-request.yml * Update .github/workflows/benchmark-pull-request.yml --------- * fix constant_keyword field type (#14807) test * [Remote Store Migration] Reconcile remote store based index settings during STRICT mode switch (#14792) * Add prefix mode verification setting for repository verification (#14790) * Add prefix mode verification setting for repository verification * Add UTs and randomise prefix mode repository verification * Incorporate PR review feedback --------- * add length check on comment body for benchmark workflow (#14834) * Add restore-from-snapshot test procedure for snapshot run benchmark config (#14842) * Fix env variable name typo (#14843) * Use circuit breaker in InternalHistogram when adding empty buckets (#14754) * introduce circuit breaker in InternalHistogram * use circuit breaker from reduce context * add test * revert use_real_memory change in OpenSearchNode * add change log --------- * [Remote State] Create interface RemoteEntitiesManager (#14671) * Create interface RemoteEntitiesManager * Optimise TransportNodesAction to not send DiscoveryNodes for NodeStat… (#14749) * Optimize TransportNodesAction to not send DiscoveryNodes for NodeStats, NodesInfo and ClusterStats call * Enabling term version check on local state for all ClusterManager Read Transport Actions (#14273) * enabling term version check on local state for all admin read actions * Reduce logging in DEBUG for MasterService:run (#14795) * Reduce logging in DEBUG for MasteService:run by introducing short and long summary in Taskbatcher * Add SplitResponseProcessor to Search Pipelines (#14800) * Add SplitResponseProcessor for search pipelines * Register the split processor factory * Address code review comments * Avoid list copy by casting array --------- * Add integration tests for RemoteRoutingTable Service. (#14631) * Add SortResponseProcessor to Search Pipelines (#14785) * Add SortResponseProcessor for search pipelines * Add stupid and unnecessary javadocs to satisfy overly strict CI * Split casting and sorting methods for readability * Register the sort processor factory * Address code review comments * Cast individual list elements to avoid creating two lists * Add yamlRestTests * Clarify why there's unusual sorting * Use instanceof instead of isAssignableFrom --------- * Fix allowUnmappedFields, mapUnmappedFieldAsString settings to be applied when parsing query string query (#13957) * Modify to invoke QueryShardContext.fieldMapper() method to apply allowUnmappedFields and mapUnmappedFieldAsString settings * Add test cases to verify returning 400 responses if unmapped fields are included for some types of query * Add changelog --------- * Bump com.microsoft.azure:msal4j from 1.16.0 to 1.16.1 in /plugins/repository-azure (#14857) * Bump com.microsoft.azure:msal4j in /plugins/repository-azure Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.16.0 to 1.16.1. - [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases) - [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt) - [Commits](AzureAD/microsoft-authentication-library-for-java@v1.16.0...v1.16.1) --- updated-dependencies: - dependency-name: com.microsoft.azure:msal4j dependency-type: direct:production update-type: version-update:semver-patch ... * Updating SHAs * Update changelog --------- * Bump com.gradle.develocity from 3.17.5 to 3.17.6 (#14856) * Bump com.gradle.develocity from 3.17.5 to 3.17.6 Bumps com.gradle.develocity from 3.17.5 to 3.17.6. --- updated-dependencies: - dependency-name: com.gradle.develocity dependency-type: direct:production update-type: version-update:semver-patch ... * Update changelog --------- * Bump org.jline:jline in /test/fixtures/hdfs-fixture (#14859) Bumps [org.jline:jline](https://github.com/jline/jline3) from 3.26.2 to 3.26.3. - [Release notes](https://github.com/jline/jline3/releases) - [Changelog](https://github.com/jline/jline3/blob/master/changelog.md) - [Commits](jline/jline3@jline-parent-3.26.2...jline-parent-3.26.3) --- updated-dependencies: - dependency-name: org.jline:jline dependency-type: direct:production update-type: version-update:semver-patch ... * Use Lucene provided Persian stem (#14847) Lucene provided Persian stem apparently isn't hooked yet and this change is doing that based on what is done for Arabic stem support. * Bump actions/checkout from 2 to 4 (#14858) * Bump actions/checkout from 2 to 4 Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 4. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v2...v4) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... * Update changelog --------- * Deprecate batch_size parameter on bulk API (#14725) By default the full _bulk payload will be passed to ingest processors as a batch, with any sub batching logic to be implemented by each processor if necessary. * Add perms for remote snapshot cache eviction on scripted query (#14411) * add transport interceptor to populate queryGroupId in task headers * Add rest, transport layer changes for Hot to warm tiering - dedicated setup (#13980) * Create listener to refresh search thread resource usage (#14832) * [bug fix] fix incorrect coordinator node search resource usages * fix bug on serialization when passing task resource usage to coordinator * add more unit tests * remove query insights plugin related code * create per request listener to refresh task resource usage * Make new listener API public * Add changelog * Remove wrong files added * Address review comments * Build fix * Make singleton * Address review comments * Make sure listener runs before plugin listeners * Spotless * Minor fix --------- * Caching avg total bytes and avg free bytes inside ClusterInfo (#14851) * Use default value when index.number_of_replicas is null (#14812) * Use default value when index.number_of_replicas is null * Add integration test * Add changelog --------- * [Remote Routing Table] Implement write and read flow for shard diff file. (#14684) * Implement write and read flow to upload/download shard diff file. * Optimized ClusterStatsIndices to precomute shard stats (#14426) * Optimize Cluster Stats Indices to precomute node level stats * Fix constraint bug which allows more primary shards than average primary shards per index (#14908) * Optmising AwarenessAllocationDecider for hashmap.get call (#14761) * update comment * Fix IngestServiceTests.testBulkRequestExecutionWithFailures (#14918) The test would previously fail if the randomness led to only a single indexing request being included in the bulk payload. This change guarantees multiple indexing requests in order to ensure the batch logic kicks in. Also replace some unneeded mocks with real classes. * add queryGroupTask * remove unnecessary imports * add QueryGroupTask tests * rename WLM transport request handler * add CHANGELOG entry * fix ut * address comments * fix UT to remove the verify for final method * apply spotless --------- (cherry picked from commit eb306d2) Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> Signed-off-by: Shivansh Arora <hishiv@amazon.com> Signed-off-by: Gao Binlong <gbinlong@amazon.com> Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Chenyang Ji <cyji@amazon.com> Signed-off-by: Craig Perkins <cwperx@amazon.com> Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Zelin Hao <zelinhao@amazon.com> Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io> Signed-off-by: kkewwei <kkewwei@163.com> Signed-off-by: Swetha Guptha <gupthasg@amazon.com> Signed-off-by: Sooraj Sinha <soosinha@amazon.com> Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Signed-off-by: Peter Alfonsi <petealft@amazon.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> Signed-off-by: Sachin Kale <kalsac@amazon.com> Signed-off-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com> Signed-off-by: Craig Perkins <craig5008@gmail.com> Signed-off-by: mgodwan <mgodwan@amazon.com> Signed-off-by: Mohit Godwani <mgodwan@amazon.com> Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sandeep Kumawat <skumwt@amazon.com> Signed-off-by: Daniil Roman <daniilroman.cv@gmail.com> Signed-off-by: Daniil Roman <danroman17397@gmail.com> Signed-off-by: Rishabh Singh <sngri@amazon.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> Signed-off-by: Ashish Singh <ssashish@amazon.com> Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> Signed-off-by: Pranshu Shukla <pranshushukla06@gmail.com> Signed-off-by: Rajiv Kumar Vaidyanathan <rajivkv@amazon.com> Signed-off-by: Sumit Bansal <sumitsb@amazon.com> Signed-off-by: Daniel Widdis <widdis@gmail.com> Signed-off-by: Shailendra Singh <singhlhs@amazon.com> Signed-off-by: imyp92 <pyw5420@gmail.com> Signed-off-by: gaobinlong <gbinlong@amazon.com> Signed-off-by: Ebrahim Byagowi <ebrahim@gnu.org> Signed-off-by: Liyun Xiu <xiliyun@amazon.com> Signed-off-by: Finn Carroll <carrofin@amazon.com> Signed-off-by: Neetika Singhal <neetiks@amazon.com> Signed-off-by: Jay Deng <jayd0104@gmail.com> Signed-off-by: Gaurav Bafna <gbbafna@amazon.com> Signed-off-by: Andrew Ross <andrross@amazon.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Shivansh Arora <hishiv@amazon.com> Co-authored-by: Arpit-Bandejiya <abandeji@amazon.com> Co-authored-by: gaobinlong <gbinlong@amazon.com> Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com> Co-authored-by: Chenyang Ji <cyji@amazon.com> Co-authored-by: Craig Perkins <cwperx@amazon.com> Co-authored-by: Sandesh Kumar <sandeshkr419@gmail.com> Co-authored-by: Andriy Redko <andriy.redko@aiven.io> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Lukáš Vlček <lukas.vlcek@aiven.io> Co-authored-by: kkewwei <kkewwei@163.com> Co-authored-by: SwethaGuptha <156877431+SwethaGuptha@users.noreply.github.com> Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com> Co-authored-by: Ahmed Sobeh <ahmed.sobeh@aiven.io> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com> Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com> Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com> Co-authored-by: Sachin Kale <sachinpkale@gmail.com> Co-authored-by: Bharathwaj G <bharath78910@gmail.com> Co-authored-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com> Co-authored-by: Craig Perkins <craig5008@gmail.com> Co-authored-by: Andriy Redko <drreta@gmail.com> Co-authored-by: Mohit Godwani <81609427+mgodwan@users.noreply.github.com> Co-authored-by: Sarthak Aggarwal <sarthagg@amazon.com> Co-authored-by: Sagar <99425694+sgup432@users.noreply.github.com> Co-authored-by: Sandeep Kumawat <2025sandeepkumawat@gmail.com> Co-authored-by: Sandeep Kumawat <skumwt@amazon.com> Co-authored-by: Daniil Roman <danroman17397@gmail.com> Co-authored-by: Rishabh Singh <rishabhksingh@gmail.com> Co-authored-by: kkewwei <kewei.11@bytedance.com> Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> Co-authored-by: Ashish Singh <ssashish@amazon.com> Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com> Co-authored-by: Pranshu Shukla <55992439+Pranshu-S@users.noreply.github.com> Co-authored-by: rajiv-kv <157019998+rajiv-kv@users.noreply.github.com> Co-authored-by: Sumit Bansal <sumit.asr@gmail.com> Co-authored-by: Daniel Widdis <widdis@gmail.com> Co-authored-by: shailendra0811 <167273922+shailendra0811@users.noreply.github.com> Co-authored-by: Park, Yeongwu <pyw5420@gmail.com> Co-authored-by: ebraminio <ebraminio@gmail.com> Co-authored-by: Liyun Xiu <xiliyun@amazon.com> Co-authored-by: Finn <carrofin@amazon.com> Co-authored-by: Neetika Singhal <neetiks@amazon.com> Co-authored-by: Jay Deng <jayd0104@gmail.com> Co-authored-by: Gaurav Bafna <85113518+gbbafna@users.noreply.github.com> Co-authored-by: Andrew Ross <andrross@amazon.com>
harshavamsi
pushed a commit
to harshavamsi/OpenSearch
that referenced
this pull request
Aug 20, 2024
…de level (opensearch-project#14708) * add logic to add headers to Task Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add logic to add queryGroupId to task headers Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * remove redundant code Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add changelog entry Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * address comments Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * fix precommit Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add UTs for RemoteIndexMetadataManager (opensearch-project#14660) Signed-off-by: Shivansh Arora <hishiv@amazon.com> Co-authored-by: Arpit-Bandejiya <abandeji@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix match_phrase_prefix_query not working on text field with multiple values and index_prefixes (opensearch-project#10959) * Fix match_phrase_prefix_query not working on text field with multiple values and index_prefixes Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Add more test Signed-off-by: Gao Binlong <gbinlong@amazon.com> * modify change log Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Fix test failure Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Change the indexAnalyzer used by prefix field Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Skip old version for yaml test Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Optimize some code Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Fix test failure Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Modify yaml test description Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Remove the name parameter for setAnalyzer() Signed-off-by: Gao Binlong <gbinlong@amazon.com> --------- Signed-off-by: Gao Binlong <gbinlong@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Offline calculation of total shard per node and caching it for weight calculation inside LocalShardBalancer (opensearch-project#14675) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * [bug fix] validate lower bound for top n size (opensearch-project#14587) Signed-off-by: Chenyang Ji <cyji@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Create SystemIndexRegistry with helper method matchesSystemIndex (opensearch-project#14415) * Create new extension point in SystemIndexPlugin for a single plugin to get registered system indices Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add to CHANGELOG Signed-off-by: Craig Perkins <cwperx@amazon.com> * WIP on system indices from IndexNameExpressionResolver Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add test in IndexNameExpressionResolverTests Signed-off-by: Craig Perkins <cwperx@amazon.com> * Remove changes in SystemIndexPlugin Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add method in IndexNameExpressionResolver to get matching system indices Signed-off-by: Craig Perkins <cwperx@amazon.com> * Show how resolver can be chained to get system indices Signed-off-by: Craig Perkins <cwperx@amazon.com> * Fix forbiddenApis check Signed-off-by: Craig Perkins <cwperx@amazon.com> * Update CHANGELOG Signed-off-by: Craig Perkins <cwperx@amazon.com> * Make SystemIndices internal Signed-off-by: Craig Perkins <cwperx@amazon.com> * Remove unneeded changes Signed-off-by: Craig Perkins <cwperx@amazon.com> * Fix CI failures Signed-off-by: Craig Perkins <cwperx@amazon.com> * Fix precommit errors Signed-off-by: Craig Perkins <cwperx@amazon.com> * Use Regex instead of WildcardMatcher Signed-off-by: Craig Perkins <cwperx@amazon.com> * Address code review feedback Signed-off-by: Craig Perkins <cwperx@amazon.com> * Allow caller to pass index expressions Signed-off-by: Craig Perkins <cwperx@amazon.com> * Create SystemIndexRegistry Signed-off-by: Craig Perkins <cwperx@amazon.com> * Update CHANGELOG Signed-off-by: Craig Perkins <cwperx@amazon.com> * Remove singleton limitation Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add javadoc Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add @experimentalapi annotation Signed-off-by: Craig Perkins <cwperx@amazon.com> --------- Signed-off-by: Craig Perkins <cwperx@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Refactor Grok validate pattern to iterative approach (opensearch-project#14206) * grok validate patterns recusrion to iterative Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> * Add max depth in resolving a pattern to avoid OOM Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> * change path from deque to arraylist Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> * rename queue to stack Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> * Change max depth to 500 Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> * typo originPatternName fix Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> * spotless Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> --------- Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump opentelemetry from 1.39.0 to 1.40.0 (opensearch-project#14674) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump jackson from 2.17.1 to 2.17.2 (opensearch-project#14687) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add release notes for release 1.3.18 (opensearch-project#14699) Signed-off-by: Zelin Hao <zelinhao@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump reactor from 3.5.19 to 3.5.20 (opensearch-project#14697) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add unit tests for read flow of RemoteClusterStateService and bug fix for transient settings (opensearch-project#14476) Signed-off-by: Shivansh Arora <hishiv@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Update version check for the bug fix of match_phrase_prefix_query not working on text field with multiple values and index_prefixes (opensearch-project#14703) Signed-off-by: Gao Binlong <gbinlong@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Remove unnecessary cast to int from test (opensearch-project#14696) Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * print reason why parent task was cancelled (opensearch-project#14604) Signed-off-by: kkewwei <kkewwei@163.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Use set of shard routing for shard in unassigned shard batch check. (opensearch-project#14533) Signed-off-by: Swetha Guptha <gupthasg@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add versioning for UploadedIndexMetadata (opensearch-project#14677) * Add versioning for UploadedIndexMetadata * Handle componentPrefix for backward compatibility Signed-off-by: Sooraj Sinha <soosinha@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix: update help output for _cat (opensearch-project#14722) * fixed help output for _cat Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io> * updated changelog Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io> * updated changelog Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io> --------- Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix hdfs-fixture kerb-admin & hadoop-minicluster dependencies are not being updated / false positive reports on CVEs (opensearch-project#14729) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Update to Gradle 8.9 (opensearch-project#14574) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix hdfs-fixture hadoop-minicluster dependencies are not being updated / false positive reports on CVEs (opensearch-project#14732) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add `strict_allow_templates` dynamic mapping option (opensearch-project#14555) * The dynamic mapping parameter supports strict_allow_templates Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Modify change log Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Modify skip version in yml test file Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Refactor some code Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Keep the old methods Signed-off-by: Gao Binlong <gbinlong@amazon.com> * change public to private Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Optimize some code Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Do not override toString method for Dynamic Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Optimize some code and modify the changelog Signed-off-by: Gao Binlong <gbinlong@amazon.com> --------- Signed-off-by: Gao Binlong <gbinlong@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump net.minidev:json-smart from 2.5.0 to 2.5.1 in /plugins/repository-azure (opensearch-project#14748) * Bump net.minidev:json-smart in /plugins/repository-azure Bumps [net.minidev:json-smart](https://github.com/netplex/json-smart-v2) from 2.5.0 to 2.5.1. - [Release notes](https://github.com/netplex/json-smart-v2/releases) - [Commits](netplex/json-smart-v2@2.5.0...2.5.1) --- updated-dependencies: - dependency-name: net.minidev:json-smart dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * remove query insights plugin from core (opensearch-project#14743) Signed-off-by: Chenyang Ji <cyji@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add `strict_allow_templates` dynamic mapping option (opensearch-project#14555) (opensearch-project#14737) (opensearch-project#14742) * The dynamic mapping parameter supports strict_allow_templates * Modify change log * Modify skip version in yml test file * Refactor some code * Keep the old methods * change public to private * Optimize some code * Do not override toString method for Dynamic * Optimize some code and modify the changelog --------- (cherry picked from commit 6b8b3ef) Signed-off-by: Gao Binlong <gbinlong@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix create or update alias API doesn't throw exception for unsupported parameters (opensearch-project#14719) * Fix create or update alias API doesn't throw exception for unsupported parameters Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Update version check in yml test Signed-off-by: Gao Binlong <gbinlong@amazon.com> * modify change log Signed-off-by: Gao Binlong <gbinlong@amazon.com> --------- Signed-off-by: Gao Binlong <gbinlong@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Remove query categorization from core (opensearch-project#14759) * Remove query categorization from core Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Add changelog Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Trigger Build Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> --------- Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add changes to propagate queryGroupId across child requests and nodes (opensearch-project#14614) * add query group header propagator Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * apply spotless check Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add new propagator in ThreadContext Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * spotlessApply Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * address comments Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (opensearch-project#14610) * Bump com.microsoft.azure:msal4j in /plugins/repository-azure Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0. - [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases) - [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt) - [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0) --- updated-dependencies: - dependency-name: com.microsoft.azure:msal4j dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * [Bugfix] Fix ICacheKeySerializerTests flakiness (opensearch-project#14564) * Fix testInvalidInput flakiness Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Addressed andrross's comment Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun security check Signed-off-by: Peter Alfonsi <petealft@amazon.com> --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Correct typo in method name (opensearch-project#14621) Signed-off-by: vatsal <vatsal.v.anand@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Refactoring FilterPath.parse by using an iterative approach instead of recursion. (opensearch-project#14200) * Refactor FilterPath parse function (opensearch-project#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Implement unit tests for FilterPathTests (opensearch-project#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Write warn log if Filter is empty; Add comments (opensearch-project#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Add changelog Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove unnecessary log statement Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove unused logger Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Spotless apply Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove incorrect changelog Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> --------- Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(opensearch-project#14612) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (opensearch-project#14513) * Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata * Correct the check for deciding upload of hashes of consistent settings Signed-off-by: Sooraj Sinha <soosinha@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add changelog Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add PR link changelog Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (opensearch-project#14613) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix assertion failure while deleting remote backed index (opensearch-project#14601) Signed-off-by: Sachin Kale <kalsac@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Allow system index warning in OpenSearchRestTestCase.refreshAllIndices (opensearch-project#14635) * Allow system index warning Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add to CHANGELOG Signed-off-by: Craig Perkins <cwperx@amazon.com> * Address code review comments Signed-off-by: Craig Perkins <cwperx@amazon.com> --------- Signed-off-by: Craig Perkins <cwperx@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Star tree codec changes (opensearch-project#14514) --------- Signed-off-by: Bharathwaj G <bharath78910@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump com.github.spullara.mustache.java:compiler from 0.9.13 to 0.9.14 in /modules/lang-mustache (opensearch-project#14672) * Bump com.github.spullara.mustache.java:compiler Bumps [com.github.spullara.mustache.java:compiler](https://github.com/spullara/mustache.java) from 0.9.13 to 0.9.14. - [Commits](spullara/mustache.java@mustache.java-0.9.13...mustache.java-0.9.14) --- updated-dependencies: - dependency-name: com.github.spullara.mustache.java:compiler dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump net.minidev:accessors-smart from 2.5.0 to 2.5.1 in /plugins/repository-azure (opensearch-project#14673) * Bump net.minidev:accessors-smart in /plugins/repository-azure Bumps [net.minidev:accessors-smart](https://github.com/netplex/json-smart-v2) from 2.5.0 to 2.5.1. - [Release notes](https://github.com/netplex/json-smart-v2/releases) - [Commits](netplex/json-smart-v2@2.5.0...2.5.1) --- updated-dependencies: - dependency-name: net.minidev:accessors-smart dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * move query group thread context propagator out of ThreadContext Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> --------- Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Peter Alfonsi <petealft@amazon.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Sooraj Sinha <soosinha@amazon.com> Signed-off-by: Sachin Kale <kalsac@amazon.com> Signed-off-by: Craig Perkins <cwperx@amazon.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com> Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com> Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com> Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com> Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com> Co-authored-by: Sachin Kale <sachinpkale@gmail.com> Co-authored-by: Craig Perkins <cwperx@amazon.com> Co-authored-by: Bharathwaj G <bharath78910@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add consumers to remote store based index settings (opensearch-project#14764) Signed-off-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add matchesPluginSystemIndexPattern to SystemIndexRegistry (opensearch-project#14750) * Add matchesPluginSystemIndexPattern to SystemIndexRegistry Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add to CHANGELOG Signed-off-by: Craig Perkins <cwperx@amazon.com> * Use single data structure to keep track of system indices Signed-off-by: Craig Perkins <cwperx@amazon.com> * Address code review comments Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add test for getAllDescriptors Signed-off-by: Craig Perkins <cwperx@amazon.com> * Update server/src/main/java/org/opensearch/indices/SystemIndexRegistry.java Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Craig Perkins <craig5008@gmail.com> --------- Signed-off-by: Craig Perkins <cwperx@amazon.com> Signed-off-by: Craig Perkins <craig5008@gmail.com> Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * SPI for loading ABC templates (opensearch-project#14659) * SPI for loading ABC templates Signed-off-by: mgodwan <mgodwan@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix bulk upsert ignores the default_pipeline and final_pipeline when the auto-created index matches the index template (opensearch-project#12891) * Fix bulk upsert ignores the default_pipeline and final_pipeline when auto-created index matches with the index template Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Modify changelog & comment Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Use new approach Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Fix test failure Signed-off-by: Gao Binlong <gbinlong@amazon.com> --------- Signed-off-by: Gao Binlong <gbinlong@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix flaky test due to node being used across all tests (opensearch-project#14787) Signed-off-by: Mohit Godwani <mgodwan@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Star Tree Implementation [OnHeap] (opensearch-project#14512) --------- Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add Gao Binlong as maintainer (opensearch-project#14796) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Clear ehcache disk cache files during initialization (opensearch-project#14738) * Clear ehcache disk cache files during initialization Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding UT to fix line coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comment Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding more Uts for better line coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Throwing exception in case we fail to clear cache files during startup Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding more UTs Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding a UT for more coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing gradle build Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Update ehcache disk cache close() logic Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Refactor remote-routing-table service inline with remote state interfaces (opensearch-project#14668) --------- Signed-off-by: Arpit Bandejiya <abandeji@amazon.com> Signed-off-by: Arpit-Bandejiya <abandeji@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Set version to 2.15 for determining metadata during migration to remote store Signed-off-by: Sandeep Kumawat <skumwt@amazon.com> Co-authored-by: Sandeep Kumawat <skumwt@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix bulk upsert ignores the default_pipeline and final_pipeline when the auto-created index matches the index template (opensearch-project#14793) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix create or update alias API doesn't throw exception for unsupported parameters (opensearch-project#14769) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Change RCSS info logs to debug (opensearch-project#14814) Signed-off-by: Shivansh Arora <hishiv@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * [Bugfix] Fix NPE in ReplicaShardAllocator (opensearch-project#13993) (opensearch-project#14385) * [Bugfix] Fix NPE in ReplicaShardAllocator (opensearch-project#13993) Signed-off-by: Daniil Roman <daniilroman.cv@gmail.com> * Add fix info to CHANGELOG.md Signed-off-by: Daniil Roman <danroman17397@gmail.com> --------- Signed-off-by: Daniil Roman <daniilroman.cv@gmail.com> Signed-off-by: Daniil Roman <danroman17397@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Run performance benchmark on pull requests (opensearch-project#14760) * add performance benchmark workflow for pull requests Signed-off-by: Rishabh Singh <sngri@amazon.com> * Update PERFORMANCE_BENCHMARKS.md Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> * Update PERFORMANCE_BENCHMARKS.md Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> * Update .github/workflows/benchmark-pull-request.yml Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> * Update .github/workflows/benchmark-pull-request.yml Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> * Update .github/workflows/benchmark-pull-request.yml Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> * Update .github/workflows/benchmark-pull-request.yml Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> --------- Signed-off-by: Rishabh Singh <sngri@amazon.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * fix constant_keyword field type (opensearch-project#14807) Signed-off-by: kkewwei <kkewwei@163.com> test Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * [Remote Store Migration] Reconcile remote store based index settings during STRICT mode switch (opensearch-project#14792) Signed-off-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add prefix mode verification setting for repository verification (opensearch-project#14790) * Add prefix mode verification setting for repository verification Signed-off-by: Ashish Singh <ssashish@amazon.com> * Add UTs and randomise prefix mode repository verification Signed-off-by: Ashish Singh <ssashish@amazon.com> * Incorporate PR review feedback Signed-off-by: Ashish Singh <ssashish@amazon.com> --------- Signed-off-by: Ashish Singh <ssashish@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add length check on comment body for benchmark workflow (opensearch-project#14834) Signed-off-by: Rishabh Singh <sngri@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add restore-from-snapshot test procedure for snapshot run benchmark config (opensearch-project#14842) Signed-off-by: Rishabh Singh <sngri@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix env variable name typo (opensearch-project#14843) Signed-off-by: Rishabh Singh <sngri@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Use circuit breaker in InternalHistogram when adding empty buckets (opensearch-project#14754) * introduce circuit breaker in InternalHistogram Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * use circuit breaker from reduce context Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * add test Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * revert use_real_memory change in OpenSearchNode Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * add change log Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> --------- Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * [Remote State] Create interface RemoteEntitiesManager (opensearch-project#14671) * Create interface RemoteEntitiesManager Signed-off-by: Shivansh Arora <hishiv@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Optimise TransportNodesAction to not send DiscoveryNodes for NodeStat… (opensearch-project#14749) * Optimize TransportNodesAction to not send DiscoveryNodes for NodeStats, NodesInfo and ClusterStats call Signed-off-by: Pranshu Shukla <pranshushukla06@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Enabling term version check on local state for all ClusterManager Read Transport Actions (opensearch-project#14273) * enabling term version check on local state for all admin read actions Signed-off-by: Rajiv Kumar Vaidyanathan <rajivkv@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Reduce logging in DEBUG for MasterService:run (opensearch-project#14795) * Reduce logging in DEBUG for MasteService:run by introducing short and long summary in Taskbatcher Signed-off-by: Sumit Bansal <sumitsb@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add SplitResponseProcessor to Search Pipelines (opensearch-project#14800) * Add SplitResponseProcessor for search pipelines Signed-off-by: Daniel Widdis <widdis@gmail.com> * Register the split processor factory Signed-off-by: Daniel Widdis <widdis@gmail.com> * Address code review comments Signed-off-by: Daniel Widdis <widdis@gmail.com> * Avoid list copy by casting array Signed-off-by: Daniel Widdis <widdis@gmail.com> --------- Signed-off-by: Daniel Widdis <widdis@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add integration tests for RemoteRoutingTable Service. (opensearch-project#14631) Signed-off-by: Shailendra Singh <singhlhs@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add SortResponseProcessor to Search Pipelines (opensearch-project#14785) * Add SortResponseProcessor for search pipelines Signed-off-by: Daniel Widdis <widdis@gmail.com> * Add stupid and unnecessary javadocs to satisfy overly strict CI Signed-off-by: Daniel Widdis <widdis@gmail.com> * Split casting and sorting methods for readability Signed-off-by: Daniel Widdis <widdis@gmail.com> * Register the sort processor factory Signed-off-by: Daniel Widdis <widdis@gmail.com> * Address code review comments Signed-off-by: Daniel Widdis <widdis@gmail.com> * Cast individual list elements to avoid creating two lists Signed-off-by: Daniel Widdis <widdis@gmail.com> * Add yamlRestTests Signed-off-by: Daniel Widdis <widdis@gmail.com> * Clarify why there's unusual sorting Signed-off-by: Daniel Widdis <widdis@gmail.com> * Use instanceof instead of isAssignableFrom Signed-off-by: Daniel Widdis <widdis@gmail.com> --------- Signed-off-by: Daniel Widdis <widdis@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix allowUnmappedFields, mapUnmappedFieldAsString settings to be applied when parsing query string query (opensearch-project#13957) * Modify to invoke QueryShardContext.fieldMapper() method to apply allowUnmappedFields and mapUnmappedFieldAsString settings Signed-off-by: imyp92 <pyw5420@gmail.com> * Add test cases to verify returning 400 responses if unmapped fields are included for some types of query Signed-off-by: imyp92 <pyw5420@gmail.com> * Add changelog Signed-off-by: imyp92 <pyw5420@gmail.com> --------- Signed-off-by: imyp92 <pyw5420@gmail.com> Signed-off-by: gaobinlong <gbinlong@amazon.com> Co-authored-by: gaobinlong <gbinlong@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump com.microsoft.azure:msal4j from 1.16.0 to 1.16.1 in /plugins/repository-azure (opensearch-project#14857) * Bump com.microsoft.azure:msal4j in /plugins/repository-azure Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.16.0 to 1.16.1. - [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases) - [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt) - [Commits](AzureAD/microsoft-authentication-library-for-java@v1.16.0...v1.16.1) --- updated-dependencies: - dependency-name: com.microsoft.azure:msal4j dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump com.gradle.develocity from 3.17.5 to 3.17.6 (opensearch-project#14856) * Bump com.gradle.develocity from 3.17.5 to 3.17.6 Bumps com.gradle.develocity from 3.17.5 to 3.17.6. --- updated-dependencies: - dependency-name: com.gradle.develocity dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump org.jline:jline in /test/fixtures/hdfs-fixture (opensearch-project#14859) Bumps [org.jline:jline](https://github.com/jline/jline3) from 3.26.2 to 3.26.3. - [Release notes](https://github.com/jline/jline3/releases) - [Changelog](https://github.com/jline/jline3/blob/master/changelog.md) - [Commits](jline/jline3@jline-parent-3.26.2...jline-parent-3.26.3) --- updated-dependencies: - dependency-name: org.jline:jline dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Use Lucene provided Persian stem (opensearch-project#14847) Lucene provided Persian stem apparently isn't hooked yet and this change is doing that based on what is done for Arabic stem support. Signed-off-by: Ebrahim Byagowi <ebrahim@gnu.org> Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump actions/checkout from 2 to 4 (opensearch-project#14858) * Bump actions/checkout from 2 to 4 Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 4. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v2...v4) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Deprecate batch_size parameter on bulk API (opensearch-project#14725) By default the full _bulk payload will be passed to ingest processors as a batch, with any sub batching logic to be implemented by each processor if necessary. Signed-off-by: Liyun Xiu <xiliyun@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add perms for remote snapshot cache eviction on scripted query (opensearch-project#14411) Signed-off-by: Finn Carroll <carrofin@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add transport interceptor to populate queryGroupId in task headers Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add rest, transport layer changes for Hot to warm tiering - dedicated setup (opensearch-project#13980) Signed-off-by: Neetika Singhal <neetiks@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Create listener to refresh search thread resource usage (opensearch-project#14832) * [bug fix] fix incorrect coordinator node search resource usages Signed-off-by: Chenyang Ji <cyji@amazon.com> * fix bug on serialization when passing task resource usage to coordinator Signed-off-by: Chenyang Ji <cyji@amazon.com> * add more unit tests Signed-off-by: Chenyang Ji <cyji@amazon.com> * remove query insights plugin related code Signed-off-by: Chenyang Ji <cyji@amazon.com> * create per request listener to refresh task resource usage Signed-off-by: Chenyang Ji <cyji@amazon.com> * Make new listener API public Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Add changelog Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove wrong files added Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Address review comments Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Build fix Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Make singleton Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Address review comments Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Make sure listener runs before plugin listeners Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Spotless Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Minor fix Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> --------- Signed-off-by: Chenyang Ji <cyji@amazon.com> Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Signed-off-by: Jay Deng <jayd0104@gmail.com> Co-authored-by: Chenyang Ji <cyji@amazon.com> Co-authored-by: Jay Deng <jayd0104@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Caching avg total bytes and avg free bytes inside ClusterInfo (opensearch-project#14851) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Use default value when index.number_of_replicas is null (opensearch-project#14812) * Use default value when index.number_of_replicas is null Signed-off-by: Liyun Xiu <xiliyun@amazon.com> * Add integration test Signed-off-by: Liyun Xiu <xiliyun@amazon.com> * Add changelog Signed-off-by: Liyun Xiu <xiliyun@amazon.com> --------- Signed-off-by: Liyun Xiu <xiliyun@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * [Remote Routing Table] Implement write and read flow for shard diff file. (opensearch-project#14684) * Implement write and read flow to upload/download shard diff file. Signed-off-by: Shailendra Singh <singhlhs@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Optimized ClusterStatsIndices to precomute shard stats (opensearch-project#14426) * Optimize Cluster Stats Indices to precomute node level stats Signed-off-by: Pranshu Shukla <pranshushukla06@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix constraint bug which allows more primary shards than average primary shards per index (opensearch-project#14908) Signed-off-by: Gaurav Bafna <gbbafna@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Optmising AwarenessAllocationDecider for hashmap.get call (opensearch-project#14761) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * update comment Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix IngestServiceTests.testBulkRequestExecutionWithFailures (opensearch-project#14918) The test would previously fail if the randomness led to only a single indexing request being included in the bulk payload. This change guarantees multiple indexing requests in order to ensure the batch logic kicks in. Also replace some unneeded mocks with real classes. Signed-off-by: Andrew Ross <andrross@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add queryGroupTask Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * remove unnecessary imports Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add QueryGroupTask tests Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * rename WLM transport request handler Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add CHANGELOG entry Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * fix ut Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * address comments Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * fix UT to remove the verify for final method Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * apply spotless Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> --------- Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> Signed-off-by: Shivansh Arora <hishiv@amazon.com> Signed-off-by: Gao Binlong <gbinlong@amazon.com> Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Chenyang Ji <cyji@amazon.com> Signed-off-by: Craig Perkins <cwperx@amazon.com> Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Zelin Hao <zelinhao@amazon.com> Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io> Signed-off-by: kkewwei <kkewwei@163.com> Signed-off-by: Swetha Guptha <gupthasg@amazon.com> Signed-off-by: Sooraj Sinha <soosinha@amazon.com> Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Signed-off-by: Peter Alfonsi <petealft@amazon.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> Signed-off-by: Sachin Kale <kalsac@amazon.com> Signed-off-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com> Signed-off-by: Craig Perkins <craig5008@gmail.com> Signed-off-by: mgodwan <mgodwan@amazon.com> Signed-off-by: Mohit Godwani <mgodwan@amazon.com> Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sandeep Kumawat <skumwt@amazon.com> Signed-off-by: Daniil Roman <daniilroman.cv@gmail.com> Signed-off-by: Daniil Roman <danroman17397@gmail.com> Signed-off-by: Rishabh Singh <sngri@amazon.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> Signed-off-by: Ashish Singh <ssashish@amazon.com> Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> Signed-off-by: Pranshu Shukla <pranshushukla06@gmail.com> Signed-off-by: Rajiv Kumar Vaidyanathan <rajivkv@amazon.com> Signed-off-by: Sumit Bansal <sumitsb@amazon.com> Signed-off-by: Daniel Widdis <widdis@gmail.com> Signed-off-by: Shailendra Singh <singhlhs@amazon.com> Signed-off-by: imyp92 <pyw5420@gmail.com> Signed-off-by: gaobinlong <gbinlong@amazon.com> Signed-off-by: Ebrahim Byagowi <ebrahim@gnu.org> Signed-off-by: Liyun Xiu <xiliyun@amazon.com> Signed-off-by: Finn Carroll <carrofin@amazon.com> Signed-off-by: Neetika Singhal <neetiks@amazon.com> Signed-off-by: Jay Deng <jayd0104@gmail.com> Signed-off-by: Gaurav Bafna <gbbafna@amazon.com> Signed-off-by: Andrew Ross <andrross@amazon.com> Co-authored-by: Shivansh Arora <hishiv@amazon.com> Co-authored-by: Arpit-Bandejiya <abandeji@amazon.com> Co-authored-by: gaobinlong <gbinlong@amazon.com> Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com> Co-authored-by: Chenyang Ji <cyji@amazon.com> Co-authored-by: Craig Perkins <cwperx@amazon.com> Co-authored-by: Sandesh Kumar <sandeshkr419@gmail.com> Co-authored-by: Andriy Redko <andriy.redko@aiven.io> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Lukáš Vlček <lukas.vlcek@aiven.io> Co-authored-by: kkewwei <kkewwei@163.com> Co-authored-by: SwethaGuptha <156877431+SwethaGuptha@users.noreply.github.com> Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com> Co-authored-by: Ahmed Sobeh <ahmed.sobeh@aiven.io> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com> Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com> Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com> Co-authored-by: Sachin Kale <sachinpkale@gmail.com> Co-authored-by: Bharathwaj G <bharath78910@gmail.com> Co-authored-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com> Co-authored-by: Craig Perkins <craig5008@gmail.com> Co-authored-by: Andriy Redko <drreta@gmail.com> Co-authored-by: Mohit Godwani <81609427+mgodwan@users.noreply.github.com> Co-authored-by: Sarthak Aggarwal <sarthagg@amazon.com> Co-authored-by: Sagar <99425694+sgup432@users.noreply.github.com> Co-authored-by: Sandeep Kumawat <2025sandeepkumawat@gmail.com> Co-authored-by: Sandeep Kumawat <skumwt@amazon.com> Co-authored-by: Daniil Roman <danroman17397@gmail.com> Co-authored-by: Rishabh Singh <rishabhksingh@gmail.com> Co-authored-by: kkewwei <kewei.11@bytedance.com> Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> Co-authored-by: Ashish Singh <ssashish@amazon.com> Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com> Co-authored-by: Pranshu Shukla <55992439+Pranshu-S@users.noreply.github.com> Co-authored-by: rajiv-kv <157019998+rajiv-kv@users.noreply.github.com> Co-authored-by: Sumit Bansal <sumit.asr@gmail.com> Co-authored-by: Daniel Widdis <widdis@gmail.com> Co-authored-by: shailendra0811 <167273922+shailendra0811@users.noreply.github.com> Co-authored-by: Park, Yeongwu <pyw5420@gmail.com> Co-authored-by: ebraminio <ebraminio@gmail.com> Co-authored-by: Liyun Xiu <xiliyun@amazon.com> Co-authored-by: Finn <carrofin@amazon.com> Co-authored-by: Neetika Singhal <neetiks@amazon.com> Co-authored-by: Jay Deng <jayd0104@gmail.com> Co-authored-by: Gaurav Bafna <85113518+gbbafna@users.noreply.github.com> Co-authored-by: Andrew Ross <andrross@amazon.com>
wdongyu
pushed a commit
to wdongyu/OpenSearch
that referenced
this pull request
Aug 22, 2024
…ect#14738) * Clear ehcache disk cache files during initialization Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding UT to fix line coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comment Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding more Uts for better line coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Throwing exception in case we fail to clear cache files during startup Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding more UTs Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding a UT for more coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing gradle build Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Update ehcache disk cache close() logic Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
wdongyu
pushed a commit
to wdongyu/OpenSearch
that referenced
this pull request
Aug 22, 2024
…de level (opensearch-project#14708) * add logic to add headers to Task Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add logic to add queryGroupId to task headers Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * remove redundant code Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add changelog entry Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * address comments Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * fix precommit Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add UTs for RemoteIndexMetadataManager (opensearch-project#14660) Signed-off-by: Shivansh Arora <hishiv@amazon.com> Co-authored-by: Arpit-Bandejiya <abandeji@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix match_phrase_prefix_query not working on text field with multiple values and index_prefixes (opensearch-project#10959) * Fix match_phrase_prefix_query not working on text field with multiple values and index_prefixes Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Add more test Signed-off-by: Gao Binlong <gbinlong@amazon.com> * modify change log Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Fix test failure Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Change the indexAnalyzer used by prefix field Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Skip old version for yaml test Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Optimize some code Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Fix test failure Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Modify yaml test description Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Remove the name parameter for setAnalyzer() Signed-off-by: Gao Binlong <gbinlong@amazon.com> --------- Signed-off-by: Gao Binlong <gbinlong@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Offline calculation of total shard per node and caching it for weight calculation inside LocalShardBalancer (opensearch-project#14675) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * [bug fix] validate lower bound for top n size (opensearch-project#14587) Signed-off-by: Chenyang Ji <cyji@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Create SystemIndexRegistry with helper method matchesSystemIndex (opensearch-project#14415) * Create new extension point in SystemIndexPlugin for a single plugin to get registered system indices Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add to CHANGELOG Signed-off-by: Craig Perkins <cwperx@amazon.com> * WIP on system indices from IndexNameExpressionResolver Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add test in IndexNameExpressionResolverTests Signed-off-by: Craig Perkins <cwperx@amazon.com> * Remove changes in SystemIndexPlugin Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add method in IndexNameExpressionResolver to get matching system indices Signed-off-by: Craig Perkins <cwperx@amazon.com> * Show how resolver can be chained to get system indices Signed-off-by: Craig Perkins <cwperx@amazon.com> * Fix forbiddenApis check Signed-off-by: Craig Perkins <cwperx@amazon.com> * Update CHANGELOG Signed-off-by: Craig Perkins <cwperx@amazon.com> * Make SystemIndices internal Signed-off-by: Craig Perkins <cwperx@amazon.com> * Remove unneeded changes Signed-off-by: Craig Perkins <cwperx@amazon.com> * Fix CI failures Signed-off-by: Craig Perkins <cwperx@amazon.com> * Fix precommit errors Signed-off-by: Craig Perkins <cwperx@amazon.com> * Use Regex instead of WildcardMatcher Signed-off-by: Craig Perkins <cwperx@amazon.com> * Address code review feedback Signed-off-by: Craig Perkins <cwperx@amazon.com> * Allow caller to pass index expressions Signed-off-by: Craig Perkins <cwperx@amazon.com> * Create SystemIndexRegistry Signed-off-by: Craig Perkins <cwperx@amazon.com> * Update CHANGELOG Signed-off-by: Craig Perkins <cwperx@amazon.com> * Remove singleton limitation Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add javadoc Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add @experimentalapi annotation Signed-off-by: Craig Perkins <cwperx@amazon.com> --------- Signed-off-by: Craig Perkins <cwperx@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Refactor Grok validate pattern to iterative approach (opensearch-project#14206) * grok validate patterns recusrion to iterative Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> * Add max depth in resolving a pattern to avoid OOM Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> * change path from deque to arraylist Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> * rename queue to stack Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> * Change max depth to 500 Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> * typo originPatternName fix Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> * spotless Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> --------- Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump opentelemetry from 1.39.0 to 1.40.0 (opensearch-project#14674) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump jackson from 2.17.1 to 2.17.2 (opensearch-project#14687) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add release notes for release 1.3.18 (opensearch-project#14699) Signed-off-by: Zelin Hao <zelinhao@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump reactor from 3.5.19 to 3.5.20 (opensearch-project#14697) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add unit tests for read flow of RemoteClusterStateService and bug fix for transient settings (opensearch-project#14476) Signed-off-by: Shivansh Arora <hishiv@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Update version check for the bug fix of match_phrase_prefix_query not working on text field with multiple values and index_prefixes (opensearch-project#14703) Signed-off-by: Gao Binlong <gbinlong@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Remove unnecessary cast to int from test (opensearch-project#14696) Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * print reason why parent task was cancelled (opensearch-project#14604) Signed-off-by: kkewwei <kkewwei@163.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Use set of shard routing for shard in unassigned shard batch check. (opensearch-project#14533) Signed-off-by: Swetha Guptha <gupthasg@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add versioning for UploadedIndexMetadata (opensearch-project#14677) * Add versioning for UploadedIndexMetadata * Handle componentPrefix for backward compatibility Signed-off-by: Sooraj Sinha <soosinha@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix: update help output for _cat (opensearch-project#14722) * fixed help output for _cat Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io> * updated changelog Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io> * updated changelog Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io> --------- Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix hdfs-fixture kerb-admin & hadoop-minicluster dependencies are not being updated / false positive reports on CVEs (opensearch-project#14729) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Update to Gradle 8.9 (opensearch-project#14574) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix hdfs-fixture hadoop-minicluster dependencies are not being updated / false positive reports on CVEs (opensearch-project#14732) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add `strict_allow_templates` dynamic mapping option (opensearch-project#14555) * The dynamic mapping parameter supports strict_allow_templates Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Modify change log Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Modify skip version in yml test file Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Refactor some code Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Keep the old methods Signed-off-by: Gao Binlong <gbinlong@amazon.com> * change public to private Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Optimize some code Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Do not override toString method for Dynamic Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Optimize some code and modify the changelog Signed-off-by: Gao Binlong <gbinlong@amazon.com> --------- Signed-off-by: Gao Binlong <gbinlong@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump net.minidev:json-smart from 2.5.0 to 2.5.1 in /plugins/repository-azure (opensearch-project#14748) * Bump net.minidev:json-smart in /plugins/repository-azure Bumps [net.minidev:json-smart](https://github.com/netplex/json-smart-v2) from 2.5.0 to 2.5.1. - [Release notes](https://github.com/netplex/json-smart-v2/releases) - [Commits](netplex/json-smart-v2@2.5.0...2.5.1) --- updated-dependencies: - dependency-name: net.minidev:json-smart dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * remove query insights plugin from core (opensearch-project#14743) Signed-off-by: Chenyang Ji <cyji@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add `strict_allow_templates` dynamic mapping option (opensearch-project#14555) (opensearch-project#14737) (opensearch-project#14742) * The dynamic mapping parameter supports strict_allow_templates * Modify change log * Modify skip version in yml test file * Refactor some code * Keep the old methods * change public to private * Optimize some code * Do not override toString method for Dynamic * Optimize some code and modify the changelog --------- (cherry picked from commit 6b8b3ef) Signed-off-by: Gao Binlong <gbinlong@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix create or update alias API doesn't throw exception for unsupported parameters (opensearch-project#14719) * Fix create or update alias API doesn't throw exception for unsupported parameters Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Update version check in yml test Signed-off-by: Gao Binlong <gbinlong@amazon.com> * modify change log Signed-off-by: Gao Binlong <gbinlong@amazon.com> --------- Signed-off-by: Gao Binlong <gbinlong@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Remove query categorization from core (opensearch-project#14759) * Remove query categorization from core Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Add changelog Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Trigger Build Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> --------- Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add changes to propagate queryGroupId across child requests and nodes (opensearch-project#14614) * add query group header propagator Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * apply spotless check Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add new propagator in ThreadContext Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * spotlessApply Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * address comments Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (opensearch-project#14610) * Bump com.microsoft.azure:msal4j in /plugins/repository-azure Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0. - [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases) - [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt) - [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0) --- updated-dependencies: - dependency-name: com.microsoft.azure:msal4j dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * [Bugfix] Fix ICacheKeySerializerTests flakiness (opensearch-project#14564) * Fix testInvalidInput flakiness Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Addressed andrross's comment Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun security check Signed-off-by: Peter Alfonsi <petealft@amazon.com> --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Correct typo in method name (opensearch-project#14621) Signed-off-by: vatsal <vatsal.v.anand@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Refactoring FilterPath.parse by using an iterative approach instead of recursion. (opensearch-project#14200) * Refactor FilterPath parse function (opensearch-project#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Implement unit tests for FilterPathTests (opensearch-project#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Write warn log if Filter is empty; Add comments (opensearch-project#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Add changelog Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove unnecessary log statement Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove unused logger Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Spotless apply Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove incorrect changelog Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> --------- Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(opensearch-project#14612) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (opensearch-project#14513) * Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata * Correct the check for deciding upload of hashes of consistent settings Signed-off-by: Sooraj Sinha <soosinha@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add changelog Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add PR link changelog Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (opensearch-project#14613) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix assertion failure while deleting remote backed index (opensearch-project#14601) Signed-off-by: Sachin Kale <kalsac@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Allow system index warning in OpenSearchRestTestCase.refreshAllIndices (opensearch-project#14635) * Allow system index warning Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add to CHANGELOG Signed-off-by: Craig Perkins <cwperx@amazon.com> * Address code review comments Signed-off-by: Craig Perkins <cwperx@amazon.com> --------- Signed-off-by: Craig Perkins <cwperx@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Star tree codec changes (opensearch-project#14514) --------- Signed-off-by: Bharathwaj G <bharath78910@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump com.github.spullara.mustache.java:compiler from 0.9.13 to 0.9.14 in /modules/lang-mustache (opensearch-project#14672) * Bump com.github.spullara.mustache.java:compiler Bumps [com.github.spullara.mustache.java:compiler](https://github.com/spullara/mustache.java) from 0.9.13 to 0.9.14. - [Commits](spullara/mustache.java@mustache.java-0.9.13...mustache.java-0.9.14) --- updated-dependencies: - dependency-name: com.github.spullara.mustache.java:compiler dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump net.minidev:accessors-smart from 2.5.0 to 2.5.1 in /plugins/repository-azure (opensearch-project#14673) * Bump net.minidev:accessors-smart in /plugins/repository-azure Bumps [net.minidev:accessors-smart](https://github.com/netplex/json-smart-v2) from 2.5.0 to 2.5.1. - [Release notes](https://github.com/netplex/json-smart-v2/releases) - [Commits](netplex/json-smart-v2@2.5.0...2.5.1) --- updated-dependencies: - dependency-name: net.minidev:accessors-smart dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * move query group thread context propagator out of ThreadContext Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> --------- Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Peter Alfonsi <petealft@amazon.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Sooraj Sinha <soosinha@amazon.com> Signed-off-by: Sachin Kale <kalsac@amazon.com> Signed-off-by: Craig Perkins <cwperx@amazon.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com> Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com> Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com> Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com> Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com> Co-authored-by: Sachin Kale <sachinpkale@gmail.com> Co-authored-by: Craig Perkins <cwperx@amazon.com> Co-authored-by: Bharathwaj G <bharath78910@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add consumers to remote store based index settings (opensearch-project#14764) Signed-off-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add matchesPluginSystemIndexPattern to SystemIndexRegistry (opensearch-project#14750) * Add matchesPluginSystemIndexPattern to SystemIndexRegistry Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add to CHANGELOG Signed-off-by: Craig Perkins <cwperx@amazon.com> * Use single data structure to keep track of system indices Signed-off-by: Craig Perkins <cwperx@amazon.com> * Address code review comments Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add test for getAllDescriptors Signed-off-by: Craig Perkins <cwperx@amazon.com> * Update server/src/main/java/org/opensearch/indices/SystemIndexRegistry.java Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Craig Perkins <craig5008@gmail.com> --------- Signed-off-by: Craig Perkins <cwperx@amazon.com> Signed-off-by: Craig Perkins <craig5008@gmail.com> Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * SPI for loading ABC templates (opensearch-project#14659) * SPI for loading ABC templates Signed-off-by: mgodwan <mgodwan@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix bulk upsert ignores the default_pipeline and final_pipeline when the auto-created index matches the index template (opensearch-project#12891) * Fix bulk upsert ignores the default_pipeline and final_pipeline when auto-created index matches with the index template Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Modify changelog & comment Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Use new approach Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Fix test failure Signed-off-by: Gao Binlong <gbinlong@amazon.com> --------- Signed-off-by: Gao Binlong <gbinlong@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix flaky test due to node being used across all tests (opensearch-project#14787) Signed-off-by: Mohit Godwani <mgodwan@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Star Tree Implementation [OnHeap] (opensearch-project#14512) --------- Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add Gao Binlong as maintainer (opensearch-project#14796) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Clear ehcache disk cache files during initialization (opensearch-project#14738) * Clear ehcache disk cache files during initialization Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding UT to fix line coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comment Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding more Uts for better line coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Throwing exception in case we fail to clear cache files during startup Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding more UTs Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding a UT for more coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing gradle build Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Update ehcache disk cache close() logic Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Refactor remote-routing-table service inline with remote state interfaces (opensearch-project#14668) --------- Signed-off-by: Arpit Bandejiya <abandeji@amazon.com> Signed-off-by: Arpit-Bandejiya <abandeji@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Set version to 2.15 for determining metadata during migration to remote store Signed-off-by: Sandeep Kumawat <skumwt@amazon.com> Co-authored-by: Sandeep Kumawat <skumwt@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix bulk upsert ignores the default_pipeline and final_pipeline when the auto-created index matches the index template (opensearch-project#14793) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix create or update alias API doesn't throw exception for unsupported parameters (opensearch-project#14769) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Change RCSS info logs to debug (opensearch-project#14814) Signed-off-by: Shivansh Arora <hishiv@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * [Bugfix] Fix NPE in ReplicaShardAllocator (opensearch-project#13993) (opensearch-project#14385) * [Bugfix] Fix NPE in ReplicaShardAllocator (opensearch-project#13993) Signed-off-by: Daniil Roman <daniilroman.cv@gmail.com> * Add fix info to CHANGELOG.md Signed-off-by: Daniil Roman <danroman17397@gmail.com> --------- Signed-off-by: Daniil Roman <daniilroman.cv@gmail.com> Signed-off-by: Daniil Roman <danroman17397@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Run performance benchmark on pull requests (opensearch-project#14760) * add performance benchmark workflow for pull requests Signed-off-by: Rishabh Singh <sngri@amazon.com> * Update PERFORMANCE_BENCHMARKS.md Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> * Update PERFORMANCE_BENCHMARKS.md Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> * Update .github/workflows/benchmark-pull-request.yml Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> * Update .github/workflows/benchmark-pull-request.yml Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> * Update .github/workflows/benchmark-pull-request.yml Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> * Update .github/workflows/benchmark-pull-request.yml Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> --------- Signed-off-by: Rishabh Singh <sngri@amazon.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * fix constant_keyword field type (opensearch-project#14807) Signed-off-by: kkewwei <kkewwei@163.com> test Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * [Remote Store Migration] Reconcile remote store based index settings during STRICT mode switch (opensearch-project#14792) Signed-off-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add prefix mode verification setting for repository verification (opensearch-project#14790) * Add prefix mode verification setting for repository verification Signed-off-by: Ashish Singh <ssashish@amazon.com> * Add UTs and randomise prefix mode repository verification Signed-off-by: Ashish Singh <ssashish@amazon.com> * Incorporate PR review feedback Signed-off-by: Ashish Singh <ssashish@amazon.com> --------- Signed-off-by: Ashish Singh <ssashish@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add length check on comment body for benchmark workflow (opensearch-project#14834) Signed-off-by: Rishabh Singh <sngri@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add restore-from-snapshot test procedure for snapshot run benchmark config (opensearch-project#14842) Signed-off-by: Rishabh Singh <sngri@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix env variable name typo (opensearch-project#14843) Signed-off-by: Rishabh Singh <sngri@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Use circuit breaker in InternalHistogram when adding empty buckets (opensearch-project#14754) * introduce circuit breaker in InternalHistogram Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * use circuit breaker from reduce context Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * add test Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * revert use_real_memory change in OpenSearchNode Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * add change log Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> --------- Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * [Remote State] Create interface RemoteEntitiesManager (opensearch-project#14671) * Create interface RemoteEntitiesManager Signed-off-by: Shivansh Arora <hishiv@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Optimise TransportNodesAction to not send DiscoveryNodes for NodeStat… (opensearch-project#14749) * Optimize TransportNodesAction to not send DiscoveryNodes for NodeStats, NodesInfo and ClusterStats call Signed-off-by: Pranshu Shukla <pranshushukla06@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Enabling term version check on local state for all ClusterManager Read Transport Actions (opensearch-project#14273) * enabling term version check on local state for all admin read actions Signed-off-by: Rajiv Kumar Vaidyanathan <rajivkv@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Reduce logging in DEBUG for MasterService:run (opensearch-project#14795) * Reduce logging in DEBUG for MasteService:run by introducing short and long summary in Taskbatcher Signed-off-by: Sumit Bansal <sumitsb@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add SplitResponseProcessor to Search Pipelines (opensearch-project#14800) * Add SplitResponseProcessor for search pipelines Signed-off-by: Daniel Widdis <widdis@gmail.com> * Register the split processor factory Signed-off-by: Daniel Widdis <widdis@gmail.com> * Address code review comments Signed-off-by: Daniel Widdis <widdis@gmail.com> * Avoid list copy by casting array Signed-off-by: Daniel Widdis <widdis@gmail.com> --------- Signed-off-by: Daniel Widdis <widdis@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add integration tests for RemoteRoutingTable Service. (opensearch-project#14631) Signed-off-by: Shailendra Singh <singhlhs@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add SortResponseProcessor to Search Pipelines (opensearch-project#14785) * Add SortResponseProcessor for search pipelines Signed-off-by: Daniel Widdis <widdis@gmail.com> * Add stupid and unnecessary javadocs to satisfy overly strict CI Signed-off-by: Daniel Widdis <widdis@gmail.com> * Split casting and sorting methods for readability Signed-off-by: Daniel Widdis <widdis@gmail.com> * Register the sort processor factory Signed-off-by: Daniel Widdis <widdis@gmail.com> * Address code review comments Signed-off-by: Daniel Widdis <widdis@gmail.com> * Cast individual list elements to avoid creating two lists Signed-off-by: Daniel Widdis <widdis@gmail.com> * Add yamlRestTests Signed-off-by: Daniel Widdis <widdis@gmail.com> * Clarify why there's unusual sorting Signed-off-by: Daniel Widdis <widdis@gmail.com> * Use instanceof instead of isAssignableFrom Signed-off-by: Daniel Widdis <widdis@gmail.com> --------- Signed-off-by: Daniel Widdis <widdis@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix allowUnmappedFields, mapUnmappedFieldAsString settings to be applied when parsing query string query (opensearch-project#13957) * Modify to invoke QueryShardContext.fieldMapper() method to apply allowUnmappedFields and mapUnmappedFieldAsString settings Signed-off-by: imyp92 <pyw5420@gmail.com> * Add test cases to verify returning 400 responses if unmapped fields are included for some types of query Signed-off-by: imyp92 <pyw5420@gmail.com> * Add changelog Signed-off-by: imyp92 <pyw5420@gmail.com> --------- Signed-off-by: imyp92 <pyw5420@gmail.com> Signed-off-by: gaobinlong <gbinlong@amazon.com> Co-authored-by: gaobinlong <gbinlong@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump com.microsoft.azure:msal4j from 1.16.0 to 1.16.1 in /plugins/repository-azure (opensearch-project#14857) * Bump com.microsoft.azure:msal4j in /plugins/repository-azure Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.16.0 to 1.16.1. - [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases) - [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt) - [Commits](AzureAD/microsoft-authentication-library-for-java@v1.16.0...v1.16.1) --- updated-dependencies: - dependency-name: com.microsoft.azure:msal4j dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump com.gradle.develocity from 3.17.5 to 3.17.6 (opensearch-project#14856) * Bump com.gradle.develocity from 3.17.5 to 3.17.6 Bumps com.gradle.develocity from 3.17.5 to 3.17.6. --- updated-dependencies: - dependency-name: com.gradle.develocity dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump org.jline:jline in /test/fixtures/hdfs-fixture (opensearch-project#14859) Bumps [org.jline:jline](https://github.com/jline/jline3) from 3.26.2 to 3.26.3. - [Release notes](https://github.com/jline/jline3/releases) - [Changelog](https://github.com/jline/jline3/blob/master/changelog.md) - [Commits](jline/jline3@jline-parent-3.26.2...jline-parent-3.26.3) --- updated-dependencies: - dependency-name: org.jline:jline dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Use Lucene provided Persian stem (opensearch-project#14847) Lucene provided Persian stem apparently isn't hooked yet and this change is doing that based on what is done for Arabic stem support. Signed-off-by: Ebrahim Byagowi <ebrahim@gnu.org> Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump actions/checkout from 2 to 4 (opensearch-project#14858) * Bump actions/checkout from 2 to 4 Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 4. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v2...v4) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Deprecate batch_size parameter on bulk API (opensearch-project#14725) By default the full _bulk payload will be passed to ingest processors as a batch, with any sub batching logic to be implemented by each processor if necessary. Signed-off-by: Liyun Xiu <xiliyun@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add perms for remote snapshot cache eviction on scripted query (opensearch-project#14411) Signed-off-by: Finn Carroll <carrofin@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add transport interceptor to populate queryGroupId in task headers Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Add rest, transport layer changes for Hot to warm tiering - dedicated setup (opensearch-project#13980) Signed-off-by: Neetika Singhal <neetiks@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Create listener to refresh search thread resource usage (opensearch-project#14832) * [bug fix] fix incorrect coordinator node search resource usages Signed-off-by: Chenyang Ji <cyji@amazon.com> * fix bug on serialization when passing task resource usage to coordinator Signed-off-by: Chenyang Ji <cyji@amazon.com> * add more unit tests Signed-off-by: Chenyang Ji <cyji@amazon.com> * remove query insights plugin related code Signed-off-by: Chenyang Ji <cyji@amazon.com> * create per request listener to refresh task resource usage Signed-off-by: Chenyang Ji <cyji@amazon.com> * Make new listener API public Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Add changelog Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove wrong files added Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Address review comments Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Build fix Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Make singleton Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Address review comments Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Make sure listener runs before plugin listeners Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Spotless Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Minor fix Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> --------- Signed-off-by: Chenyang Ji <cyji@amazon.com> Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Signed-off-by: Jay Deng <jayd0104@gmail.com> Co-authored-by: Chenyang Ji <cyji@amazon.com> Co-authored-by: Jay Deng <jayd0104@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Caching avg total bytes and avg free bytes inside ClusterInfo (opensearch-project#14851) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Use default value when index.number_of_replicas is null (opensearch-project#14812) * Use default value when index.number_of_replicas is null Signed-off-by: Liyun Xiu <xiliyun@amazon.com> * Add integration test Signed-off-by: Liyun Xiu <xiliyun@amazon.com> * Add changelog Signed-off-by: Liyun Xiu <xiliyun@amazon.com> --------- Signed-off-by: Liyun Xiu <xiliyun@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * [Remote Routing Table] Implement write and read flow for shard diff file. (opensearch-project#14684) * Implement write and read flow to upload/download shard diff file. Signed-off-by: Shailendra Singh <singhlhs@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Optimized ClusterStatsIndices to precomute shard stats (opensearch-project#14426) * Optimize Cluster Stats Indices to precomute node level stats Signed-off-by: Pranshu Shukla <pranshushukla06@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix constraint bug which allows more primary shards than average primary shards per index (opensearch-project#14908) Signed-off-by: Gaurav Bafna <gbbafna@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Optmising AwarenessAllocationDecider for hashmap.get call (opensearch-project#14761) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * update comment Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix IngestServiceTests.testBulkRequestExecutionWithFailures (opensearch-project#14918) The test would previously fail if the randomness led to only a single indexing request being included in the bulk payload. This change guarantees multiple indexing requests in order to ensure the batch logic kicks in. Also replace some unneeded mocks with real classes. Signed-off-by: Andrew Ross <andrross@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add queryGroupTask Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * remove unnecessary imports Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add QueryGroupTask tests Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * rename WLM transport request handler Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add CHANGELOG entry Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * fix ut Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * address comments Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * fix UT to remove the verify for final method Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * apply spotless Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> --------- Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> Signed-off-by: Shivansh Arora <hishiv@amazon.com> Signed-off-by: Gao Binlong <gbinlong@amazon.com> Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Chenyang Ji <cyji@amazon.com> Signed-off-by: Craig Perkins <cwperx@amazon.com> Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Zelin Hao <zelinhao@amazon.com> Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io> Signed-off-by: kkewwei <kkewwei@163.com> Signed-off-by: Swetha Guptha <gupthasg@amazon.com> Signed-off-by: Sooraj Sinha <soosinha@amazon.com> Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Signed-off-by: Peter Alfonsi <petealft@amazon.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> Signed-off-by: Sachin Kale <kalsac@amazon.com> Signed-off-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com> Signed-off-by: Craig Perkins <craig5008@gmail.com> Signed-off-by: mgodwan <mgodwan@amazon.com> Signed-off-by: Mohit Godwani <mgodwan@amazon.com> Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sandeep Kumawat <skumwt@amazon.com> Signed-off-by: Daniil Roman <daniilroman.cv@gmail.com> Signed-off-by: Daniil Roman <danroman17397@gmail.com> Signed-off-by: Rishabh Singh <sngri@amazon.com> Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com> Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> Signed-off-by: Ashish Singh <ssashish@amazon.com> Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> Signed-off-by: Pranshu Shukla <pranshushukla06@gmail.com> Signed-off-by: Rajiv Kumar Vaidyanathan <rajivkv@amazon.com> Signed-off-by: Sumit Bansal <sumitsb@amazon.com> Signed-off-by: Daniel Widdis <widdis@gmail.com> Signed-off-by: Shailendra Singh <singhlhs@amazon.com> Signed-off-by: imyp92 <pyw5420@gmail.com> Signed-off-by: gaobinlong <gbinlong@amazon.com> Signed-off-by: Ebrahim Byagowi <ebrahim@gnu.org> Signed-off-by: Liyun Xiu <xiliyun@amazon.com> Signed-off-by: Finn Carroll <carrofin@amazon.com> Signed-off-by: Neetika Singhal <neetiks@amazon.com> Signed-off-by: Jay Deng <jayd0104@gmail.com> Signed-off-by: Gaurav Bafna <gbbafna@amazon.com> Signed-off-by: Andrew Ross <andrross@amazon.com> Co-authored-by: Shivansh Arora <hishiv@amazon.com> Co-authored-by: Arpit-Bandejiya <abandeji@amazon.com> Co-authored-by: gaobinlong <gbinlong@amazon.com> Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com> Co-authored-by: Chenyang Ji <cyji@amazon.com> Co-authored-by: Craig Perkins <cwperx@amazon.com> Co-authored-by: Sandesh Kumar <sandeshkr419@gmail.com> Co-authored-by: Andriy Redko <andriy.redko@aiven.io> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Lukáš Vlček <lukas.vlcek@aiven.io> Co-authored-by: kkewwei <kkewwei@163.com> Co-authored-by: SwethaGuptha <156877431+SwethaGuptha@users.noreply.github.com> Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com> Co-authored-by: Ahmed Sobeh <ahmed.sobeh@aiven.io> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com> Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com> Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com> Co-authored-by: Sachin Kale <sachinpkale@gmail.com> Co-authored-by: Bharathwaj G <bharath78910@gmail.com> Co-authored-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com> Co-authored-by: Craig Perkins <craig5008@gmail.com> Co-authored-by: Andriy Redko <drreta@gmail.com> Co-authored-by: Mohit Godwani <81609427+mgodwan@users.noreply.github.com> Co-authored-by: Sarthak Aggarwal <sarthagg@amazon.com> Co-authored-by: Sagar <99425694+sgup432@users.noreply.github.com> Co-authored-by: Sandeep Kumawat <2025sandeepkumawat@gmail.com> Co-authored-by: Sandeep Kumawat <skumwt@amazon.com> Co-authored-by: Daniil Roman <danroman17397@gmail.com> Co-authored-by: Rishabh Singh <rishabhksingh@gmail.com> Co-authored-by: kkewwei <kewei.11@bytedance.com> Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> Co-authored-by: Ashish Singh <ssashish@amazon.com> Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com> Co-authored-by: Pranshu Shukla <55992439+Pranshu-S@users.noreply.github.com> Co-authored-by: rajiv-kv <157019998+rajiv-kv@users.noreply.github.com> Co-authored-by: Sumit Bansal <sumit.asr@gmail.com> Co-authored-by: Daniel Widdis <widdis@gmail.com> Co-authored-by: shailendra0811 <167273922+shailendra0811@users.noreply.github.com> Co-authored-by: Park, Yeongwu <pyw5420@gmail.com> Co-authored-by: ebraminio <ebraminio@gmail.com> Co-authored-by: Liyun Xiu <xiliyun@amazon.com> Co-authored-by: Finn <carrofin@amazon.com> Co-authored-by: Neetika Singhal <neetiks@amazon.com> Co-authored-by: Jay Deng <jayd0104@gmail.com> Co-authored-by: Gaurav Bafna <85113518+gbbafna@users.noreply.github.com> Co-authored-by: Andrew Ross <andrross@amazon.com>
peteralfonsi
pushed a commit
to peteralfonsi/OpenSearch
that referenced
this pull request
Sep 3, 2024
…ect#14738) * Clear ehcache disk cache files during initialization Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding UT to fix line coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comment Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding more Uts for better line coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Throwing exception in case we fail to clear cache files during startup Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding more UTs Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding a UT for more coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing gradle build Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Update ehcache disk cache close() logic Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
peteralfonsi
pushed a commit
to peteralfonsi/OpenSearch
that referenced
this pull request
Sep 4, 2024
commit 93a3132 Author: Peter Alfonsi <petealft@amazon.com> Date: Wed Sep 4 11:44:03 2024 -0700 fix broken ITs Signed-off-by: Peter Alfonsi <petealft@amazon.com> commit fd92c5a Author: Marc Handalian <handalm@amazon.com> Date: Thu Jan 11 06:41:55 2024 -0800 Update runTask to optionally install plugins (opensearch-project#11844) Signed-off-by: Marc Handalian <marc.handalian@gmail.com> commit 4d78661 Author: Peter Alfonsi <petealft@amazon.com> Date: Tue Sep 3 16:02:31 2024 -0700 Misc test fixes Signed-off-by: Peter Alfonsi <petealft@amazon.com> commit 70a6935 Author: Peter Alfonsi <peter.alfonsi@gmail.com> Date: Fri Aug 23 10:42:05 2024 -0700 [Bugfix] Fixes IRC NPE bug for timed-out cacheable queries (opensearch-project#15327) * Fix IRC timeout bug Signed-off-by: Peter Alfonsi <petealft@amazon.com> * addressed Sagar's comments Signed-off-by: Peter Alfonsi <petealft@amazon.com> * addressed Ankit's comments Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Add UT for test coverage Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun gradle Signed-off-by: Peter Alfonsi <petealft@amazon.com> * tweak imports in new UT Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun gradle Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun gradle Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun gradle Signed-off-by: Peter Alfonsi <petealft@amazon.com> --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> commit f9943f8 Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Wed Jul 17 19:02:05 2024 -0700 Clear ehcache disk cache files during initialization (opensearch-project#14738) * Clear ehcache disk cache files during initialization Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding UT to fix line coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comment Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding more Uts for better line coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Throwing exception in case we fail to clear cache files during startup Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding more UTs Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding a UT for more coverage Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing gradle build Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Update ehcache disk cache close() logic Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> commit fc2007e Author: Peter Alfonsi <peter.alfonsi@gmail.com> Date: Mon Jul 1 15:54:39 2024 -0700 [Bugfix] Fix ICacheKeySerializerTests flakiness (opensearch-project#14564) * Fix testInvalidInput flakiness Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Addressed andrross's comment Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun security check Signed-off-by: Peter Alfonsi <petealft@amazon.com> --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> commit 1ef62e2 Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Wed Jun 26 11:14:42 2024 -0700 Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently (opensearch-project#14550) * Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comment Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com> Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com> commit 29ec9d6 Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Tue Jun 25 12:04:17 2024 -0700 [Tiered Caching] Moving query recomputation logic outside of write lock (opensearch-project#14187) * Moving query recompute out of write lock Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * [Tiered Caching] Moving query recomputation logic outside of write lock Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding java doc for the completable map Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Changes to call future handler only once per key Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing spotless check Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Added changelog Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comments Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing gradle fail Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comments to refactor unit test Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * minor UT refactor Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com> commit 8347779 Author: Peter Alfonsi <peter.alfonsi@gmail.com> Date: Tue Jun 25 13:26:54 2024 -0700 Fix flaky DefaultCacheStatsHolderTests (opensearch-project#14462) Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> commit 40f868d Author: Kiran Prakash <awskiran@amazon.com> Date: Thu Jun 20 14:17:23 2024 -0700 Update IndicesRequestCacheCleanupIT.java (opensearch-project#14478) Signed-off-by: Kiran Prakash <awskiran@amazon.com> commit 094a9f2 Author: Kiran Prakash <awskiran@amazon.com> Date: Wed Jun 19 15:36:59 2024 -0700 Fix Indices Request Cache ITs from flaking and timing out (opensearch-project#14369) Signed-off-by: Kiran Prakash <awskiran@amazon.com> commit 3a110a8 Author: Kiran Prakash <awskiran@amazon.com> Date: Tue Jun 18 10:18:25 2024 -0700 [Tiered Cache] Use ConcurrentHashMap explicitly in IndicesRequestCache (opensearch-project#14409) Signed-off-by: Kiran Prakash <awskiran@amazon.com> commit bcf0f59 Author: Peter Alfonsi <peter.alfonsi@gmail.com> Date: Fri Jun 14 13:20:51 2024 -0700 [Bugfix] Fix TieredSpilloverCache flaky tests (opensearch-project#14333) * Fix flaky TSC stats tests Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Addressed andrross's comment Signed-off-by: Peter Alfonsi <petealft@amazon.com> * fix forbidden API Signed-off-by: Peter Alfonsi <petealft@amazon.com> --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> commit a222a03 Author: Kiran Prakash <awskiran@amazon.com> Date: Wed Jun 12 18:30:06 2024 -0700 [Tiered Caching] [Bug Fix] Use concurrentMap instead of HashMap to fix Concurrent Modification Exception (opensearch-project#14221) * use concurrentmap Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update CHANGELOG.md Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCache.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * revert feature flags Signed-off-by: Kiran Prakash <awskiran@amazon.com> * changelog to releaselog Signed-off-by: Kiran Prakash <awskiran@amazon.com> * use concurrentmap Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update CHANGELOG.md Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCache.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * revert feature flags Signed-off-by: Kiran Prakash <awskiran@amazon.com> * changelog to releaselog Signed-off-by: Kiran Prakash <awskiran@amazon.com> * revert the test removal Signed-off-by: Kiran Prakash <awskiran@amazon.com> * revert the conflict resolutions Signed-off-by: Kiran Prakash <awskiran@amazon.com> --------- Signed-off-by: Kiran Prakash <awskiran@amazon.com> commit f85cd00 Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Wed Jun 12 08:22:38 2024 -0700 Fix ShardNotFoundException during request cache clean up (opensearch-project#14219) * Fix for ShardNotFoundException during request cache clean up Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Added changelog Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fix forbidden gradle check Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> commit 406ae78 Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Thu Jun 6 15:02:00 2024 -0700 [Caching] Move cache removal notifications outside lru lock (opensearch-project#14017) --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> commit c5fd7fe Author: Peter Alfonsi <petealft@amazon.com> Date: Tue Sep 3 14:38:26 2024 -0700 Change IRC UT to correctly create shard Signed-off-by: Peter Alfonsi <petealft@amazon.com> commit 9c4d4af Author: Peter Alfonsi <peter.alfonsi@gmail.com> Date: Tue Jun 4 16:09:18 2024 -0700 [Tiered Caching] Additional ITs for cache stats (opensearch-project#13655) * Adds cache clear IT Signed-off-by: Peter Alfonsi <petealft@amazon.com> Cleaned up logic for cache stats ITs Signed-off-by: Peter Alfonsi <petealft@amazon.com> Adds more tests around tiered spillover cache Signed-off-by: Peter Alfonsi <petealft@amazon.com> Fixed cache stats behavior for overall /_nodes/stats call Signed-off-by: Peter Alfonsi <petealft@amazon.com> cleanup Signed-off-by: Peter Alfonsi <petealft@amazon.com> Fixed folder structure Signed-off-by: Peter Alfonsi <petealft@amazon.com> Addressed Sagar's comments Signed-off-by: Peter Alfonsi <petealft@amazon.com> Addressed Ankit's comments Signed-off-by: Peter Alfonsi <petealft@amazon.com> Break horrifyingly long test case into many shorter cases Signed-off-by: Peter Alfonsi <petealft@amazon.com> Added unsupported operation exception to TSC stats holder incrementEvictions() Signed-off-by: Peter Alfonsi <petealft@amazon.com> Addressed Sorabh's comments Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun assemble Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun gradle Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun gradle Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun gradle Signed-off-by: Peter Alfonsi <petealft@amazon.com> --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> commit be97e90 Author: Peter Alfonsi <peter.alfonsi@gmail.com> Date: Thu May 23 12:02:22 2024 -0700 [Bugfix] [Tiered Caching] Fixes issues when integrating tiered cache with disk cache (opensearch-project#13784) --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> commit 60ee715 Author: Kiran Prakash <awskiran@amazon.com> Date: Mon May 20 12:52:03 2024 -0700 Update IndicesRequestCacheIT.java (opensearch-project#13678) Signed-off-by: Kiran Prakash <awskiran@amazon.com> commit 51f6c1b Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Wed May 8 09:15:36 2024 -0700 Fix IndicesRequestCache clean up logic (opensearch-project#13597) Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com> Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com> commit 49e2701 Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Mon May 6 17:25:45 2024 -0700 Fix negative requestStats memory_size issue (opensearch-project#13553) This solves the bug where RequestStats memory_size metric was going negative in certain scenarios as reported in the issue. It turns out that the issue occurs when an indexShard is deleted and then reallocated on the same node. So whenever stale entries from older shard are deleted, those are accounted for the new shard which has the same shardId. --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> commit a5c5675 Author: Peter Alfonsi <peter.alfonsi@gmail.com> Date: Thu May 2 17:29:07 2024 -0700 [Tiered Caching] Adds stats implementation for TieredSpilloverCache (opensearch-project#13236) Stats rework part 4 of 4 --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> commit 46944cb Author: peteralfonsi <peter.alfonsi@gmail.com> Date: Tue Apr 30 10:59:03 2024 -0700 Fix flaky test CacheStatsAPIIndicesRequestCacheIT.testNullLevels() (opensearch-project#13457) * Fix flaky test Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Initialize CommonStatsFlags with empty array for levels Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Fixes tests using incorrect null levels Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun Signed-off-by: Peter Alfonsi <petealft@amazon.com> --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> commit 71fcad4 Author: Kiran Prakash <awskiran@amazon.com> Date: Mon Apr 29 18:41:33 2024 -0700 Fix Flaky test IndicesRequestCacheIT.testStaleKeysCleanupWithMultipleIndices (opensearch-project#13453) * Update IndicesRequestCacheIT.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheIT.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> --------- Signed-off-by: Kiran Prakash <awskiran@amazon.com> commit f60339f Author: peteralfonsi <peter.alfonsi@gmail.com> Date: Mon Apr 29 22:37:05 2024 -0700 [Tiered Caching] Bump versions for serialization in new cache stats API (opensearch-project#13460) --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> commit 88718a2 Author: peteralfonsi <peter.alfonsi@gmail.com> Date: Mon Apr 29 14:47:52 2024 -0700 [Tiered Caching] Expose new cache stats API (opensearch-project#13237) Step 3 out of 4 --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> commit 5e97082 Author: peteralfonsi <peter.alfonsi@gmail.com> Date: Sun Apr 28 08:15:18 2024 -0700 [Tiered Caching] Gate CacheStatsHolder logic behind FeatureFlags.PLUGGABLE_CACHE setting (opensearch-project#13238) Stats rework step 2 of 4 --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> commit ed881f7 Author: Kiran Prakash <awskiran@amazon.com> Date: Fri Apr 26 11:52:31 2024 -0700 [Tiered Caching] Make Indices Request Cache Stale Key Mgmt Threshold setting dynamic (opensearch-project#12941) * Update IndicesRequestCache.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update ClusterSettings.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheIT.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * spotless Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheIT.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * some refactoring Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCache.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * address existing tests Signed-off-by: Kiran Prakash <awskiran@amazon.com> * UTs Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update CHANGELOG.md Signed-off-by: Kiran Prakash <awskiran@amazon.com> * ITs Signed-off-by: Kiran Prakash <awskiran@amazon.com> * spotless Signed-off-by: Kiran Prakash <awskiran@amazon.com> * refactor Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheIT.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheIT.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheIT.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheIT.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update CHANGELOG.md Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update CHANGELOG.md Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheIT.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCache.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheIT.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * resolve conflicts Signed-off-by: Kiran Prakash <awskiran@amazon.com> * address code comments Signed-off-by: Kiran Prakash <awskiran@amazon.com> * address code comments Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheIT.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * rename tests Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheIT.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheIT.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * resolve conflicts Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCache.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * code comments Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheIT.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> --------- Signed-off-by: Kiran Prakash <awskiran@amazon.com> commit 424ccc9 Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Thu Apr 25 22:09:42 2024 -0700 [Tiered Caching] Expose a dynamic setting to disable/enable disk cache (opensearch-project#13373) * [Tiered Caching] Expose a dynamic setting to disable/enable disk cache Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Putting tiered cache settings behind feature flag Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding a changelog Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing Sorabh's comments Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Putting new setting behind feature flag Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> commit acbbb2f Author: Kiran Prakash <awskiran@amazon.com> Date: Wed Apr 24 14:50:39 2024 -0700 [Tiered Caching] Bug fix for IndicesRequestCache StaleKey management (opensearch-project#13070) * Update IndicesRequestCache.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update CHANGELOG.md Signed-off-by: Kiran Prakash <awskiran@amazon.com> * revert Signed-off-by: Kiran Prakash <awskiran@amazon.com> * revert Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCache.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCache.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCache.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCache.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCache.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCache.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * code comments only Signed-off-by: Kiran Prakash <awskiran@amazon.com> * docs changes Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update CHANGELOG.md Signed-off-by: Kiran Prakash <awskiran@amazon.com> * revert catching AlreadyClosedException Signed-off-by: Kiran Prakash <awskiran@amazon.com> * assert Signed-off-by: Kiran Prakash <awskiran@amazon.com> * conflicts Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCacheTests.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCache.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * address comments Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCache.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update IndicesRequestCache.java Signed-off-by: Kiran Prakash <awskiran@amazon.com> * address conflicts Signed-off-by: Kiran Prakash <awskiran@amazon.com> * spotless apply Signed-off-by: Kiran Prakash <awskiran@amazon.com> * address comments Signed-off-by: Kiran Prakash <awskiran@amazon.com> * update code comments Signed-off-by: Kiran Prakash <awskiran@amazon.com> * address bug & add tests Signed-off-by: Kiran Prakash <awskiran@amazon.com> --------- Signed-off-by: Kiran Prakash <awskiran@amazon.com> commit f1d2e72 Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Mon Apr 15 10:43:22 2024 -0800 [Tiered Caching] Ehcache Disk cache IT (opensearch-project#12904) * Ehcache IT tests Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding some logs to print key/value size Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Add ehcache related invalidation IT Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Remvoing unnecessary IndicesRequestCache IT Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Indentation fix Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Added tests around expiration time and invalidation Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> commit b0de406 Author: peteralfonsi <peter.alfonsi@gmail.com> Date: Fri Apr 12 16:09:06 2024 -0700 [Tiered Caching] Stats rework (1/3): Interfaces and implementations for individual tiers (opensearch-project#12531) As part of tiered caching stats, changes the common ICache interface to use ICacheKey as its key. This key contains dimensions (for example, shard ID, index name, or tier) that can be used to aggregate stats. Also changes the CacheStats interface to store the necessary cache stats, and to support getting stats either as a total or aggregated by these dimensions. Integrates these changes with OpenSearchOnHeapCache and EhcacheDiskCache. The stats implementation for the TieredSpilloverCache will be in a followup PR. --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> commit 40a8d34 Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Thu Apr 11 14:49:03 2024 -0800 [Tiered Caching] Make took time policy dynamic and add additional integ tests (opensearch-project#13063) --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> commit 6f88777 Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Tue Mar 19 19:41:59 2024 -0700 Fixing ehcache flaky test (opensearch-project#12764) * Fixing ehcache flaky test Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding a ehcache issue reference for thread leak issue Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Updating comment Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> commit f69c089 Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Mon Mar 18 22:20:52 2024 -0700 [Tiered Caching] Clear up disk cache(ehcache) files during node shutdown (opensearch-project#12734) * Adding logic to clear up the disk cache files during close() * Adding logic to update entries count after invalidateAll() * Removing unneeded system log statement * Added comment in test for readability * Fixing issue where we were sending compacted byte[] array to ehcache but calculating size with padded byte[] --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> commit bf6488d Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Mon Mar 18 22:17:14 2024 -0700 [Tiered Caching] Fix test testComputeIfAbsentWithFactoryBasedCacheCreation (opensearch-project#12700) --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> commit 0fc1f40 Author: peteralfonsi <peter.alfonsi@gmail.com> Date: Mon Mar 18 13:39:35 2024 -0700 [Tiered Caching] Serializers for ehcache (opensearch-project#12709) Adds serializers and integrates them into ehcache disk cache --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> commit a45be0f Author: Kiran Prakash <awskiran@amazon.com> Date: Sat Mar 16 17:12:27 2024 -0700 [Tiered Caching] Indices Request cache stalekey management (opensearch-project#12625) * Introduce IndicesRequestCacheCleanupManager Signed-off-by: Kiran Prakash <awskiran@amazon.com> * using cleanup mgr to enqueue cleanups Signed-off-by: Kiran Prakash <awskiran@amazon.com> * readability improvements Signed-off-by: Kiran Prakash <awskiran@amazon.com> * update updateCleanupKeyToCountMap on new cache entry Signed-off-by: Kiran Prakash <awskiran@amazon.com> * create IndicesRequestCacheCleanupManager & settings and validators Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Add IRC specific cache cleaner and remove from IndicesService Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Unit Tests Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update CHANGELOG.md Signed-off-by: Kiran Prakash <awskiran@amazon.com> * move cachecleaner inside mgr Signed-off-by: Kiran Prakash <awskiran@amazon.com> * remove processCleanupKeys Signed-off-by: Kiran Prakash <awskiran@amazon.com> * minor cleanups Signed-off-by: Kiran Prakash <awskiran@amazon.com> * add updateCleanupKeyToCountMapOnCacheEviction Signed-off-by: Kiran Prakash <awskiran@amazon.com> * remove locks and make all methods synchronized Signed-off-by: Kiran Prakash <awskiran@amazon.com> * spotless Signed-off-by: Kiran Prakash <awskiran@amazon.com> * updateCleanupKeyToCountMapOnCacheEviction Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Testing Signed-off-by: Kiran Prakash <awskiran@amazon.com> * add Reschedule back to indices service Signed-off-by: Kiran Prakash <awskiran@amazon.com> * rename updateStaleKeysCount to incrementStaleKeysCount Signed-off-by: Kiran Prakash <awskiran@amazon.com> * rename getStaleKeysCountForTesting to getStaleKeysCount Signed-off-by: Kiran Prakash <awskiran@amazon.com> * rename threshold to stalenessThreshold Signed-off-by: Kiran Prakash <awskiran@amazon.com> * check for cleanupKey.entity == null Signed-off-by: Kiran Prakash <awskiran@amazon.com> * use computeIfPresent with keycountmap Signed-off-by: Kiran Prakash <awskiran@amazon.com> * log both staleKeysInCache & Staleness in debug logs Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Use HashMap instead of ConcurrentMap Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Address b/w compatibility Signed-off-by: Kiran Prakash <awskiran@amazon.com> * remove synchronized for updateCleanupKeyToCountMapOnCacheEviction Signed-off-by: Kiran Prakash <awskiran@amazon.com> * make cleanCache synchronized Signed-off-by: Kiran Prakash <awskiran@amazon.com> * remove shouldRemoveKey Signed-off-by: Kiran Prakash <awskiran@amazon.com> * spotlessApply Signed-off-by: Kiran Prakash <awskiran@amazon.com> --------- Signed-off-by: Kiran Prakash <awskiran@amazon.com> commit bad412c Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Fri Mar 15 16:53:54 2024 -0700 [Tiered caching] Supporting removal function on EhcacheDiskCache iterator (opensearch-project#12653) * [Tiered caching] Supporting removal function on EhcacheDiskCache iterator Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Minor refactoring in unit test Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> commit 15c4afa Author: peteralfonsi <peter.alfonsi@gmail.com> Date: Fri Mar 15 15:36:22 2024 -0700 [Tiered Caching] Cache tier policies (opensearch-project#12542) * Adds policy interface and took time policy impl Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Changes IndicesService to write a CachePolicyInfoWrapper before the QSR Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Moved took time logic from QSR to IndicesService Signed-off-by: Peter Alfonsi <petealft@amazon.com> * spotlessApply Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Addressed ansjcy's comments Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Partial rebase on most recent changes Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Integrated policies with new TSC changes Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Reverted unintended change to idea/vcs.xml Signed-off-by: Peter Alfonsi <petealft@amazon.com> * javadocs Signed-off-by: Peter Alfonsi <petealft@amazon.com> * github actions Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Set default threshold value to 10 ms Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Addressed Sorabh's comments Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Addressed Sorabh's second round of comments Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Set cachedQueryParser in IRC Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Addressed Sorabh's comments besides dynamic setting Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Removed dynamic setting, misc comments Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Added changelog entry Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Added missing javadoc Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Fixed failed gradle run Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Added setting validation test Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun gradle for flaky IT Signed-off-by: Peter Alfonsi <petealft@amazon.com> * javadocs Signed-off-by: Peter Alfonsi <petealft@amazon.com> --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> commit 45fce15 Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Thu Mar 14 13:00:24 2024 -0700 [Tiered Caching] Fixing flaky tiered cache test (opensearch-project#12650) * Fixing flaky tiered cache test Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Removing unnecessary comment Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Removing unused variable Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> commit bb66699 Author: Peter Alfonsi <petealft@amazon.com> Date: Fri Aug 30 11:36:05 2024 -0700 Attempt to fix broken ITs Signed-off-by: Peter Alfonsi <petealft@amazon.com> commit 58170bf Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Mon Mar 11 14:12:55 2024 -0700 [Tiered caching] Integrating IndicesRequestCache with CacheService controlled by a feature flag (opensearch-project#12533) * Adding changelog * Fixing gradle build issue * Fixing CacheService test * Adding UT in IndicesRequestCache with feature flag for more coverage * Updating changelog and renaming feature flag setting * Moving feature flag setting handling logic to CacheService by maintaining backward compatibility * Fixing broken UTs --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> commit 80d2130 Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Fri Mar 1 11:10:28 2024 -0800 [Tiered caching] Introducing cache plugins and exposing Ehcache as one of the pluggable disk cache option (opensearch-project#11874) Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> commit f372c2c Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Wed Jan 10 17:06:29 2024 -0800 [Tiered Caching] Enable serialization of IndicesRequestCache.Key (opensearch-project#10275) --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> Co-authored-by: Kiran Prakash <awskiran@amazon.com> commit b880c0b Author: Sagar <99425694+sgup432@users.noreply.github.com> Date: Tue Jan 9 10:57:52 2024 -0800 [Tiered caching] Framework changes (opensearch-project#10753) * [Tiered caching] Framework changes Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Added javadoc for new files/packages Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Added changelog Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing javadoc warnings Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comments Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing additional minor comments Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Moving non null check to builder for OS onHeapCache Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding package-info for new packages Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Removing service and adding different cache interfaces along with event listener support Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing gradle missingDoc issue Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Changing listener logic, removing tiered cache integration with IRC Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding opensearch.internal tag for LoadAwareCacheLoader Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing thread safety issue Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Remove compute function and event listener logic change for TieredCache Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Making Cache.compute function private Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding javadoc and more test for cache.put Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding write locks to refresh API as well Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Removing unwanted EventType class and refactoring one UT Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Removing TieredCache interface Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> Signed-off-by: Peter Alfonsi <petealft@amazon.com>
This was referenced Sep 11, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This change is around clearing disk cache files(if present) during cache initialization or when the node/process starts/restarts. As of now, we do not persist the disk cache data between process restarts which is why this is needed. We do clear files during gracefule shutdown when close() is called, this change is to handle the scenarios where there are non-graceful restarts in which case we might end up not clearing older/stale data.
Related Issues
Resolves #[Issue number to be closed when this PR is merged]
Check List
[ ] API changes companion pull request created, if applicable.[ ] Public documentation issue/PR created, if applicable.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.