Merge branch 'main' into 11654-segment-matchall

CHANGELOG.md

@@ -15,6 +15,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 - GHA to verify checklist items completion in PR descriptions ([#10800](https://github.com/opensearch-project/OpenSearch/pull/10800))
 - Allow to pass the list settings through environment variables (like [], ["a", "b", "c"], ...) ([#10625](https://github.com/opensearch-project/OpenSearch/pull/10625))
 - [Admission Control] Integrate CPU AC with ResourceUsageCollector and add CPU AC stats to nodes/stats ([#10887](https://github.com/opensearch-project/OpenSearch/pull/10887))
+- [S3 Repository] Add setting to control connection count for sync client ([#12028](https://github.com/opensearch-project/OpenSearch/pull/12028))
 
 ### Dependencies
 - Bump `log4j-core` from 2.18.0 to 2.19.0
@@ -133,6 +134,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 - Bump OpenTelemetry from 1.32.0 to 1.34.1 ([#11891](https://github.com/opensearch-project/OpenSearch/pull/11891))
 - Support index level allocation filtering for searchable snapshot index ([#11563](https://github.com/opensearch-project/OpenSearch/pull/11563))
 - Add `org.opensearch.rest.MethodHandlers` and `RestController#getAllHandlers` ([11876](https://github.com/opensearch-project/OpenSearch/pull/11876))
+- New DateTime format for RFC3339 compatible date fields ([#11465](https://github.com/opensearch-project/OpenSearch/pull/11465))
 
 ### Dependencies
 - Bumps jetty version to 9.4.52.v20230823 to fix GMS-2023-1857 ([#9822](https://github.com/opensearch-project/OpenSearch/pull/9822))
@@ -151,12 +153,12 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 - Bump `stefanzweifel/git-auto-commit-action` from 4 to 5 ([#11171](https://github.com/opensearch-project/OpenSearch/pull/11171))
 - Bump `actions/github-script` from 6 to 7 ([#11271](https://github.com/opensearch-project/OpenSearch/pull/11271))
 - Bump `jackson` and `jackson_databind` from 2.15.2 to 2.16.0 ([#11273](https://github.com/opensearch-project/OpenSearch/pull/11273))
-- Bump `netty` from 4.1.100.Final to 4.1.104.Final ([#11294](https://github.com/opensearch-project/OpenSearch/pull/11294), [#11775](https://github.com/opensearch-project/OpenSearch/pull/11775))
+- Bump `netty` from 4.1.100.Final to 4.1.106.Final ([#11294](https://github.com/opensearch-project/OpenSearch/pull/11294), [#11775](https://github.com/opensearch-project/OpenSearch/pull/11775)), [#12034](https://github.com/opensearch-project/OpenSearch/pull/12034))
 - Bump `com.avast.gradle:gradle-docker-compose-plugin` from 0.16.12 to 0.17.6 ([#10163](https://github.com/opensearch-project/OpenSearch/pull/10163), [#11692](https://github.com/opensearch-project/OpenSearch/pull/11692))
 - Bump `com.squareup.okhttp3:okhttp` from 4.11.0 to 4.12.0 ([#10861](https://github.com/opensearch-project/OpenSearch/pull/10861))
 - Bump `org.apache.commons:commons-text` from 1.10.0 to 1.11.0 ([#11344](https://github.com/opensearch-project/OpenSearch/pull/11344))
-- Bump `reactor-netty-core` from 1.1.12 to 1.1.13 ([#11350](https://github.com/opensearch-project/OpenSearch/pull/11350))
-- Bump `com.gradle.enterprise` from 3.14.1 to 3.16.1 ([#11339](https://github.com/opensearch-project/OpenSearch/pull/11339), [#11629](https://github.com/opensearch-project/OpenSearch/pull/11629))
+- Bump `reactor-netty-core` from 1.1.12 to 1.1.15 ([#11350](https://github.com/opensearch-project/OpenSearch/pull/11350)), ([#12042](https://github.com/opensearch-project/OpenSearch/pull/12042))
+- Bump `com.gradle.enterprise` from 3.14.1 to 3.16.2 ([#11339](https://github.com/opensearch-project/OpenSearch/pull/11339), [#11629](https://github.com/opensearch-project/OpenSearch/pull/11629), [#12056](https://github.com/opensearch-project/OpenSearch/pull/12056))
 - Bump `actions/setup-java` from 3 to 4 ([#11447](https://github.com/opensearch-project/OpenSearch/pull/11447))
 - Bump `commons-net:commons-net` from 3.9.0 to 3.10.0 ([#11450](https://github.com/opensearch-project/OpenSearch/pull/11450))
 - Bump `org.apache.maven:maven-model` from 3.9.4 to 3.9.6 ([#11445](https://github.com/opensearch-project/OpenSearch/pull/11445))
@@ -173,12 +175,14 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 - Bump `com.maxmind.db:maxmind-db` from 3.0.0 to 3.1.0 ([#11693](https://github.com/opensearch-project/OpenSearch/pull/11693))
 - Bump `net.java.dev.jna:jna` from 5.13.0 to 5.14.0 ([#11798](https://github.com/opensearch-project/OpenSearch/pull/11798))
 - Bump `lycheeverse/lychee-action` from 1.8.0 to 1.9.1 ([#11795](https://github.com/opensearch-project/OpenSearch/pull/11795), [#11887](https://github.com/opensearch-project/OpenSearch/pull/11887))
-- Bump `Lucene` from 9.8.0 to 9.9.1 ([#11421](https://github.com/opensearch-project/OpenSearch/pull/11421))
+- Bump `Lucene` from 9.8.0 to 9.9.2 ([#11421](https://github.com/opensearch-project/OpenSearch/pull/11421)), ([#12063](https://github.com/opensearch-project/OpenSearch/pull/12063))
 - Bump `com.networknt:json-schema-validator` from 1.0.86 to 1.2.0 ([#11886](https://github.com/opensearch-project/OpenSearch/pull/11886), [#11963](https://github.com/opensearch-project/OpenSearch/pull/11963))
 - Bump `com.google.api:gax-httpjson` from 0.103.1 to 2.39.0 ([#11794](https://github.com/opensearch-project/OpenSearch/pull/11794))
 - Bump `com.google.oauth-client:google-oauth-client` from 1.34.1 to 1.35.0 ([#11960](https://github.com/opensearch-project/OpenSearch/pull/11960))
-- Bump `com.diffplug.spotless` from 6.23.2 to 6.24.0 ([#11962](https://github.com/opensearch-project/OpenSearch/pull/11962))
+- Bump `com.diffplug.spotless` from 6.23.2 to 6.25.0 ([#11962](https://github.com/opensearch-project/OpenSearch/pull/11962), [#12055](https://github.com/opensearch-project/OpenSearch/pull/12055))
 - Bump `com.google.cloud:google-cloud-core` from 2.5.10 to 2.30.0 ([#11961](https://github.com/opensearch-project/OpenSearch/pull/11961))
+- Bump `reactor-core` from 3.5.11 to 3.5.14 ([#12042](https://github.com/opensearch-project/OpenSearch/pull/12042))
+- Bump `com.google.http-client:google-http-client-jackson2` from 1.43.3 to 1.44.1 ([#12059](https://github.com/opensearch-project/OpenSearch/pull/12059))
 
 ### Changed
 - Mute the query profile IT with concurrent execution ([#9840](https://github.com/opensearch-project/OpenSearch/pull/9840))
@@ -209,7 +213,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 - Use slice_size == shard_size heuristic in terms aggs for concurrent segment search and properly calculate the doc_count_error ([#11732](https://github.com/opensearch-project/OpenSearch/pull/11732))
 - Added Support for dynamically adding SearchRequestOperationsListeners with SearchRequestOperationsCompositeListenerFactory ([#11526](https://github.com/opensearch-project/OpenSearch/pull/11526))
 - Ensure Jackson default maximums introduced in 2.16.0 do not conflict with OpenSearch settings ([#11890](https://github.com/opensearch-project/OpenSearch/pull/11890))
-- Extract cluster management for integration tests into JUnit test rule out of OpenSearchIntegTestCase ([#11877](https://github.com/opensearch-project/OpenSearch/pull/11877))
+- Extract cluster management for integration tests into JUnit test rule out of OpenSearchIntegTestCase ([#11877](https://github.com/opensearch-project/OpenSearch/pull/11877)), ([#12000](https://github.com/opensearch-project/OpenSearch/pull/12000))
 - Workaround for https://bugs.openjdk.org/browse/JDK-8323659 regression, introduced in JDK-21.0.2 ([#11968](https://github.com/opensearch-project/OpenSearch/pull/11968))
 
 ### Deprecated
@@ -241,6 +245,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 - Fix tracing context propagation for local transport instrumentation ([#11490](https://github.com/opensearch-project/OpenSearch/pull/11490))
 - Fix parsing of single line comments in `lang-painless` ([#11815](https://github.com/opensearch-project/OpenSearch/issues/11815))
 - Fix memory leak issue in ReorganizingLongHash ([#11953](https://github.com/opensearch-project/OpenSearch/issues/11953))
+- Prevent setting remote_snapshot store type on index creation ([#11867](https://github.com/opensearch-project/OpenSearch/pull/11867))
+- [BUG] Fix remote shards balancer when filtering throttled nodes ([#11724](https://github.com/opensearch-project/OpenSearch/pull/11724))
 
 ### Security
 

NOTICE.txt

@@ -10,3 +10,6 @@ Foundation (http://www.apache.org/).
 
 This product includes software developed by
 Joda.org (http://www.joda.org/).
+
+This product includes software developed by
+Morten Haraldsen (ethlo) (https://github.com/ethlo) under the Apache License, version 2.0.

build.gradle

@@ -54,7 +54,7 @@ plugins {
   id 'lifecycle-base'
   id 'opensearch.docker-support'
   id 'opensearch.global-build-info'
-  id "com.diffplug.spotless" version "6.24.0" apply false
+  id "com.diffplug.spotless" version "6.25.0" apply false
   id "org.gradle.test-retry" version "1.5.4" apply false
   id "test-report-aggregation"
   id 'jacoco-report-aggregation'

buildSrc/version.properties

@@ -1,5 +1,5 @@
 opensearch        = 3.0.0
-lucene            = 9.9.1
+lucene            = 9.9.2
 
 bundled_jdk_vendor = adoptium
 bundled_jdk = 21.0.2+13
@@ -26,12 +26,12 @@ jakarta_annotation = 1.3.5
 # when updating the JNA version, also update the version in buildSrc/build.gradle
 jna               = 5.13.0
 
-netty             = 4.1.104.Final
+netty             = 4.1.106.Final
 joda              = 2.12.2
 
 # project reactor
-reactor_netty     = 1.1.13
-reactor           = 3.5.11
+reactor_netty     = 1.1.15
+reactor           = 3.5.14
 
 # client dependencies
 httpclient5       = 5.2.1
@@ -55,7 +55,7 @@ bouncycastle=1.76
 randomizedrunner  = 2.7.1
 junit             = 4.13.2
 hamcrest          = 2.1
-mockito           = 5.5.0
+mockito           = 5.10.0
 objenesis         = 3.2
 bytebuddy         = 1.14.7
 

libs/core/licenses/lucene-core-9.9.2.jar.sha1

@@ -0,0 +1 @@
+7699f80220fc80b08413902560904623b88beb9f

libs/core/src/main/java/org/opensearch/Version.java

@@ -98,8 +98,8 @@ public class Version implements Comparable<Version>, ToXContentFragment {
     public static final Version V_2_11_0 = new Version(2110099, org.apache.lucene.util.Version.LUCENE_9_7_0);
     public static final Version V_2_11_1 = new Version(2110199, org.apache.lucene.util.Version.LUCENE_9_7_0);
     public static final Version V_2_11_2 = new Version(2110299, org.apache.lucene.util.Version.LUCENE_9_7_0);
-    public static final Version V_2_12_0 = new Version(2120099, org.apache.lucene.util.Version.LUCENE_9_9_1);
-    public static final Version V_3_0_0 = new Version(3000099, org.apache.lucene.util.Version.LUCENE_9_9_1);
+    public static final Version V_2_12_0 = new Version(2120099, org.apache.lucene.util.Version.LUCENE_9_9_2);
+    public static final Version V_3_0_0 = new Version(3000099, org.apache.lucene.util.Version.LUCENE_9_9_2);
     public static final Version CURRENT = V_3_0_0;
 
     public static Version fromId(int id) {

modules/lang-expression/licenses/lucene-expressions-9.9.2.jar.sha1

@@ -0,0 +1 @@
+045df3828b6460c032a6551040e31ea432b0aad6

modules/transport-netty4/licenses/netty-buffer-4.1.106.Final.jar.sha1

@@ -0,0 +1 @@
+c5495ba59a627641b3a7c23f6bcb801874c7f7b0

modules/transport-netty4/licenses/netty-codec-4.1.106.Final.jar.sha1

@@ -0,0 +1 @@
+025171b63aa1e7a5fd8a7e4e660d6d3110241ea7

modules/transport-netty4/licenses/netty-codec-http-4.1.106.Final.jar.sha1

@@ -0,0 +1 @@
+21a07cdf0fc46b313fe2248f1275cdbdac0ba87b

modules/transport-netty4/licenses/netty-codec-http2-4.1.106.Final.jar.sha1

@@ -0,0 +1 @@
+41515e8c51eeaaddceabdb4f86fbc5dbfc25b70e

modules/transport-netty4/licenses/netty-common-4.1.106.Final.jar.sha1

@@ -0,0 +1 @@
+132defb4689f41b51b483b7202b22b6e89fe35fd

modules/transport-netty4/licenses/netty-handler-4.1.106.Final.jar.sha1

@@ -0,0 +1 @@
+874c970c4ff958b1140dde52bc17e6a9e7cde662

modules/transport-netty4/licenses/netty-resolver-4.1.106.Final.jar.sha1

@@ -0,0 +1 @@
+e185ae573db04939215f94d6ba869758dcecbde9

modules/transport-netty4/licenses/netty-transport-4.1.106.Final.jar.sha1

@@ -0,0 +1 @@
+8fcca44ae16b98e15965093e7696832019fd6f27

modules/transport-netty4/licenses/netty-transport-native-unix-common-4.1.106.Final.jar.sha1

@@ -0,0 +1 @@
+2da179bd95903f0fa73218b8f0d02690c0cfbc94

plugins/analysis-icu/licenses/lucene-analysis-icu-9.9.2.jar.sha1

@@ -0,0 +1 @@
+f7e549fdac07140f4cd379a0f517c38434165e95

plugins/analysis-kuromoji/licenses/lucene-analysis-kuromoji-9.9.2.jar.sha1

@@ -0,0 +1 @@
+eb19738fd3ca85071fef96978a056a7c94d8e793

plugins/analysis-nori/licenses/lucene-analysis-nori-9.9.2.jar.sha1

@@ -0,0 +1 @@
+7e83a83741155ececf9193a4f967e570e170236d

plugins/analysis-phonetic/licenses/lucene-analysis-phonetic-9.9.2.jar.sha1

@@ -0,0 +1 @@
+dabdea96c7a6d00363b0093a580e7d783efa69a4

plugins/analysis-smartcn/licenses/lucene-analysis-smartcn-9.9.2.jar.sha1

@@ -0,0 +1 @@
+fb551d9ae6a1bf12bc90a4d26dd8fa3eefac8cb9

plugins/analysis-stempel/licenses/lucene-analysis-stempel-9.9.2.jar.sha1

@@ -0,0 +1 @@
+0d495b478861b2d29e0a58d273ca0e6e755292e9

plugins/analysis-ukrainian/licenses/lucene-analysis-morfologik-9.9.2.jar.sha1

@@ -0,0 +1 @@
+9958d813d4cfdd890b4611c679ed36775480fa0d

plugins/repository-azure/licenses/netty-codec-dns-4.1.106.Final.jar.sha1

@@ -0,0 +1 @@
+9d4b3315bb625ec2d73fa569fb6bce4589243d5e

plugins/repository-azure/licenses/netty-codec-http2-4.1.106.Final.jar.sha1

@@ -0,0 +1 @@
+41515e8c51eeaaddceabdb4f86fbc5dbfc25b70e

plugins/repository-azure/licenses/netty-codec-socks-4.1.106.Final.jar.sha1

@@ -0,0 +1 @@
+129f9bd6aa0cf28bfd7d45a8b6a598eed9c67702

plugins/repository-azure/licenses/netty-handler-proxy-4.1.106.Final.jar.sha1

@@ -0,0 +1 @@
+bffc88b7c56d7b553453e4244247a1b0ba1fdc8a

plugins/repository-azure/licenses/netty-resolver-dns-4.1.106.Final.jar.sha1

@@ -0,0 +1 @@
+3970474ed55aa1e3e8de5a5602c342c6b8155371

plugins/repository-azure/licenses/netty-transport-native-unix-common-4.1.106.Final.jar.sha1

@@ -0,0 +1 @@
+2da179bd95903f0fa73218b8f0d02690c0cfbc94

plugins/repository-azure/licenses/reactor-netty-core-1.1.15.jar.sha1

@@ -0,0 +1 @@
+3221d405ad55a573cf29875a8244a4217cf07185

plugins/repository-azure/licenses/reactor-netty-http-1.1.15.jar.sha1

@@ -0,0 +1 @@
+c79756fa2dfc28ac81fc9d23a14b17c656c3e560

plugins/repository-gcs/build.gradle

@@ -78,7 +78,7 @@ dependencies {
   api 'com.google.http-client:google-http-client:1.43.3'
   api 'com.google.http-client:google-http-client-appengine:1.43.3'
   api 'com.google.http-client:google-http-client-gson:1.43.3'
-  api 'com.google.http-client:google-http-client-jackson2:1.43.3'
+  api 'com.google.http-client:google-http-client-jackson2:1.44.1'
 
   api 'com.google.oauth-client:google-oauth-client:1.34.1'
 

plugins/repository-gcs/licenses/google-http-client-jackson2-1.44.1.jar.sha1

@@ -0,0 +1 @@
+3f1947de0fd9eb250af16abe6103c11e68d11635

plugins/repository-hdfs/licenses/netty-all-4.1.106.Final.jar.sha1

@@ -0,0 +1 @@
+6ec5bd2be65d5529e58e9e482b747c1135b3736b

plugins/repository-s3/licenses/netty-buffer-4.1.106.Final.jar.sha1

@@ -0,0 +1 @@
+c5495ba59a627641b3a7c23f6bcb801874c7f7b0

plugins/repository-s3/licenses/netty-codec-4.1.106.Final.jar.sha1

@@ -0,0 +1 @@
+025171b63aa1e7a5fd8a7e4e660d6d3110241ea7

plugins/repository-s3/licenses/netty-codec-http-4.1.106.Final.jar.sha1

@@ -0,0 +1 @@
+21a07cdf0fc46b313fe2248f1275cdbdac0ba87b

plugins/repository-s3/licenses/netty-codec-http2-4.1.106.Final.jar.sha1

@@ -0,0 +1 @@
+41515e8c51eeaaddceabdb4f86fbc5dbfc25b70e

plugins/repository-s3/licenses/netty-common-4.1.106.Final.jar.sha1

@@ -0,0 +1 @@
+132defb4689f41b51b483b7202b22b6e89fe35fd

plugins/repository-s3/licenses/netty-handler-4.1.106.Final.jar.sha1

@@ -0,0 +1 @@
+874c970c4ff958b1140dde52bc17e6a9e7cde662

plugins/repository-s3/licenses/netty-resolver-4.1.106.Final.jar.sha1

@@ -0,0 +1 @@
+e185ae573db04939215f94d6ba869758dcecbde9

plugins/repository-s3/licenses/netty-transport-4.1.106.Final.jar.sha1

@@ -0,0 +1 @@
+8fcca44ae16b98e15965093e7696832019fd6f27

plugins/repository-s3/licenses/netty-transport-classes-epoll-4.1.106.Final.jar.sha1

@@ -0,0 +1 @@
+c058d5c712e00e8560e519970b3d27747778b8f2

plugins/repository-s3/licenses/netty-transport-native-unix-common-4.1.106.Final.jar.sha1

@@ -0,0 +1 @@
+2da179bd95903f0fa73218b8f0d02690c0cfbc94

plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3BlobContainer.java

@@ -514,7 +514,7 @@ private static List<ListObjectsV2Response> executeListing(
             for (ListObjectsV2Response listObjectsV2Response : listObjectsIterable) {
                 results.add(listObjectsV2Response);
                 totalObjects += listObjectsV2Response.contents().size();
-                if (limit != -1 && totalObjects > limit) {
+                if (limit != -1 && totalObjects >= limit) {
                     break;
                 }
             }