[Refactor] Remove CollectionUtils.sortAndDedup and use java TreeSet instead of hppc ObjectArrayList

[nknize]

HPPC ObjectArrayLists provide no benefit in CollectionUtils.sortAndDedup and adds an unnecessary library dependency. This commit removes that dependency on hppc by switching the sortAndDeup method to use java.util.Lists. BinaryFieldMapper logic is updated to use the generic java list.

relates #5910

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: UNSTABLE ❕
• TEST FAILURES:

      1 org.opensearch.indices.replication.SegmentReplicationIT.testCancellation
      1 org.opensearch.client.ReindexIT.testReindexTask
      1 org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=indices.shrink/30_copy_settings/Copy settings during shrink index}

• URL: https://build.ci.opensearch.org/job/gradle-check/13058/
• CommitID: d8daa42
  Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[codecov-commenter]

Codecov Report

Merging #6884 (d8daa42) into main (7b708c4) will increase coverage by 0.27%.
The diff coverage is 100.00%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@             Coverage Diff              @@
##               main    #6884      +/-   ##
============================================
+ Coverage     70.52%   70.79%   +0.27%     
- Complexity    59112    59255     +143     
============================================
  Files          4814     4811       -3     
  Lines        283743   283702      -41     
  Branches      40915    40912       -3     
============================================
+ Hits         200110   200850     +740     
+ Misses        67154    66297     -857     
- Partials      16479    16555      +76     

Impacted Files	Coverage Δ
...va/org/opensearch/common/util/CollectionUtils.java	79.57% <100.00%> (+1.51%)	⬆️
...org/opensearch/index/mapper/BinaryFieldMapper.java	80.00% <100.00%> (-0.25%)	⬇️

... and 491 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/13078/
• CommitID: bf84acf
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/13082/
• CommitID: 4b9d24d
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/13083/
• CommitID: 33943f5
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/13086/
• CommitID: ae6571f
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[nknize]

I opened a separate PR to explore improving the bytes() method in CustomBinaryDocValuesField.

Note also that I have an upstream Lucene issue opened for multi value support for binary doc values since the size limit was removed. We have a community member interested in contributing their implementation so it's partially stalled. I'd like to not overwork this here in hopes that it will be migrated as a top level Lucene capability.

I'll poke that issue again. If there's no movement we'll float the implementation upstream. @navneet1v has expressed desire to work on this as a first Lucene contribution so we could move this conversation there for his awareness if/when he works on adding it.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS ✅
• URL: https://build.ci.opensearch.org/job/gradle-check/13108/
• CommitID: 3aa3164

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS ✅
• URL: https://build.ci.opensearch.org/job/gradle-check/13109/
• CommitID: aa5c38f

[reta]

LGTM from my side, @andrross wdyt?

[nknize]

Thanks all! I've moved the performance conversation to #6897. Getting this multi-value support in Lucene will be a big step (especially for geo_shape doc values! and geo_ aggs).

[andrross]

@nknize I see you tagged this for 3.0. Any reason not to backport? I don't see any usage of sortAndDedup beyond this code.

[nknize]

I see you tagged this for 3.0. Any reason not to backport?

No we can retag as v2.7.0 and backport