Remove fuzzy set from experimental

[mgodwan]

Description

#12126

Related Issues

#12126

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
• Commits are signed per the DCO using --signoff
• Commit changes are listed out in CHANGELOG.md file (See: Changelog)
• Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

Compatibility status:

Checks if related components are compatible with change 6b0a9ef

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/flow-framework.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/sql.git]

[github-actions]

❌ Gradle check result for 5825d99: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

✅ Gradle check result for 064b910: SUCCESS

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 71.49%. Comparing base (b15cb0c) to head (9517c31).
Report is 44 commits behind head on main.

❗ Current head 9517c31 differs from pull request most recent head 6b0a9ef. Consider uploading reports for the commit 6b0a9ef to get more accurate results

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #12631      +/-   ##
============================================
+ Coverage     71.42%   71.49%   +0.07%     
- Complexity    59978    60033      +55     
============================================
  Files          4985     4985              
  Lines        282275   282386     +111     
  Branches      40946    40964      +18     
============================================
+ Hits         201603   201894     +291     
+ Misses        63999    63729     -270     
- Partials      16673    16763      +90     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[github-actions]

✅ Gradle check result for ffe16fb: SUCCESS

[github-actions]

✅ Gradle check result for 9517c31: SUCCESS

[github-actions]

❌ Gradle check result for 6b0a9ef: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[shwetathareja]

Unrelated test failures

org.opensearch.telemetry.metrics.TelemetryMetricsEnabledSanityIT.testObservableGauge - #12717
org.opensearch.index.shard.IndexShardTests.testCommitLevelRestoreShardFromRemoteStore - #10704

[opensearch-trigger-bot]

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.x
# Create a new branch
git switch --create backport/backport-12631-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 1f5df547aea4d03d285bd326680f07944e232cb1
# Push it to GitHub
git push --set-upstream origin backport/backport-12631-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-12631-to-2.x.

[reta]

@shwetathareja why we are merging pull request with failing Gradle checks?

[shwetathareja]

@shwetathareja why we are merging pull request with failing Gradle checks?

@reta those were known test flaky test failures. I have added the comment as well before merging.

[reta]

@shwetathareja please do not merge pull requests when checks are red, the flaky test procedure is described in description, but it is difficult to understand when test becomes non flaky because of change in the request itself, green checks at least give some guarantees, I hope you would agree. Thank you.

[shwetathareja]

@reta : Is there a consensus from all maintainers to not merge with gradle check failing due to known flaky test failures (with same stack traces)? Gradle checks don't pass even after multiple attempts.

it is difficult to understand when test becomes non flaky because of change in the request itself,

This may not be true with green gradle check after 10 or so failed attempts.

Also, authors don't have even permission to trigger gradle check re-run.

[mgodwan]

Also, authors don't have even permission to trigger gradle check re-run.

+1, it is not feasible to trigger a gradle check run only. As authors, we usually need to put an empty commit just to trigger gradle checks which does not seem correct.

[reta]

@reta : Is there a consensus from all maintainers to not merge with gradle check failing due to known flaky test failures (with same stack traces)? Gradle checks don't pass even after multiple attempts.

@shwetathareja we have checks for a reason right? We sadly we very low engagement among maintainers on any subject but the issue is under discussion as we speak

This may not be true with green gradle check after 10 or so failed attempts.

If it is about the same test - this is at least a hint

Also, authors don't have even permission to trigger gradle check re-run.

Authors could do force push, this is non issue by and large (inconvenient - yes, blocker - no)

[shwetathareja]

we have checks for a reason right?

Those reasons shouldn't be wasting Authors and maintainers time, re-running gradle-check for the whole day.

[reta]

Those reasons shouldn't be wasting Authors and maintainers time, re-running gradle-check for the whole day.

Totally, so why we (maintainers) led to that to happen? Why instead of merging with failed check not to go mute / fix / remove test? You are a maintainer as well - please help.

[dblock]

My 0.02c is that if we start merging changes with "unrelated failed tests", we will never have a green build, nor will contributors feel the pain of flaky tests and those will never get fixed. I am with @reta to keep the requirement of a passing gradle check, unrelated failure or not.

[shwetathareja]

@dblock thanks for sharing your thoughts. The point I am trying to bring here is that mindless re-running of gradle check doesn't help much. We should focus more on fixing flaky tests instead. I know we have consensus on that already but we lack any mechanism on it. Making the contributors go through the pain while merging may not be sufficient.

Should we assign one flaky test to each maintainer/ frequent contributors (which aligns to the component they work closely in OpenSearch core) so that everyone contributes and help us bring the flaky test count down.

[reta]

@shwetathareja we had a bunch of initiative to address that (we very low participation from maintainers), just to name a few that were formalized:

• Block merging code/features into 2.12 until all flaky tests are fixed. #10371
• Mechanism to ensure newly added tests are not flaky #9462