Update Github pull request template to have a task for inspecting failing checks

[reta]

Description

Update Github pull request template to have a task for inspecting failing checks. Since we have not got the consensus on #10371, and this issue does not only persist but gets worst with mostly every new feature integrated, there is an alternative path to tackle the problem making it a part of the process.

The idea is to at least ask every contributor to inspect the failing checks (we are talking about Gradle checks primarily) and make a reference (by commenting) to:

• an existing known issue
• or, create and attach a new issue

It won't solve the problem of flaky tests going away but at least we may get attention (hopefully engaging more engineers into fixing flaky tests) and we should at least spot a new flaky tests proactively (since right now most people just push or retrigger the checks).

Related Issues

• Relates to Block merging code/features into 2.12 until all flaky tests are fixed. #10371

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Failing checks are inspected and point to the corresponding known issue(s)
• Commits are signed per the DCO using --signoff
• Commit changes are listed out in CHANGELOG.md file (See: Changelog)
• Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[dblock]

+1 to this change, but I suspect most people ignore these check boxes, and we already have text that says the same thing. We can highlight that, or maybe find a way to block PRs where the failures haven't been examined?

[reta]

We can highlight that, or maybe find a way to block PRs where the failures haven't been examined?

Thanks @dblock , I was thinking about making that a maintainers responsibility (or, alternatively, asking for 2 approvals on PR so at least other person crosschecks)

[github-actions]

Compatibility status:

Checks if related components are compatible with change 891344c

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/reporting.git]

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/28462/
• CommitID: 083628e
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[reta]

• URL: https://build.ci.opensearch.org/job/gradle-check/28462/

Should be fixed by #10754

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS ✅
• URL: https://build.ci.opensearch.org/job/gradle-check/28500/
• CommitID: d209f05

[codecov]

Codecov Report

Merging #10751 (891344c) into main (6641ef8) will decrease coverage by 0.13%.
Report is 1 commits behind head on main.
The diff coverage is n/a.

@@             Coverage Diff              @@
##               main   #10751      +/-   ##
============================================
- Coverage     71.20%   71.07%   -0.13%     
+ Complexity    58611    58527      -84     
============================================
  Files          4859     4859              
  Lines        276267   276267              
  Branches      40189    40189              
============================================
- Hits         196709   196356     -353     
- Misses        63150    63502     +352     
- Partials      16408    16409       +1     

see 483 files with indirect coverage changes

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: UNSTABLE ❕
• TEST FAILURES:

      2 org.opensearch.index.shard.RemoteIndexShardTests.classMethod
      1 org.opensearch.index.shard.RemoteIndexShardTests.testSegRepSucceedsOnPreviousCopiedFiles

• URL: https://build.ci.opensearch.org/job/gradle-check/28584/
• CommitID: 891344c
  Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[reta]

@dblock are you good with that? :-)

[opensearch-trigger-bot]

The backport to 1.3 failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-1.3 1.3
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-1.3
# Create a new branch
git switch --create backport/backport-10751-to-1.3
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 ffe9371b38f98a2c9314bb6cfed4f8f224ff085c
# Push it to GitHub
git push --set-upstream origin backport/backport-10751-to-1.3
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-1.3

Then, create a pull request where the base branch is 1.3 and the compare/head branch is backport/backport-10751-to-1.3.