Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add logs for remote store metadata intermittent read failures #8618

Merged

Conversation

linuxpi
Copy link
Collaborator

@linuxpi linuxpi commented Jul 11, 2023

Description

Adds logs to understand metadata checksum validation intermittent failures.

Related Issues

Helps with #8491

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

  • RESULT: UNSTABLE ❕
  • TEST FAILURES:
      1 org.opensearch.search.SearchTimeoutIT.testSimpleTimeout

@linuxpi linuxpi force-pushed the metadata-read-failure-log branch from 3766d7a to 092882a Compare July 11, 2023 10:56
@linuxpi linuxpi marked this pull request as ready for review July 11, 2023 10:56
@linuxpi linuxpi changed the title add logs for remote store metadata iontermittent read failures add logs for remote store metadata intermittent read failures Jul 11, 2023
@linuxpi linuxpi force-pushed the metadata-read-failure-log branch from 092882a to e1e2a2f Compare July 11, 2023 10:57
@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@linuxpi linuxpi force-pushed the metadata-read-failure-log branch from e1e2a2f to 346a84e Compare July 11, 2023 11:25
@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@codecov
Copy link

codecov bot commented Jul 11, 2023

Codecov Report

Merging #8618 (1198689) into main (a07aac8) will decrease coverage by 0.03%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##               main    #8618      +/-   ##
============================================
- Coverage     70.97%   70.94%   -0.03%     
+ Complexity    57076    57059      -17     
============================================
  Files          4759     4759              
  Lines        269652   269659       +7     
  Branches      39453    39453              
============================================
- Hits         191380   191321      -59     
- Misses        62123    62169      +46     
- Partials      16149    16169      +20     
Impacted Files Coverage Δ
...nsearch/common/io/VersionedCodecStreamWrapper.java 92.85% <100.00%> (+2.38%) ⬆️

... and 479 files with indirect coverage changes

@sachinpkale sachinpkale changed the title add logs for remote store metadata intermittent read failures Add logs for remote store metadata intermittent read failures Jul 11, 2023
@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@linuxpi linuxpi force-pushed the metadata-read-failure-log branch from 346a84e to 25ed20a Compare July 11, 2023 12:03
@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

  • RESULT: UNSTABLE ❕
  • TEST FAILURES:
      1 org.opensearch.remotestore.RemoteStoreIT.testStaleCommitDeletionWithInvokeFlush
      1 org.opensearch.indices.replication.SegmentReplicationRelocationIT.testPrimaryRelocation
      1 org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=snapshot.restore/10_basic/Create a snapshot and then restore it}

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@linuxpi
Copy link
Collaborator Author

linuxpi commented Jul 11, 2023

Test Failures:
Seems like flaky

    [org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=snapshot.restore/10_basic/Create a snapshot and then restore it}](https://build.ci.opensearch.org/job/gradle-check/19886/testReport/junit/org.opensearch.backwards/MixedClusterClientYamlTestSuiteIT/test__p0_snapshot_restore_10_basic_Create_a_snapshot_and_then_restore_it_/)
    [org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=snapshot.restore/10_basic/Create a snapshot and then restore it}](https://build.ci.opensearch.org/job/gradle-check/19886/testReport/junit/org.opensearch.backwards/MixedClusterClientYamlTestSuiteIT/test__p0_snapshot_restore_10_basic_Create_a_snapshot_and_then_restore_it__2/)
    [org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=snapshot.restore/10_basic/Create a snapshot and then restore it}](https://build.ci.opensearch.org/job/gradle-check/19886/testReport/junit/org.opensearch.backwards/MixedClusterClientYamlTestSuiteIT/test__p0_snapshot_restore_10_basic_Create_a_snapshot_and_then_restore_it__3/)
    [org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=snapshot.restore/10_basic/Create a snapshot and then restore it}](https://build.ci.opensearch.org/job/gradle-check/19886/testReport/junit/org.opensearch.backwards/MixedClusterClientYamlTestSuiteIT/test__p0_snapshot_restore_10_basic_Create_a_snapshot_and_then_restore_it__4/)
    [org.opensearch.remotestore.SegmentReplicationRemoteStoreIT.testPressureServiceStats](https://build.ci.opensearch.org/job/gradle-check/19886/testReport/junit/org.opensearch.remotestore/SegmentReplicationRemoteStoreIT/testPressureServiceStats/)
    [org.opensearch.remotestore.SegmentReplicationRemoteStoreIT.classMethod](https://build.ci.opensearch.org/job/gradle-check/19886/testReport/junit/org.opensearch.remotestore/SegmentReplicationRemoteStoreIT/classMethod/)
    [org.opensearch.remotestore.SegmentReplicationRemoteStoreIT.classMethod](https://build.ci.opensearch.org/job/gradle-check/19886/testReport/junit/org.opensearch.remotestore/SegmentReplicationRemoteStoreIT/classMethod_2/)

linuxpi added 2 commits July 11, 2023 21:04
Signed-off-by: bansvaru <bansvaru@amazon.com>
Signed-off-by: bansvaru <bansvaru@amazon.com>
@linuxpi linuxpi force-pushed the metadata-read-failure-log branch from 1cbda4f to 1198689 Compare July 11, 2023 15:35
@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@sachinpkale sachinpkale merged commit 542041f into opensearch-project:main Jul 11, 2023
@sachinpkale sachinpkale added the backport 2.x Backport to 2.x branch label Jul 11, 2023
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jul 11, 2023
---------

Signed-off-by: bansvaru <bansvaru@amazon.com>
(cherry picked from commit 542041f)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
sachinpkale pushed a commit that referenced this pull request Jul 11, 2023
…#8628)

---------


(cherry picked from commit 542041f)

Signed-off-by: bansvaru <bansvaru@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@linuxpi linuxpi deleted the metadata-read-failure-log branch July 12, 2023 05:01
vikasvb90 pushed a commit to raghuvanshraj/OpenSearch that referenced this pull request Jul 12, 2023
raghuvanshraj pushed a commit to raghuvanshraj/OpenSearch that referenced this pull request Jul 12, 2023
dzane17 pushed a commit to dzane17/OpenSearch that referenced this pull request Jul 12, 2023
buddharajusahil pushed a commit to buddharajusahil/OpenSearch that referenced this pull request Jul 18, 2023
…arch-project#8618)

---------

Signed-off-by: bansvaru <bansvaru@amazon.com>
Signed-off-by: sahil buddharaju <sahilbud@amazon.com>
baba-devv pushed a commit to baba-devv/OpenSearch that referenced this pull request Jul 29, 2023
shiv0408 pushed a commit to Gaurav614/OpenSearch that referenced this pull request Apr 25, 2024
…arch-project#8618)

---------

Signed-off-by: bansvaru <bansvaru@amazon.com>
Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch skip-changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants