Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix snapshot repository registration failure in mixed OpenSearch version setups #16831

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

ashking94
Copy link
Member

@ashking94 ashking94 commented Dec 11, 2024

Description

This PR addresses issue #16830, where snapshot repository registration fails in clusters with mixed OpenSearch versions, specifically when nodes running OS 2.17 are present alongside nodes running earlier 1.x or 2.x versions.

Problem:

  • Repository registration fails due to different handling of test paths in OS 2.17 compared to earlier versions.
  • This issue occurs during version upgrades, causing snapshot functionality to break.

Solution:

  • Implement version-aware logic for handling test paths.
  • Ensure backward compatibility with earlier OpenSearch versions.
  • Add checks to use the appropriate test path handling method based on the minimum node version in the cluster.

Changes:

  1. Modified code to include version checks.
  2. Updated test path handling logic to accommodate mixed version scenarios.

Testing:

  • Tested repository registration in clusters with:
    • OS 2.15 and OS 2.17 nodes
  • Verified successful repository registration in all scenarios.
  • Ensured existing functionality remains intact for homogeneous version clusters.

This fix ensures smooth operation of snapshot repositories during cluster upgrades and in environments where different OpenSearch versions coexist.

Related Issues

Resolves #16830

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link
Contributor

❌ Gradle check result for 2c427aa: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@ashking94 ashking94 changed the title Handle version during repository registration for prefix mode verific… Handle version during repository registration for prefix mode verification Dec 13, 2024
…ation

Signed-off-by: Ashish Singh <ssashish@amazon.com>
@ashking94 ashking94 force-pushed the repo-verification-fix branch from 2c427aa to 51a6e67 Compare December 13, 2024 16:12
@ashking94 ashking94 changed the title Handle version during repository registration for prefix mode verification Fix snapshot repository registration failure in mixed OpenSearch version setups Dec 13, 2024
@github-actions github-actions bot added bug Something isn't working Storage:Snapshots labels Dec 13, 2024
Copy link
Member Author

@ashking94 ashking94 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's difficult to add unit tests as the BlobStoreRepositoryTest.java is actually a single node spin up and testing. There are existing integration tests that invokes this flow. The code is partially tested in the ITs that spins up the remote store nodes.

Copy link
Contributor

❌ Gradle check result for 51a6e67: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@ashking94
Copy link
Member Author

❌ Gradle check result for 51a6e67: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Flaky test - #15831

Copy link
Contributor

❌ Gradle check result for 51a6e67: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for 51a6e67: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@ashking94
Copy link
Member Author

❌ Gradle check result for 51a6e67: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Flaky test - #16015

Copy link
Contributor

✅ Gradle check result for 51a6e67: SUCCESS

Copy link

codecov bot commented Dec 14, 2024

Codecov Report

Attention: Patch coverage is 40.00000% with 3 lines in your changes missing coverage. Please review.

Project coverage is 72.15%. Comparing base (5ba909a) to head (51a6e67).
Report is 7 commits behind head on main.

Files with missing lines Patch % Lines
...ch/repositories/blobstore/BlobStoreRepository.java 33.33% 0 Missing and 2 partials ⚠️
...nsearch/cluster/service/ClusterApplierService.java 0.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main   #16831      +/-   ##
============================================
+ Coverage     72.11%   72.15%   +0.04%     
- Complexity    65237    65271      +34     
============================================
  Files          5318     5318              
  Lines        304003   304015      +12     
  Branches      43992    43995       +3     
============================================
+ Hits         219228   219377     +149     
+ Misses        66874    66649     -225     
- Partials      17901    17989      +88     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

[BUG] Snapshot repository fails if minNode version is below OS 2.17 in mixed os version setup
1 participant