Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable remote integrity check for encrypted repos for ChecksumBlobstoreFormat #9908

Merged

Conversation

linuxpi
Copy link
Collaborator

@linuxpi linuxpi commented Sep 7, 2023

Description

  • Disable remote data integrity for encrypted repos in ChecksumBlobstoreFormat

Impact

Remote Metadata Store would break with encrypted repositories as there will be checksum mismatch while uploading to remote .

Related Issues

NA

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

…toreFormat

Signed-off-by: bansvaru <bansvaru@amazon.com>
@github-actions
Copy link
Contributor

github-actions bot commented Sep 7, 2023

Compatibility status:

Checks if related components are compatible with change ac73764

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git]

@github-actions
Copy link
Contributor

github-actions bot commented Sep 7, 2023

Gradle Check (Jenkins) Run Completed with:

@gbbafna gbbafna changed the title disable remote integritiy check for encrypted repos for ChecksumBlobS… Disable remote integritiy check for encrypted repos for ChecksumBlobS… Sep 7, 2023
Signed-off-by: bansvaru <bansvaru@amazon.com>
@linuxpi linuxpi force-pushed the remote-cluster-state-crypto-fix branch from ac73764 to 772c8b0 Compare September 7, 2023 19:32
@github-actions
Copy link
Contributor

github-actions bot commented Sep 7, 2023

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

github-actions bot commented Sep 7, 2023

Compatibility status:

Checks if related components are compatible with change 772c8b0

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git]

@linuxpi
Copy link
Collaborator Author

linuxpi commented Sep 7, 2023

build is UNSTABLE. maybe because it took 1.5 hours due to jenkins issue

> Task :server:internalClusterTest

Tests with failures:
 - org.opensearch.remotestore.SegmentReplicationUsingRemoteStoreIT.testPrimaryReceivesDocsDuringReplicaRecovery

3051 tests completed, 1 failed, 34 skipped

ran this locally and it passed. test seems flaky
./gradlew ':server:internalClusterTest' --tests "org.opensearch.remotestore.SegmentReplicationUsingRemoteStoreIT.testPrimaryReceivesDocsDuringReplicaRecovery" -Dtests.iters=5

@sachinpkale sachinpkale changed the title Disable remote integritiy check for encrypted repos for ChecksumBlobS… Disable remote integrity check for encrypted repos for ChecksumBlobS… Sep 7, 2023
@sachinpkale sachinpkale changed the title Disable remote integrity check for encrypted repos for ChecksumBlobS… Disable remote integrity check for encrypted repos for ChecksumBlobstoreFormat Sep 7, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Sep 7, 2023

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

github-actions bot commented Sep 8, 2023

Gradle Check (Jenkins) Run Completed with:

  • RESULT: UNSTABLE ❕
  • TEST FAILURES:
      1 org.opensearch.remotestore.SegmentReplicationUsingRemoteStoreIT.testCancelPrimaryAllocation
      1 org.opensearch.remotestore.RemoteStoreStatsIT.testStatsResponseFromLocalNode

@codecov
Copy link

codecov bot commented Sep 8, 2023

Codecov Report

Merging #9908 (772c8b0) into main (951f051) will increase coverage by 0.06%.
Report is 14 commits behind head on main.
The diff coverage is 80.74%.

@@             Coverage Diff              @@
##               main    #9908      +/-   ##
============================================
+ Coverage     71.02%   71.09%   +0.06%     
- Complexity    58040    58079      +39     
============================================
  Files          4831     4831              
  Lines        273927   274000      +73     
  Branches      39913    39920       +7     
============================================
+ Hits         194567   194796     +229     
+ Misses        63061    62830     -231     
- Partials      16299    16374      +75     
Files Changed Coverage Δ
...ensearch/action/bulk/TransportShardBulkAction.java 75.29% <0.00%> (+0.58%) ⬆️
...rg/opensearch/common/settings/ClusterSettings.java 92.85% <ø> (-0.33%) ⬇️
...pensearch/common/settings/FeatureFlagSettings.java 50.00% <ø> (ø)
...pensearch/common/settings/IndexScopedSettings.java 100.00% <ø> (ø)
.../java/org/opensearch/common/util/FeatureFlags.java 90.90% <ø> (-0.76%) ⬇️
...in/java/org/opensearch/index/shard/IndexShard.java 69.41% <ø> (+0.19%) ⬆️
...a/org/opensearch/index/translog/TranslogStats.java 90.27% <0.00%> (ø)
...g/opensearch/repositories/RepositoriesService.java 73.05% <ø> (+0.65%) ⬆️
server/src/main/java/org/opensearch/node/Node.java 85.54% <50.00%> (-0.19%) ⬇️
...rch/node/remotestore/RemoteStoreNodeAttribute.java 73.91% <50.00%> (-0.72%) ⬇️
... and 12 more

... and 435 files with indirect coverage changes

📢 Have feedback on the report? Share it here.

@gbbafna gbbafna merged commit 967ef31 into opensearch-project:main Sep 8, 2023
14 checks passed
@gbbafna gbbafna added backport 2.x Backport to 2.x branch backport 2.10 Backport to 2.10 branch labels Sep 8, 2023
opensearch-trigger-bot bot pushed a commit that referenced this pull request Sep 8, 2023
…oreFormat (#9908)

Signed-off-by: bansvaru <bansvaru@amazon.com>
(cherry picked from commit 967ef31)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
opensearch-trigger-bot bot pushed a commit that referenced this pull request Sep 8, 2023
…oreFormat (#9908)

Signed-off-by: bansvaru <bansvaru@amazon.com>
(cherry picked from commit 967ef31)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
gbbafna pushed a commit that referenced this pull request Sep 8, 2023
…oreFormat (#9908) (#9923)

(cherry picked from commit 967ef31)

Signed-off-by: bansvaru <bansvaru@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@linuxpi linuxpi deleted the remote-cluster-state-crypto-fix branch September 8, 2023 07:58
gbbafna pushed a commit that referenced this pull request Sep 8, 2023
…oreFormat (#9908) (#9924)

(cherry picked from commit 967ef31)

Signed-off-by: bansvaru <bansvaru@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
kaushalmahi12 pushed a commit to kaushalmahi12/OpenSearch that referenced this pull request Sep 12, 2023
…oreFormat (opensearch-project#9908)

Signed-off-by: bansvaru <bansvaru@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>
sarthakaggarwal97 pushed a commit to sarthakaggarwal97/OpenSearch that referenced this pull request Sep 20, 2023
brusic pushed a commit to brusic/OpenSearch that referenced this pull request Sep 25, 2023
…oreFormat (opensearch-project#9908)

Signed-off-by: bansvaru <bansvaru@amazon.com>
Signed-off-by: Ivan Brusic <ivan.brusic@flocksafety.com>
shiv0408 pushed a commit to Gaurav614/OpenSearch that referenced this pull request Apr 25, 2024
…oreFormat (opensearch-project#9908)

Signed-off-by: bansvaru <bansvaru@amazon.com>
Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch backport 2.10 Backport to 2.10 branch skip-changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants