Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] GoogleCloudStorageThirdPartyTests testCleanup failing #75023

Closed
original-brownbear opened this issue Jul 7, 2021 · 1 comment · Fixed by #75028
Closed

[CI] GoogleCloudStorageThirdPartyTests testCleanup failing #75023

original-brownbear opened this issue Jul 7, 2021 · 1 comment · Fixed by #75028
Assignees
@original-brownbear
Labels
:Distributed Coordination/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. >test-failure Triaged test failures from CI

Comments

@original-brownbear
Copy link
Member

This is the result of a changed exception type after upgrading the GCS dependency I think. Looking into it now.

Build scan:
https://gradle-enterprise.elastic.co/s/venckon4avoas/tests/:plugins:repository-gcs:gcsThirdPartyTest/org.elasticsearch.repositories.gcs.GoogleCloudStorageThirdPartyTests/testCleanup

Reproduction line:
./gradlew ':plugins:repository-gcs:gcsThirdPartyTest' --tests "org.elasticsearch.repositories.gcs.GoogleCloudStorageThirdPartyTests.testCleanup" -Dtests.seed=6316403C7D97DB97 -Dtests.locale=mk -Dtests.timezone=Africa/Bangui -Druntime.java=11 -Dtests.fips.enabled=true

Applicable branches:
master

Reproduces locally?:
Didn't try

Failure history:
https://gradle-enterprise.elastic.co/scans/tests?tests.container=org.elasticsearch.repositories.gcs.GoogleCloudStorageThirdPartyTests&tests.test=testCleanup

Failure excerpt:

org.elasticsearch.repositories.RepositoryVerificationException: [test-repo] path [integration_test_third_party_tests_6316403C7D97DB97] is not accessible on master node


  Caused by: java.lang.ClassCastException: class java.security.KeyStoreException cannot be cast to class java.io.IOException (java.security.KeyStoreException and java.io.IOException are in module java.base of loader 'bootstrap')

    at org.elasticsearch.repositories.gcs.SocketAccess.doPrivilegedIOException(SocketAccess.java:35)
    at org.elasticsearch.repositories.gcs.GoogleCloudStorageService.createClient(GoogleCloudStorageService.java:125)
    at org.elasticsearch.repositories.gcs.GoogleCloudStorageService.client(GoogleCloudStorageService.java:105)
    at org.elasticsearch.repositories.gcs.GoogleCloudStorageBlobStore.client(GoogleCloudStorageBlobStore.java:117)
    at org.elasticsearch.repositories.gcs.GoogleCloudStorageBlobStore.lambda$writeBlobMultipart$8(GoogleCloudStorageBlobStore.java:455)
    at org.elasticsearch.repositories.gcs.SocketAccess.lambda$doPrivilegedVoidIOException$0(SocketAccess.java:43)
    at java.security.AccessController.doPrivileged(AccessController.java:-2)
    at org.elasticsearch.repositories.gcs.SocketAccess.doPrivilegedVoidIOException(SocketAccess.java:42)
    at org.elasticsearch.repositories.gcs.GoogleCloudStorageBlobStore.writeBlobMultipart(GoogleCloudStorageBlobStore.java:454)
    at org.elasticsearch.repositories.gcs.GoogleCloudStorageBlobStore.writeBlob(GoogleCloudStorageBlobStore.java:244)
    at org.elasticsearch.repositories.gcs.GoogleCloudStorageBlobContainer.writeBlob(GoogleCloudStorageBlobContainer.java:78)
    at org.elasticsearch.repositories.gcs.GoogleCloudStorageBlobContainer.writeBlobAtomic(GoogleCloudStorageBlobContainer.java:91)
    at org.elasticsearch.repositories.blobstore.BlobStoreRepository.startVerification(BlobStoreRepository.java:1617)
    at org.elasticsearch.repositories.RepositoriesService$4.doRun(RepositoriesService.java:393)
    at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:737)
    at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.lang.Thread.run(Thread.java:834)
@original-brownbear original-brownbear added :Distributed Coordination/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs >test-failure Triaged test failures from CI labels Jul 7, 2021
@elasticmachine elasticmachine added the Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. label Jul 7, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (Team:Distributed)

@original-brownbear original-brownbear self-assigned this Jul 7, 2021
original-brownbear added a commit to original-brownbear/elasticsearch that referenced this issue Jul 7, 2021
@original-brownbear
Fix GCS Keystore Handling in FIPS Mode
63a137e 
In FIPS mode loading the `.p12` keystore used by the new SDK version is not supported
because of "PBE AlgorithmParameters not available". Fortunately, the SDK still includes
the old jks trust store so we can just manually load it the same way it was loaded by
the previous version to fix things.
Also, fixed `SocketAccess` to properly rethrow this kind of exception and not run into
a class cast issue.

Closes elastic#75023

relates googleapis/google-api-java-client#1738
@original-brownbear original-brownbear mentioned this issue Jul 7, 2021
Merged
original-brownbear added a commit that referenced this issue Jul 7, 2021
@original-brownbear
Fix GCS Keystore Handling in FIPS Mode (#75028)
01872e8 
In FIPS mode loading the `.p12` keystore used by the new SDK version is not supported
because of "PBE AlgorithmParameters not available". Fortunately, the SDK still includes
the old jks trust store so we can just manually load it the same way it was loaded by
the previous version to fix things.
Also, fixed `SocketAccess` to properly rethrow this kind of exception and not run into
a class cast issue.

Closes #75023

relates googleapis/google-api-java-client#1738
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@original-brownbear original-brownbear

Labels
:Distributed Coordination/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. >test-failure Triaged test failures from CI
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix GCS Keystore Handling in FIPS Mode original-brownbear/elasticsearch
2 participants
@original-brownbear @elasticmachine