-
Notifications
You must be signed in to change notification settings - Fork 109
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Not compatible with aws-java-sdk-bundle #36
Comments
I dont understand this purpose, adding AWS support for Spark is broken: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-aws/3.3.1 is adding AWS SDK bundle, so AWS SDK S3 will not work. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
aws-java-sdk-bundle
is one of the modules in the AWS SDK repository. Its purpose is to shade all of the SDK's third-party dependencies such as Jackson ororg.apache.commons.logging
.In the
AmazonSQSExtendedClient
class there is a methodgetTextFromS3()
. Within that method, there is a call toIOUtils.closeQuietly(is, LOG);
.LOG
is of typeorg.apache.commons.logging.Log
. The problem is, if you use the shaded version of the SDK, thenIOUtils.closeQuietly
is expectingcom.amazonaws.thirdparty.apache.logging.Log
instead, which causes it to throw ajava.lang.NoSuchMethodError
.I believe this problem could be solved by converting this library into multiple modules and offer a shading module with the exact same rules as
aws-java-sdk-bundle
.The text was updated successfully, but these errors were encountered: