-
Notifications
You must be signed in to change notification settings - Fork 2.9k
Core: Pass storage credentials from LoadTableResponse to FileIO #12591
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
9bc2853 to
8db24df
Compare
core/src/main/java/org/apache/iceberg/rest/responses/LoadTableResponse.java
Outdated
Show resolved
Hide resolved
8db24df to
efe6592
Compare
efe6592 to
716a064
Compare
7848acb to
0c21c44
Compare
2f64867 to
0ca4c6b
Compare
amogh-jahagirdar
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @nastra! Passing through the credentials to the fileIO implementations looks great. Just had a comment for adding a JavaDoc on the new mixin interface since it is public.
core/src/main/java/org/apache/iceberg/io/SupportsStorageCredentials.java
Show resolved
Hide resolved
core/src/main/java/org/apache/iceberg/rest/responses/LoadTableResponse.java
Outdated
Show resolved
Hide resolved
0ca4c6b to
7d75581
Compare
|
thanks for the reviews @amogh-jahagirdar and @danielcweeks |
| } | ||
|
|
||
| @Override | ||
| public void setCredentials(List<StorageCredential> credentials) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From what I observe, the this.storageCredentials can be wrapped in ImmutableList.copyOf. This way, credentials() doesn't have to create a copy every time.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this will not work with Kryo ser/de because Kryo will try to add elements back after deserialization into the list and then fail
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the getter can be written this way (with additional field called copy):
first time getter assigns ImmutableList.copyOf() to copy field
subsequently copy field is returned directly until setter is called
No description provided.