Skip to content

feat: pass vended credentials to Iceberg native scan#3523

Merged
mbutrovich merged 4 commits intoapache:mainfrom
tokoko:iceberg-native-vended-credentials
Feb 16, 2026
Merged

feat: pass vended credentials to Iceberg native scan#3523
mbutrovich merged 4 commits intoapache:mainfrom
tokoko:iceberg-native-vended-credentials

Conversation

@tokoko
Copy link
Contributor

@tokoko tokoko commented Feb 14, 2026

Closes #3522.

What changes are included in this PR?

  • obtains object storage credentials from FileIO objects through reflection and merges them with hadoop properties that are passed to native scanner.
  • there is SupportsStorageCredentials interface in latest iceberg which should eventually make reflection code less ugly, but we can't use it yet.
  • updates RestCatalogAdapter test utility to enable fake credential vending for testing.
  • this won't handle credential refresh in case temporary credentials expire. that's much harder problem to tackle.

How are these changes tested?

  • tested with RestCatalogAdapter that injects minio credentials into LoadTable response objects.
  • a negative test with wrong temporary credentials to make sure that it's the vended credentials that are being used rather than the ones set in hadoop configuration.

@mbutrovich mbutrovich self-requested a review February 16, 2026 16:26
@mbutrovich
Copy link
Contributor

Thank you for the PR @tokoko! I'm thrilled that others are trying to improve the native Iceberg integration. I will take a look today.

Copy link
Contributor

@mbutrovich mbutrovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again @tokoko. This looks really close! I have a few small suggestions.

Copy link
Contributor

@mbutrovich mbutrovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the great improvement, @tokoko! Approved pending CI.

@mbutrovich mbutrovich merged commit e29bf65 into apache:main Feb 16, 2026
108 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support credential vending in Iceberg native integration

2 participants