Enforce that S3 credentials are vended when requested #2711

dimas-b · 2025-09-29T17:34:28Z

This is a follow-up change to #2672 striving to improve user-facing error reporting for S3 storage systems without STS.

Add property to AccessConfig to indicate whether the backing storage integration can produce credentials.
Add a check to IcebergCatalogHandler (leading to 400) that storage credentials are vended when requested and the backend is capable of vending credentials in principle.
Update PolarisStorageIntegrationProviderImpl to indicate that FILE storage does not support credential vending (requesitng redential vending with FILE storage does not produce any credentials and does not flag an error, which matches current Polaris behaviour).
Only those S3 systems where STS is not available (or disabled / not permitted) are affected.
Other storage integrations are not affected by this PR.

flyrain · 2025-09-30T00:44:27Z

.../service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalogHandler.java

              tableIdentifier, tableMetadata, actions, refreshCredentialsEndpoint);
      Map<String, String> credentialConfig = accessConfig.credentials();
-      if (!credentialConfig.isEmpty() && delegationModes.contains(VENDED_CREDENTIALS)) {
+      if (delegationModes.contains(VENDED_CREDENTIALS)) {


Thanks @dimas-b for the change. But I think this is not the right place to check. SKIP_CREDENTIAL_SUBSCOPING_INDIRECTION is mainly for POC or test env, where we allow Polaris to skip credential vending no matter what clients ask.

I believe the right place to check is in AwsCredentialsStorageIntegration::getSubscopedCreds(). Clients ask for credential vending, but this storage doesn't have the capability to do so.

AwsCredentialsStorageIntegration.getSubscopedCreds() may be called when STS is disabled (so no credentials will be vended). It's a valid use case for non-AWS S3 storage. Clients still need non-credential config from the "storage integration" code (e.g. endpoints).

We should put the check within if (shouldUseSts(storageConfig)) { ... }, sorry, the method shouldUseSts() itself.

@flyrain : could you propose pseudo-code (or java example) for this check?

Updated. PTAL.

This is follow-up bugfix for apache#2589 The bugfix part apache#2711 is extracted here since apache#2711 proved to be non-trivial and may require extra time. * Use the `delegationModes` method parameter as intended (as opposed to a local constant).

…2713) This is follow-up bugfix for #2589 The bugfix part #2711 is extracted here since #2711 proved to be non-trivial and may require extra time. * Use the `delegationModes` method parameter as intended (as opposed to a local constant).

dimas-b · 2025-09-30T04:41:21Z

client/python/integration_tests/conftest.py

            "warehouse": test_catalog.name,
            "credential": f"{test_principal.principal.client_id}:{test_principal.credentials.client_secret.get_secret_value()}",
            "scope": "PRINCIPAL_ROLE:ALL",
-            "header.X-Iceberg-Access-Delegation": "vended-credentials",


Cf. https://github.com/apache/iceberg-python/blob/pyiceberg-0.10.0/pyiceberg/catalog/rest/__init__.py#L487

singhpk234

Thank you @dimas-b !

singhpk234 · 2025-09-30T05:11:57Z

CHANGELOG.md

 - The EclipseLink Persistence implementation has been deprecated since 1.0.0 and will be completely removed
  in 1.3.0 or in 2.0.0 (whichever happens earlier).
+- Client requests for vended credentials will fail by default if no credentials are available (they used to return
+  no credentials). This can happen with FILE and S3 storage that does not have STS. Clients should normally be updated


Suggested change

no credentials). This can happen with FILE and S3 storage that does not have STS. Clients should normally be updated

no credentials). This can happen with FILE and S3 compatible storage that does not have STS. Clients should normally be updated

AWS S3 may be used without STS (admin's choice).

How about This can happen with FILE and S3 storage if STS is not available or disabled (not permitted).?

SG ! for my understanding, when is the case when its actual S3 but we don't credential vending ? is it for case where the client already has credentials to work on storage, but then would they be sending the header ? or we just want admins to have this knob

Admins sometimes put restrictions on various policies / permissions even in AWS where STS endpoints are available.

Clients should not request vended credentials when they are known to be unavailable. The extra check is basically to make this kind of error more visible, I guess.

A changelog entry is not needed for the reworked changeset... WDYT?

singhpk234 · 2025-09-30T05:14:53Z

.../src/main/java/org/apache/polaris/service/it/test/PolarisRestCatalogFileIntegrationTest.java

+  @Test
+  @Override
+  @Disabled // Polaris does not provide credentials for FILE storage
+  public void testLoadCredentials() {
+    super.testLoadCredentials();
+  }


should we run these test with the config to allow empty creds on ?

This is no longer disabled.

runtime/service/src/intTest/java/org/apache/polaris/service/it/RestCatalogMinIOSpecialIT.java

dimas-b · 2025-09-30T17:54:15Z

It looks like the ripple effects of validating that vended credentials are available at the REST API levels are too wide.

I'll reset this PR and take a different approach (as @flyrain suggested).

dimas-b · 2025-10-01T02:08:44Z

Reworked this PR completely. Please review again.

HonahX · 2025-10-01T20:52:01Z

polaris-core/src/main/java/org/apache/polaris/core/storage/cache/StorageCredentialCacheKey.java

        allowedWriteLocations,
-        refreshCredentialsEndpoint);
+        refreshCredentialsEndpoint,
+        credentialsRequired);


[non-blocker] I think this implies some extra cache miss for azure and gcp where credentialsRequired does not make any difference.

Do we have some general use-case of the credentialsRequired on gcp and azure, such that we could unify the credential existence check at storageCredentialCache, such that the persistence/credentialVendor just do its best effort to generate an access config, and let the cache api to decide whether the result is valid?

Good point about cache 🤔 Thx!

Do we have some general use-case of the credentialsRequired on gcp and azure

My first attempt in this PR was do enforce credential presence close to the REST level, but it proved to have too wide a ripple effect because many use cases (including PyIceberg) request credential vending by default and that breaks on FILE storage.

I'll see if I can refactor the code to maintain cache validity, but still delegate config validation to storage integrations.

HonahX · 2025-10-01T20:58:25Z

...core/src/main/java/org/apache/polaris/core/storage/aws/AwsCredentialsStorageIntegration.java


-    if (shouldUseSts(storageConfig)) {
+    boolean shouldUseSts = shouldUseSts(storageConfig);
+    Preconditions.checkArgument(


I think this error will be wrapped in to an error SubScopedResult and finally be thrown as UnprocessableEntityException in the StorageCredentialCache line 155. Just check if this is intended

On the java client side it shows up as org.apache.iceberg.exceptions.RESTException: Unable to process: Failed to get subscoped credentials: Credential vending was requested, but STS is not available. Is that reasonable?

In the REST response it is indeed a 422 with type = UnprocessableEntityException

Sorry to clarify, I think this is ok since 422 implies the same error will occur again if client retries, which is the current case. But in the PR description

Add checks to AwsCredentialsStorageIntegration (leading to 400) that S3 storage credentials are vended when requested.

says this check will lead to 400. So just point out in case it is intended to be 400 : )

Good catch - thx! I'm reworking the impl. again... it will be 400 / IllegalArgumentException at the REST level.

Re-implemented. PTAL.

dimas-b · 2025-10-02T02:35:59Z

I revamped this PR again 😅 Since there were too many "undo" changes, I squashed and force-pushed. Sorry about losing change history, but I think this revision is has the smallest "diff" so far 😉 Please review again.

HonahX · 2025-10-02T20:37:27Z

polaris-core/src/main/java/org/apache/polaris/core/storage/PolarisStorageIntegration.java

+  public void validateCredentials(
+      @Nonnull AccessConfig accessConfig, boolean credentialsRequired) {}


Seems this is not used?

Thanks for catching this and sorry about leftovers from previous code revisions 😓 ... removed.

HonahX · 2025-10-02T20:44:08Z

...ava/org/apache/polaris/core/persistence/transactional/TransactionalMetaStoreManagerImpl.java

    PolarisStorageIntegration<PolarisStorageConfigurationInfo> storageIntegration =
        ms.loadPolarisStorageIntegrationInCurrentTxn(callCtx, reloadedEntity.getEntity());

+    storageIntegration.config();


I think we could remove this?

polaris-core/src/main/java/org/apache/polaris/core/storage/PolarisStorageIntegration.java

This is a follow-up change to apache#2672 striving to improve user-facing error reporting for S3 storage systems without STS. * Add property to `AccessConfig` to indicate whether the backing storage integration can produce credentials. * Add a check to `IcebergCatalogHandler` (leading to 400) that storage credentials are vended when requested and the backend is capable of vending credentials in principle. * Update `PolarisStorageIntegrationProviderImpl` to indicate that FILE storage does not support credential vending (requesitng redential vending with FILE storage does not produce any credentials and does not flag an error, which matches current Polaris behaviour). * Only those S3 systems where STS is not available (or disabled / not permitted) are affected. * Other storage integrations are not affected by this PR.

HonahX

LGTM! Thanks @dimas-b for adding the enforcement!

HonahX · 2025-10-03T19:10:47Z

.../service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalogHandler.java

+          Preconditions.checkArgument(
+              !accessConfig.supportsCredentialVending() || skipCredIndirection,
+              "Credential vending was requested for table %s, but no credentials are available",
+              tableIdentifier);


WDYT of moving this check to getAccessConfig /FileIOUtil.refreshAccessConfig in the later refactoring PR: #2736, and we probably do not need the additional supportsCredentialVending field in AccessConfig, just adding an additional arg to let caller decide whether to require the credential to be vended.

But I think that should be in a follow-up and after the 1.2.0 cut to avoid additional noise on the release. If I understand correctly, we do not offer backward compatibility of these inner interface. So we could do this if needed in the next release

Sounds reasonable to me.

TBH, I do not like AccessConfig.supportsCredentialVending(), but it seemed to be the simplest way to make the check without breaking other things 😅

* (Based on PR#2223)Support Namespace/Table level RBAC for external passthrough catalogs (apache#2673) Creates missing synthetic entities for securables in external passthrough catalogs. Based on Option 1 discussed in the RBAC section of catalog federation design doc. In the future, we could remove calls to PolarisEntity.Builder() and replace them with entities fetched from the remote catalog. (enabling Option 2). --------- Co-authored-by: Pooja Nilangekar <poojan@umd.edu> * Docs: Add more details about v1 schema user to upgrade from 1.0 to 1.1 (apache#2674) * Site: The link https://iceberg.apache.org/concepts/catalog/ doesn't exist anymore. (apache#2683) * Docs: Add analytics for polaris.apache.org (apache#2676) * Make ENABLE_SUB_CATALOG_RBAC_FOR_FEDERATED_CATALOGS configurable per catalog (apache#2688) * Update ENABLE_SUB_CATALOG_RBAC_FOR_FEDERATED_CATALOGS to be configurable per catalog * chore(deps): update postgres docker tag to v18 (apache#2692) * fix(deps): update dependency org.eclipse.persistence:eclipselink to v4.0.8 (apache#2682) * fix(deps): update dependency org.apache.logging.log4j:log4j-core to v2.25.2 (apache#2646) * chore(deps): update dependency openapi-generator-cli to v7.15.0 (apache#2410) * chore(deps): update dependency io.quarkus to v3.27.0 (apache#2663) Co-authored-by: Mend Renovate <bot@renovateapp.com> * Publish Develocity builds scans for PRs and local use (apache#2596) This PR enables Develocity build scans for all PRs and contributors w/o an Apache account. CI build scans in the `apache/polaris` repo against branches and tags and having access to the ASF's Develocity secret continue to publish to the ASF's Develocity instance (no behavioral change). All other build scans are published to Gradle's public Develocity instance: - Build scans from local developer (non-CI) runs are only published, if Gradle is invoked with the `--scan` option. - Build scans from or targeting another repository than `apache/polaris` do need be enabled explicity by accepting Gradle's terms of service, via a repository variable, because this is a decision of the owner of a repository. Advanced options to configure another Develocity server or project-ID are available (for non-`apache/polaris` repositories). Detailed instructions in the `README.md`. * Fix & enhancements to the Events API hierarchy (apache#2629) Summary of changes: - Turned `PolarisEventListener` into an interface to facilitate implementation / mocking - Added missing `implements PolarisEvent` to many event records - Removed unused method overrides - Added missing method overrides to `TestPolarisEventListener` * fix(deps): update dependency org.kordamp.gradle:jandex-gradle-plugin to v2.3.0 (apache#2694) * Auth: reorganize internal authentication components (apache#2634) This PR contains no functional and no user-facing change. It is merely a refactor to better organize auth code. Summary of changes: - Moved all internal authentication components to the `org.apache.polaris.service.auth.internal` package and subpackages - Reduced visibility of utility classes - Renamed `TokenBroker` class hierarchy to stick to the naming standard: `<Algorithm>JWTBroker` - Introduced `@PolarisImmutable` whenever appropriate - Removed unused `NoneTokenBrokerFactory` (we already have `DisabledOAuth2ApiService`) - Removed unused `TokenBrokerFactoryConfig` * Enhancement : adding support for Aurora postgres AWS IAM authentication (apache#2650) Add support for postgres AWS IAM authentication using the `apache-client` lib. * Remove unused `name` arg from findCatalogByName in PolarisAdminService (apache#2691) * remove unused name param * Rename for better readability * Fix a race condition in sendNotification where concurrent parent-namespace creation causes failures (apache#2693) * Fix a race condition in sendNotification where concurrent parent-namespace creation causes failures The semantics of the createNonExistingNamespaces method used during sendNotification were supposed to be "create if needed". However, the behavior ended up surfacing an AlreadyExistsException if multiple concurrent sendNotification attempts were made for a brand-new namespace (where the notifications may be different tables). This would cause a table sync to fail if a sibling table was being synced at the same time, even though the new table should successfully get created under the shared namespace. * Also better future-proof the createNamespaceInternal logic by explicitly checking for ENTITY_ALREADY_EXISTS, per review suggestion. Log a less scary message since it's not an error scenario type of race condition, per review suggestion * Client: add credential reset option (apache#2698) * Client: add credential reset option * Client: add credential reset option * Client: add credential reset option * Add integration testing * Fix lint * fix(deps): update dependency software.amazon.awssdk:bom to v2.34.5 (apache#2702) * fix(deps): update dependency com.gradleup.shadow:shadow-gradle-plugin to v9.2.2 (apache#2661) * Support S3 storage that does not have STS (apache#2672) * Support S3 storage that does not have STS This change is backward compatible with old catalogs that have storage configuration for S3 systems with STS. * Add new property to S3 storage config: `stsUnavailable` (defaults to "available"). * Do not call STS when unavailable in `AwsCredentialsStorageIntegration`, but still put other properties (e.g. s3.endpoint) into `AccessConfig` Relates to apache#2615 Relates apache#2207 * Docs/improve idp documentation (apache#2695) * Fix Github links in IDP documentation * Separate IDP docs for usage and development * - Add telemetry config example - Fix link to getting started from landing page - Fix mentioning role-arn as required * Fix some relative links (local Hugo resolves them properly, but PR auto checks still fails) * Docs: narrow down --role-arn usage for AWS S3 only; fix a link in keycloak guide. * Docs: fix a link in keycloak guide. * chore(deps): update gradle/actions digest to 748248d (apache#2708) * Client: fix integration testing (apache#2700) * Add fallback in case the VERSION table is not present (apache#2653) * initial commit * wire up * pastefix * change to postgres specific code * [Catalog Federation] Add feature flag to disallow setting sub-RBAC for federated catalog at catalog level (apache#2696) In apache#2688 (comment), we've identified that configuring polaris.config.enable-sub-catalog-rbac-for-federated-catalogs at catalog level should not be allowed in all cases, especially when the owner is not the same subject as the catalog user or admin. This PR add a feature flag, ALLOW_SETTING_SUB_CATALOG_RBAC_FOR_FEDERATED_CATALOGS to allow owner to disable catalog level setting polaris.config.enable-sub-catalog-rbac-for-federated-catalogs * Fix `delegationModes` parameter propagation in `createTableStaged()` (apache#2713) This is follow-up bugfix for apache#2589 The bugfix part apache#2711 is extracted here since apache#2711 proved to be non-trivial and may require extra time. * Use the `delegationModes` method parameter as intended (as opposed to a local constant). * Generate Request IDs (if not specified); Return Request ID as a Header (apache#2602) * fix(deps): update dependency org.junit:junit-bom to v5.14.0 (apache#2715) * NoSQL persistence: add Java/Vert.X executor abstraction layer (apache#2527) Provides an abstraction to submit asynchronous tasks, optionally with a delay or delay + repetition and implementations based on Java's `ThreadPoolExecutor` and Vert.X. * Fix RDS devservices config + adopt for `:polaris-admin:test` (apache#2723) Changes: * Disables devservices for `:polaris-admin` tests as well, which is necessary to _not_ spin up test containers. * Use the explicit devservices-config as everywhere else. The first bullet point can cause excessive memory usage, especially with more test classes, eventually killing the whole GH runner. * fix(deps): update dependency io.smallrye:jandex to v3.5.0 (apache#2722) * fix(deps): update dependency org.jboss.weld:weld-junit5 to v5.0.2.final (apache#2721) * chore(deps): update quay.io/keycloak/keycloak docker tag to v26.4.0 (apache#2719) * Last merged commit 4024557 * NoSQL: Minor-ish changes to "nodes" projects Adopt nodes projects to OSS PR content * NoSQL: adapt to async package rename * Build: remove unnecessary explicit vertx-core dependency The async-vertx implementation should not propagate a different Vert.X dependency than Quarkus provides. This wouldn't be an issue if we could just use `enforcedPlatform()` for all Quarkus-builds, but sadly we cannot for the spark-plugin-inttests. --------- Co-authored-by: Honah (Jonas) J. <honahx@apache.org> Co-authored-by: Pooja Nilangekar <poojan@umd.edu> Co-authored-by: Prashant Singh <35593236+singhpk234@users.noreply.github.com> Co-authored-by: JB Onofré <jbonofre@apache.org> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Alexandre Dutra <adutra@apache.org> Co-authored-by: fabio-rizzo-01 <fabio.rizzocascio@jpmorgan.com> Co-authored-by: Dennis Huo <7410123+dennishuo@users.noreply.github.com> Co-authored-by: Yong Zheng <yongzheng0809@gmail.com> Co-authored-by: Dmitri Bourlatchkov <dmitri.bourlatchkov@gmail.com> Co-authored-by: olsoloviov <40199597+olsoloviov@users.noreply.github.com> Co-authored-by: Eric Maynard <eric.maynard+oss@snowflake.com> Co-authored-by: Adnan Hemani <adnan.h@berkeley.edu>

* Build: remove code to post-process generated Quarkus jars (apache#2667) Before Quarkus 3.28, the Quarkus generated jars used the "current" timestamp for all ZIP entries, which made the jars not-reproducible. Since Quarkus 3.28, the generated jars use a fixed timestamp for all ZIP entries, so the custom code is no longer necessary. This PR depends on Quarkus 3.28. * Update docker.io/jaegertracing/all-in-one Docker tag to v1.74.0 (apache#2751) * Updating metastore documentation with Aurora postgres example (apache#2706) * added Aurora postgres to metastore documentation * Service: Add events for APIs awaiting API changes (apache#2712) * fix(enhancement): add .idea, .vscode, .venv to top level .gitignore (apache#2718) fix(enhancement): add .idea, .vscode, .venv to top level .gitignore * Fix javadocs of `PolarisPrincipal.getPrincipalRoles()` (apache#2752) * fix(enhancement): squash commits (apache#2643) * fix(deps): update dependency io.smallrye.config:smallrye-config-core to v3.14.1 (apache#2755) * Extract interface for RequestIdGenerator (apache#2720) Summary of changes: 1. Extracted an interface from `RequestIdGenerator`. 2. The `generateRequestId` method now returns a `Uni<String>` in case custom implementations need to perform I/O or other blocking calls during request ID generation. 3. Also addressed comments in apache#2602. * JDBC: Handle schema evolution (apache#2714) * Deprecate legacy management endpoints for removal (apache#2749) * Deprecate LegacyManagementEndpoints for removal * Add PolarisResolutionManifestCatalogView.getResolvedCatalogEntity helper (apache#2750) this centralizes some common code and simplifies some test setups * Enforce that S3 credentials are vended when requested (apache#2711) This is a follow-up change to apache#2672 striving to improve user-facing error reporting for S3 storage systems without STS. * Add property to `AccessConfig` to indicate whether the backing storage integration can produce credentials. * Add a check to `IcebergCatalogHandler` (leading to 400) that storage credentials are vended when requested and the backend is capable of vending credentials in principle. * Update `PolarisStorageIntegrationProviderImpl` to indicate that FILE storage does not support credential vending (requesitng redential vending with FILE storage does not produce any credentials and does not flag an error, which matches current Polaris behaviour). * Only those S3 systems where STS is not available (or disabled / not permitted) are affected. * Other storage integrations are not affected by this PR. * [Catalog Federation] Ignore JIT entities when deleting federated catalogs, add integration test for namespace/table-level RBAC (apache#2690) When enabling table/namespace level RBAC in federated catalog, JIT entities will be created during privilege grant. In the short term, we should ignore them when dropping the catalog. In the long term, we will clean-up those entities when deleting the catalog. This will be the first step towards JIT entity clean-up: 1. Ignore JIT entities when dropping federated catalog (orphan entities) 2. Register tasks/in-place cleanup JIT entities during catalog drop 3. Add new functionality to PolarisMetastoreManager to support atomic delete non-used JIT entities during revoke. 4. Global Garbage Collector to clean-up unreachable entities (entities with non-existing catalog path/parent) * SigV4 Auth Support for Catalog Federation - Part 3: Service Identity Info Injection (apache#2523) This PR introduces service identity management for SigV4 Auth Support for Catalog Federation. Unlike user-supplied parameters, the service identity represents the identity of the Polaris service itself and should be managed by Polaris. * Service Identity Injection * Return injected service identity info in response * Use AwsCredentialsProvider to retrieve the credentials * Move some logic to ServiceIdentityConfiguration * Rename ServiceIdentityRegistry to ServiceIdentityProvider * Rename ResolvedServiceIdentity to ServiceIdentityCredential * Simplify the logic and add more tests * Use SecretReference and fix some small issues * Disable Catalog Federation * Update actions/stale digest to 5f858e3 (apache#2758) * Service: RealmContextFilter test refactor (apache#2747) * Update dependency software.amazon.awssdk:bom to v2.35.0 (apache#2760) * Update apache/spark Docker tag to v3.5.7 (apache#2727) * Update eric-maynard Team entry (apache#2763) I'm no longer affiliated with Snowflake, so we should update this page accordingly * Refactor resolutionManifest handling in PolarisAdminService (apache#2748) - remove mutable `resolutionManifest` field in favor of letting the "authorize" methods return their `PolarisResolutionManifest` - replace "find" helpers with "get" helpers that have built-in error handling * Implement Finer Grained Operations and Privileges For Update Table (apache#2697) This implements finer grained operations and privileges for update table in a backwards compatible way as discussed on the mailing list. The idea is that all the existing privileges and operations will work and continue to work even after this change. (i.e. TABLE_WRITE_PROPERTIES will still ensure update table is authorized even after these changes). However, because Polaris will now be able to identify each operation within an UpdateTable request and has a privilege model with inheritance that maps to each operation, users will now have the option of restricting permissions at a finer level if desired. * [Python CLI][CI Failure] Pin pydantic version to < 2.12.0 to fix CI failure (apache#2770) * Delete ServiceSecretReference (apache#2768) * JDBC: Fix Bootstrap with schema options (apache#2762) * Site: Add puppygraph integration (apache#2753) * Update Changelog with finer grained authz (apache#2775) * Add Arguments to Various Event Records (apache#2765) * Update immutables to v2.11.5 (apache#2776) * Client: add support for policy management (apache#2701) Implementation for policy management via Polaris CLI (apache#1867). Here are the subcommands to API mapping: attach - PUT /polaris/v1/{prefix}/namespaces/{namespace}/policies/{policy-name}/mappings create - POST /polaris/v1/{prefix}/namespaces/{namespace}/policies/{policy-name}/mappings delete - DELETE /polaris/v1/{prefix}/namespaces/{namespace}/policies/{policy-name} detach - POST /polaris/v1/{prefix}/namespaces/{namespace}/policies/{policy-name}/mappings get - GET /polaris/v1/{prefix}/namespaces/{namespace}/policies/{policy-name} list - GET /polaris/v1/{prefix}/namespaces/{namespace}/policies - This is default for `list` operation - GET /polaris/v1/{prefix}/applicable-policies - This is when we have `--applicable` option provided update - PUT /polaris/v1/{prefix}/namespaces/{namespace}/policies/{policy-name} * Update dependency com.google.cloud:google-cloud-storage-bom to v2.58.1 (apache#2764) * Update dependency org.jboss.weld:weld-junit5 to v5.0.3.Final (apache#2777) * Update the LICENSE and NOTICE files in the runtime (apache#2779) * SigV4 Auth Support for Catalog Federation - Part 4: Connection Credential Manager (apache#2759) This PR introduces a flexible credential management system for Polaris. Building on Part 3's service identity management, this system combines Polaris service identities with user-provided authentication parameters to generate credentials for remote catalog access. The core of this PR is the new ConnectionCredentialVendor interface, which: Generates connection credentials by combining service identity with user auth parameters Supports different authentication types (AWS SIGV4, AZURE Entra, GCP IAM) through CDI, currently only supports SigV4. Provides on-demand credential generation Enables easy extension for new authentication types In the long term, we should move the storage credential management logic out of PolarisMetastoreManager, PolarisMetastoreManager should only provide persistence interfaces. * Extract IcebergCatalog.getAccessConfig to a separate class AccessConfigProvider (apache#2736) This PR extracts credential vending entrypoint getAccessConfig from IcebergCatalog into a new centralized AccessConfigProvider class, decoupling credential generation from catalog implementations. The old SupportsCredentialVending is removed in this PR upon discussion * Update immutables to v2.11.6 (apache#2780) * Enhance Release docs (apache#2787) * Spark: Remove unnecessary dependency (apache#2789) * Update Pull Request Template (apache#2788) * Freeze 1.2 change log (apache#2783) * [Catalog Federation] Enable Credential Vending for Passthrough Facade Catalog (apache#2784) This PR introduces credential vending support for passthrough-facade catalogs. When creating a passthrough-facade catalog, the configuration currently requires two components: StorageConfig – specifies the storage info for the remote catalog. ConnectionInfo – defines connection parameters for the underlying remote catalog. With this change, the StorageConfig is now also used to vend temporary credentials for user requests. Credential vending honors table-level RBAC policies to determine whether to issue read-only or read-write credentials, ensuring access control consistency with Polaris authorization semantics. A new test case validates the credential vending workflow, verifying both read and write credential vending. Note: the remote catalog referenced by the passthrough-facade does not need to support IRC * Site: Add docs for catalog federation (apache#2761) * Python client: update CHANGELOG.MD for recent changes (apache#2796) * Python client: remove Python 3.9 support (apache#2795) * Update dependency software.amazon.awssdk:bom to v2.35.5 (apache#2799) * FIX REG tests with cloud providers (apache#2793) * [Catalog Federation] Block credential vending for remote tables outside allowed location list (apache#2791) * Correct invalid example in management service OpenAPI spec (apache#2801) The `example` was incorrectly placed as a sibling of `$ref` within a `schema` object in `polaris-management-service.yml`. According to the OpenAPI specification, properties that are siblings of a `$ref` are ignored. This was causing a `NullPointerException` in OpenAPI Generator v7.13.0+ due to a change in how examples are processed. The generator now expects all `examples` to be valid and non-empty, and a misplaced `example` can lead to a null reference when the generator tries to access it (we are not yet using v7.13.0+, thus not a problem at the moment). This commit moves the `example` to be a sibling of the `schema` object, which is the correct placement according to the OpenAPI specification. Reference error when using newer version of openapi-generator-cli: ``` openapi-generator-cli generate -i spec/polaris-catalog-service.yaml -g python -o client/python --additional-properties=packageName=polaris.catalog --additional-properties=apiNameSuffix="" --skip-validate-spec --additional-properties=pythonVersion=3.13 --ignore-file-override /local/client/python/.openapi-generator-ignore ... Exception: Cannot invoke "io.swagger.v3.oas.models.examples.Example.getValue()" because the return value of "java.util.Map.get(Object)" is null at org.openapitools.codegen.DefaultGenerator.processOperation(DefaultGenerator.java:1606) at org.openapitools.codegen.DefaultGenerator.processPaths(DefaultGenerator.java:1474) at org.openapitools.codegen.DefaultGenerator.generateApis(DefaultGenerator.java:663) at org.openapitools.codegen.DefaultGenerator.generate(DefaultGenerator.java:1296) at org.openapitools.codegen.cmd.Generate.execute(Generate.java:535) at org.openapitools.codegen.cmd.OpenApiGeneratorCommand.run(OpenApiGeneratorCommand.java:32) at org.openapitools.codegen.OpenAPIGenerator.main(OpenAPIGenerator.java:66) Caused by: java.lang.NullPointerException: Cannot invoke "io.swagger.v3.oas.models.examples.Example.getValue()" because the return value of "java.util.Map.get(Object)" is null at org.openapitools.codegen.utils.ExamplesUtils.unaliasExamples(ExamplesUtils.java:75) at org.openapitools.codegen.DefaultCodegen.unaliasExamples(DefaultCodegen.java:2343) at org.openapitools.codegen.DefaultCodegen.fromResponse(DefaultCodegen.java:4934) at org.openapitools.codegen.DefaultCodegen.fromOperation(DefaultCodegen.java:4575) at org.openapitools.codegen.DefaultGenerator.processOperation(DefaultGenerator.java:1574) ... 6 more ``` * Update dependency io.opentelemetry:opentelemetry-bom to v1.55.0 (apache#2804) * Update dependency io.micrometer:micrometer-bom to v1.15.5 (apache#2806) * Bump version for python deps (apache#2800) * bump version for python deps * bump version for python deps * bump version for python deps * Update openapi-generatr-cli from 7.11.0.post0 to 7.12.0 * Pin poetry version * Pin poetry version * Update dependency io.projectreactor.netty:reactor-netty-http to v1.2.11 (apache#2809) * [Catalog Federation] Add Connection Credential Vendors for Other Auth Types (apache#2782) Add Connection Credential Vendors for Other Auth Types This change is a prerequisite for enabling connection credential caching. By making PolarisCredentialManager the central entry point for obtaining connection credentials, we can introduce caching cleanly and manage all credential flows in a consistent way. * Last merged commit 6b957ec --------- Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: fabio-rizzo-01 <fabio.rizzocascio@jpmorgan.com> Co-authored-by: Adnan Hemani <adnan.h@berkeley.edu> Co-authored-by: Artur Rakhmatulin <artur.rakhmatulin@gmail.com> Co-authored-by: Alexandre Dutra <adutra@apache.org> Co-authored-by: Prashant Singh <35593236+singhpk234@users.noreply.github.com> Co-authored-by: Christopher Lambert <xn137@gmx.de> Co-authored-by: Dmitri Bourlatchkov <dmitri.bourlatchkov@gmail.com> Co-authored-by: Honah (Jonas) J. <honahx@apache.org> Co-authored-by: Rulin Xing <xjdkcsq3@gmail.com> Co-authored-by: Eric Maynard <eric.maynard+oss@snowflake.com> Co-authored-by: Travis Bowen <122238243+travis-bowen@users.noreply.github.com> Co-authored-by: Jaz Ku <jsku@dons.usfca.edu> Co-authored-by: Yong Zheng <yongzheng0809@gmail.com> Co-authored-by: JB Onofré <jbonofre@apache.org> Co-authored-by: Yufei Gu <yufei@apache.org>

github-project-automation bot added this to Basic Kanban Board Sep 29, 2025

github-project-automation bot moved this to PRs In Progress in Basic Kanban Board Sep 29, 2025

dimas-b added this to the 1.2.0 milestone Sep 29, 2025

dimas-b mentioned this pull request Sep 29, 2025

Support S3 storage that does not have STS #2672

Merged

flyrain reviewed Sep 30, 2025

View reviewed changes

dimas-b mentioned this pull request Sep 30, 2025

Fix delegationModes parameter propagation in createTableStaged() #2713

Merged

dimas-b force-pushed the enforce-vended-creds-presence branch from 049867d to 68ccb0e Compare September 30, 2025 04:39

dimas-b requested review from HonahX, adutra, flyrain and singhpk234 September 30, 2025 04:39

dimas-b commented Sep 30, 2025

View reviewed changes

singhpk234 reviewed Sep 30, 2025

View reviewed changes

dimas-b force-pushed the enforce-vended-creds-presence branch from cc52cad to ee9b6f7 Compare October 1, 2025 02:07

dimas-b changed the title ~~Enforce that credentials are vended when requested~~ Enforce that S3 credentials are vended when requested Oct 1, 2025

dimas-b requested a review from singhpk234 October 1, 2025 02:10

dimas-b force-pushed the enforce-vended-creds-presence branch 2 times, most recently from ac8970d to 11f909f Compare October 1, 2025 14:40

dimas-b mentioned this pull request Oct 1, 2025

Extract IcebergCatalog.getAccessConfig to a separate class AccessConfigProvider #2736

Merged

HonahX reviewed Oct 1, 2025

View reviewed changes

dimas-b force-pushed the enforce-vended-creds-presence branch from 11f909f to cc5036f Compare October 2, 2025 02:33

dimas-b requested a review from HonahX October 2, 2025 02:36

dimas-b force-pushed the enforce-vended-creds-presence branch from cc5036f to 10b7187 Compare October 2, 2025 14:54

HonahX reviewed Oct 2, 2025

View reviewed changes

dimas-b added 3 commits October 3, 2025 12:53

review: remove validateCredentials

1d570ea

review: remove spurious storageIntegration.config();

ca7a928

review: restore protected config()

9169873

dimas-b force-pushed the enforce-vended-creds-presence branch from 10b7187 to 9169873 Compare October 3, 2025 16:55

dimas-b requested a review from HonahX October 3, 2025 17:27

HonahX approved these changes Oct 3, 2025

View reviewed changes

github-project-automation bot moved this from PRs In Progress to Ready to merge in Basic Kanban Board Oct 3, 2025

dimas-b merged commit 927bb5e into apache:main Oct 3, 2025
16 of 17 checks passed

github-project-automation bot moved this from Ready to merge to Done in Basic Kanban Board Oct 3, 2025

dimas-b deleted the enforce-vended-creds-presence branch October 3, 2025 22:05

dimas-b mentioned this pull request Oct 17, 2025

Remove credential flag from StorageAccessProperty.CLIENT_REGION #2838

Merged

	no credentials). This can happen with FILE and S3 storage that does not have STS. Clients should normally be updated
	no credentials). This can happen with FILE and S3 compatible storage that does not have STS. Clients should normally be updated

		public void validateCredentials(
		@Nonnull AccessConfig accessConfig, boolean credentialsRequired) {}

Enforce that S3 credentials are vended when requested #2711

Enforce that S3 credentials are vended when requested #2711

Uh oh!

Conversation

dimas-b commented Sep 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dimas-b Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

flyrain Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

singhpk234 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dimas-b Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

singhpk234 Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

dimas-b commented Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dimas-b commented Oct 1, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dimas-b Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dimas-b Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

HonahX Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dimas-b Oct 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dimas-b commented Oct 2, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dimas-b commented Sep 29, 2025 •

edited

Loading

dimas-b Sep 30, 2025 •

edited

Loading

flyrain Sep 30, 2025 •

edited

Loading

dimas-b Sep 30, 2025 •

edited

Loading

singhpk234 Sep 30, 2025 •

edited

Loading

dimas-b commented Sep 30, 2025 •

edited

Loading

dimas-b Oct 1, 2025 •

edited

Loading

dimas-b Oct 1, 2025 •

edited

Loading

HonahX Oct 1, 2025 •

edited

Loading

dimas-b Oct 2, 2025 •

edited

Loading

dimas-b Oct 3, 2025 •

edited

Loading