Dremio merge 2025 08 05 16 04 (apache#100)

* feat: Add `endpointInternal` to `AwsStorageConfigInfo` (apache#2213)

* feat: Add `endpointInternal` to `AwsStorageConfigInfo`

This API change is backward compatible with older clients
and server using old storage configuration.

* The `endpointInternal` allows Polaris Servers to use a different
  host name (or IP address) for accessing S3 storage than clients.
  This is not a common use case, but may be relevant is more complex
  environments.

* If not set `endpointInternal` defaults to `endpoint`.

* The STS endpoint default changes to `endpointInternal`.

Contributes to apache#1530

* Fix deprecated Quarkus log properties (apache#2216)

see https://github.com/quarkusio/quarkus/wiki/Migration-Guide-3.19#other-changes-gear-white_check_mark

CI Quarkus tests were logging this repeatedly:
```
The "quarkus.log.file.json" config property is deprecated and should not be used anymore.
The "quarkus.log.console.json" config property is deprecated and should not be used anymore.
The "quarkus.log.file.json" config property is deprecated and should not be used anymore.
The "quarkus.log.console.json" config property is deprecated and should not be used anymore.
```

* chore(deps): update dependency mypy to >=1.17, <=1.17.1 (apache#2218)

* fix(deps): update dependency boto3 to v1.40.1 (apache#2240)

* fix(deps): update dependency com.azure:azure-sdk-bom to v1.2.37 (apache#2242)

* JDBC: Log SQL statements at debug level (apache#2221)

* Add ResolutionManifestFactory (apache#2210)

after 95358a9 very little functionality
was left in `PolarisEntityManager`.

by splitting out the more dedicated `ResolutionManifestFactory` we can remove
`PolarisEntityManager` and `RealmEntityManagerFactory` completely it seems.

* Prepare upgrade to Gradle 9 (apache#2237)

* `shadowPub.kt` the change removes a special case that doesn't apply to Polaris
* `api/...` build scripts - changes due to nullable type handling (`Property<String>` vs `Property<String?>` - latter is ... weird)

* fix(deps): update dependency software.amazon.awssdk:bom to v2.32.14 (apache#2246)

* Replace TestPolarisMetaStoreManager with Mockito.spy (apache#2230)

this takes less code and is more flexible in the future

* Remove obsolete information from README-quarkus.md (apache#2252)

* fix(deps): update dependency boto3 to v1.40.2 (apache#2256)

* fix(deps): update dependency com.gradleup.shadow:shadow-gradle-plugin to v8.3.9 (apache#2260)

* fix(deps): update immutables to v2.11.2 (apache#2257)

* Only pass `RealmConfig` to `PolarisStorageIntegration` (apache#2234)

All `PolarisStorageIntegration` requite only the `RealmConfig`, not the whole `CallContext`. This makes it easier for the new tasks impleemntations (both proposals).

* QuarkusProducers: remove unneeded `BasePersistence` producer (apache#2255)

* Use application-scope clock when generating GCP credentials (apache#2241)

This change also fixes a flaky test: `StorageConfigurationTest.testCreateGcpCredentialsFromStaticToken`

* chore(deps): update plugin jetbrains-changelog to v2.4.0 (apache#2264)

* chore(deps): update gradle/actions digest to 017a9ef (apache#2265)

* Remove config parameter from `PolarisStorageIntegration#getSubscopedCreds` (apache#2235)

Instances of `PolarisStorageIntegration` are created for a particular `PolarisStorageConfigurationInfo`, the same value is then passed into `PSI.getSubscopedCreds()`.

This change removes the config parameter, as it's already known when `PolarisStorageIntegration` instances are created.

* chore(deps): update registry.access.redhat.com/ubi9/openjdk-21-runtime docker tag to v1.22-1.1753981256 (apache#2266)

* Make `*StorageConfigurationInfo` types immutable (apache#2236)

This change eventually enables usage of the `*StorageConfigurationInfo` in the `StorageCredentialCacheKey` due to the then memoized hash-code values, to eliminate a couple of JSON re-serializations.

* JDBC: SERIALIZABLE/EntityNotFoundException (apache#2219)

With serializable isolation level, if either the primary-key or the check-constraint are violated, the lookup of the conflicting entity is not guaranteed to yield a non-null result (transaction start matters). This changes works around this situation w/ serializable isolation.

* Merge polaris-service-common into polaris-runtime-service (apache#2233)

* JdbcMetaStoreManagerFactory determines schemaVersion once per realm (apache#2217)

otherwise every call to `MetaStoreManagerFactory.getOrCreateSession` was
running the query.

* Last merged commit af69d9f

---------

Co-authored-by: Dmitri Bourlatchkov <dmitri.bourlatchkov@gmail.com>
Co-authored-by: Christopher Lambert <xn137@gmx.de>
Co-authored-by: Mend Renovate <bot@renovateapp.com>
Co-authored-by: Alexandre Dutra <adutra@apache.org>

Author: 5 people

.github/actions/ci-incr-build-cache-prepare/action.yml

@@ -45,7 +45,7 @@ runs:
         fi
 
     - name: Gradle / Setup
-      uses: gradle/actions/setup-gradle@ac638b010cf58a27ee6c972d7336334ccaf61c96 # v4
+      uses: gradle/actions/setup-gradle@017a9effdb900e5b5b2fddfb590a105619dca3c3 # v4
       with:
         cache-read-only: ${{ inputs.cache-read-only }}
         validate-wrappers: false

.github/workflows/check-md-link.yml

@@ -43,5 +43,5 @@ jobs:
       with:
         use-quiet-mode: 'yes'
         config-file: '.github/workflows/check-md-link-config.json'
-        folder-path: 'regtests, .github, build-logic, polaris-core, service, runtime, persistence, spec, getting-started, helm'
+        folder-path: 'regtests, .github, build-logic, polaris-core, runtime, persistence, spec, getting-started, helm'
         file-path: 'CHAT_BYLAWS.md, CODE_OF_CONDUCT.md, CONTRIBUTING.md, README.md, SECURITY.md'

.github/workflows/gradle.yml

@@ -47,7 +47,7 @@ jobs:
           java-version: '21'
           distribution: 'temurin'
       - name: Setup Gradle
-        uses: gradle/actions/setup-gradle@ac638b010cf58a27ee6c972d7336334ccaf61c96 # v4
+        uses: gradle/actions/setup-gradle@017a9effdb900e5b5b2fddfb590a105619dca3c3 # v4
         with:
           validate-wrappers: false
       - name: Prepare Gradle build cache
@@ -87,7 +87,7 @@ jobs:
           java-version: '21'
           distribution: 'temurin'
       - name: Setup Gradle
-        uses: gradle/actions/setup-gradle@ac638b010cf58a27ee6c972d7336334ccaf61c96 # v4
+        uses: gradle/actions/setup-gradle@017a9effdb900e5b5b2fddfb590a105619dca3c3 # v4
         with:
           validate-wrappers: false
       - name: Prepare Gradle build cache
@@ -124,7 +124,7 @@ jobs:
           java-version: '21'
           distribution: 'temurin'
       - name: Setup Gradle
-        uses: gradle/actions/setup-gradle@ac638b010cf58a27ee6c972d7336334ccaf61c96 # v4
+        uses: gradle/actions/setup-gradle@017a9effdb900e5b5b2fddfb590a105619dca3c3 # v4
         with:
           validate-wrappers: false
       - name: Prepare Gradle build cache
@@ -161,7 +161,7 @@ jobs:
           java-version: '21'
           distribution: 'temurin'
       - name: Setup Gradle
-        uses: gradle/actions/setup-gradle@ac638b010cf58a27ee6c972d7336334ccaf61c96 # v4
+        uses: gradle/actions/setup-gradle@017a9effdb900e5b5b2fddfb590a105619dca3c3 # v4
         with:
           validate-wrappers: false
       - name: Collect partial Gradle build caches

.github/workflows/nightly.yml

@@ -53,7 +53,7 @@ jobs:
           java-version: '21'
           distribution: 'temurin'
       - name: Setup Gradle
-        uses: gradle/actions/setup-gradle@ac638b010cf58a27ee6c972d7336334ccaf61c96 # v4
+        uses: gradle/actions/setup-gradle@017a9effdb900e5b5b2fddfb590a105619dca3c3 # v4
         with:
           validate-wrappers: false
       - name: Publish SNAPSHOTs to Apache Nexus Repository

.github/workflows/regtest.yml

@@ -41,7 +41,7 @@ jobs:
           distribution: 'temurin'
 
       - name: Setup Gradle
-        uses: gradle/actions/setup-gradle@ac638b010cf58a27ee6c972d7336334ccaf61c96 # v4
+        uses: gradle/actions/setup-gradle@017a9effdb900e5b5b2fddfb590a105619dca3c3 # v4
         with:
           validate-wrappers: false
 

.github/workflows/spark_client_regtests.yml

@@ -41,7 +41,7 @@ jobs:
           distribution: 'temurin'
 
       - name: Setup Gradle
-        uses: gradle/actions/setup-gradle@ac638b010cf58a27ee6c972d7336334ccaf61c96 # v4
+        uses: gradle/actions/setup-gradle@017a9effdb900e5b5b2fddfb590a105619dca3c3 # v4
         with:
           validate-wrappers: false
 

LICENSE

@@ -218,11 +218,11 @@ This product includes code from Apache Iceberg.
 * spec/iceberg-rest-catalog-open-api.yaml
 * spec/polaris-catalog-apis/oauth-tokens-api.yaml
 * integration-tests/src/main/java/org/apache/polaris/service/it/test/PolarisRestCatalogIntegrationBase.java
-* service/common/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalog.java
-* service/common/src/main/java/org/apache/polaris/service/catalog/iceberg/CatalogHandlerUtils.java
+* runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalog.java
+* runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/CatalogHandlerUtils.java
 * plugins/spark/v3.5/spark/src/main/java/org/apache/polaris/spark/PolarisRESTCatalog.java
 * plugins/spark/v3.5/spark/src/main/java/org/apache/polaris/spark/SparkCatalog.java
-* service/common/src/main/java/org/apache/polaris/service/catalog/common/LocationUtils.java
+* runtime/service/src/main/java/org/apache/polaris/service/catalog/common/LocationUtils.java
 
 Copyright: Copyright 2017-2025 The Apache Software Foundation
 Home page: https://iceberg.apache.org

README.md

@@ -51,8 +51,6 @@ Apache Polaris is organized into the following modules:
   - `polaris-api-management-model` - The Polaris management model
   - `polaris-api-management-service` - The Polaris management service
   - `polaris-api-iceberg-service` - The Iceberg REST service
-- Service modules:
-  - `polaris-service-common` - The main components of the Polaris server
 - Runtime modules:
   - `polaris-runtime-service` - The runtime components of the Polaris server
   - `polaris-runtime-defaults` - The runtime configuration defaults

api/iceberg-service/build.gradle.kts

@@ -63,13 +63,13 @@ val generatedOpenApiSrcDir = project.layout.buildDirectory.dir("generated-openap
 openApiGenerate {
   // The OpenAPI generator does NOT resolve relative paths correctly against the Gradle project
   // directory
-  inputSpec = specsDir.file("polaris-catalog-service.yaml").asFile.absolutePath
+  inputSpec = provider { specsDir.file("polaris-catalog-service.yaml").asFile.absolutePath }
   generatorName = "jaxrs-resteasy"
-  outputDir = generatedDir.get().asFile.absolutePath
+  outputDir = provider { generatedDir.get().asFile.absolutePath }
   apiPackage = "org.apache.polaris.service.catalog.api"
-  ignoreFileOverride = rootDir.file(".openapi-generator-ignore").asFile.absolutePath
-  removeOperationIdPrefix = true
-  templateDir = templatesDir.asFile.absolutePath
+  ignoreFileOverride.set(provider { rootDir.file(".openapi-generator-ignore").asFile.absolutePath })
+  templateDir.set(provider { templatesDir.asFile.absolutePath })
+  removeOperationIdPrefix.set(true)
   globalProperties.put("apis", "CatalogApi,ConfigurationApi,OAuth2Api")
   globalProperties.put("models", "false")
   globalProperties.put("apiDocs", "false")

api/management-model/build.gradle.kts

@@ -50,13 +50,13 @@ val generatedOpenApiSrcDir = project.layout.buildDirectory.dir("generated-openap
 openApiGenerate {
   // The OpenAPI generator does NOT resolve relative paths correctly against the Gradle project
   // directory
-  inputSpec = specsDir.file("polaris-management-service.yml").asFile.absolutePath
+  inputSpec = provider { specsDir.file("polaris-management-service.yml").asFile.absolutePath }
   generatorName = "jaxrs-resteasy"
-  outputDir = generatedDir.get().asFile.absolutePath
+  outputDir = provider { generatedDir.get().asFile.absolutePath }
   modelPackage = "org.apache.polaris.core.admin.model"
-  ignoreFileOverride = rootDir.file(".openapi-generator-ignore").asFile.absolutePath
-  removeOperationIdPrefix = true
-  templateDir = templatesDir.asFile.absolutePath
+  ignoreFileOverride.set(provider { rootDir.file(".openapi-generator-ignore").asFile.absolutePath })
+  removeOperationIdPrefix.set(true)
+  templateDir.set(provider { templatesDir.asFile.absolutePath })
   globalProperties.put("apis", "false")
   globalProperties.put("models", "")
   globalProperties.put("apiDocs", "false")