Skip to content

Commit cf40a0a

Browse files
snazyadutragh-yzoudimas-bflyrain
authored
Dremio merge 2025 05 28 13 57 (#77)
* fix: Remove duplicated code in IcebergCatalog (apache#1681) * Fix SparkClient listGenericTable to use ListGenericTablesRESTResponse ListTableResponse is the class used by iceberg endpoint, which is the same as ListGenericTablesRESTResponse. However, we are suppose to use ListGenericTablesRESTResponse to be correct * fix: Remove info log about deprecated internal method from PolarisConfiguration (apache#1672) Remove the INFO log about calls to this method in Polaris, because users cannot do anything about these messages. Phasing out old property names requires coordination with users (e.g. release notes), so it is not a matter of merely avoiding calls to that method in Polaris code. Fixes apache#1666 * Create a single binary distribution bundle (apache#1589) * fix(quickstart): Correct Quickstart Instructions (apache#1673) * Remove Java URI validations for Blob Storage providers (apache#1604) This is a retry at apache#1586, which I'll close if this PR is on the right direction instead. Java URI does not actually apply any normalization to URIs if we do not call URI.normalize() (which we currently do not). Additionally, blob storage providers like S3 and GCS can provide ".." and "." as valid fragments in URLs - which Java URI would attempt to normalize incorrectly. As a result, attempting to validate and/or normalize URIs for blob storage providers using the Java URI class is the incorrect behavior. While we may want to add location validation via regex later, removing it first should at least unblock the bug we see in apache#1545. * Improve test coverage for invalid inputs in Policy APIs (apache#1665) * Fix getting-started docker start by PR apache#1532 (apache#1687) * Fix the manual test broken by PR apache#1532 (apache#1688) * Fix credentials printing twice (apache#1682) * main: Update dependency com.diffplug.spotless:spotless-plugin-gradle to v7.0.4 (apache#1690) * INFO: last merged commit 493de03 --------- Co-authored-by: Alexandre Dutra <adutra@users.noreply.github.com> Co-authored-by: gh-yzou <167037035+gh-yzou@users.noreply.github.com> Co-authored-by: Dmitri Bourlatchkov <dmitri.bourlatchkov@gmail.com> Co-authored-by: Yufei Gu <yufei@apache.org> Co-authored-by: Adnan Hemani <adnan.h@berkeley.edu> Co-authored-by: William Hyun <william@apache.org> Co-authored-by: Christopher Lambert <xn137@gmx.de> Co-authored-by: Mend Renovate <bot@renovateapp.com>
1 parent 9355e6f commit cf40a0a

File tree

32 files changed

+638
-396
lines changed

32 files changed

+638
-396
lines changed

‎extension/persistence/relational-jdbc/src/main/java/org/apache/polaris/extension/persistence/relational/jdbc/JdbcMetaStoreManagerFactory.java‎

Lines changed: 3 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -135,17 +135,12 @@ public synchronized Map<String, PrincipalSecretsResult> bootstrapRealms(
135135

136136
for (String realm : realms) {
137137
RealmContext realmContext = () -> realm;
138-
if (!metaStoreManagerMap.containsKey(realmContext.getRealmIdentifier())) {
138+
if (!metaStoreManagerMap.containsKey(realm)) {
139139
initializeForRealm(realmContext, rootCredentialsSet, true);
140140
PrincipalSecretsResult secretsResult =
141141
bootstrapServiceAndCreatePolarisPrincipalForRealm(
142-
realmContext, metaStoreManagerMap.get(realmContext.getRealmIdentifier()));
143-
144-
if (rootCredentialsSet.credentials().containsKey(realm)) {
145-
LOGGER.info("Bootstrapped realm {} using preset credentials.", realm);
146-
}
147-
148-
results.put(realmContext.getRealmIdentifier(), secretsResult);
142+
realmContext, metaStoreManagerMap.get(realm));
143+
results.put(realm, secretsResult);
149144
}
150145
}
151146

‎getting-started/assets/cloud_providers/deploy-aws.sh‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,7 @@ POSTGRES_ADDR=$(echo $DESCRIBE_DB | jq -r '.["DBInstances"][0]["Endpoint"]' | jq
7070
export QUARKUS_DATASOURCE_JDBC_URL=$(printf '%s' "jdbc:postgresql://$POSTGRES_ADDR/POLARIS")
7171
export QUARKUS_DATASOURCE_USERNAME=postgres
7272
export QUARKUS_DATASOURCE_PASSWORD=postgres
73-
echo ($QUARKUS_DATASOURCE_JDBC_URL)
73+
echo $QUARKUS_DATASOURCE_JDBC_URL
7474

7575
S3_BUCKET_NAME="polaris-quickstart-s3-$RANDOM_SUFFIX"
7676
echo "S3 Bucket Name: $S3_BUCKET_NAME"
@@ -84,4 +84,4 @@ export STORAGE_LOCATION="s3://$S3_BUCKET_NAME/quickstart_catalog/"
8484
-Dquarkus.container-image.build=true \
8585
--no-build-cache
8686

87-
docker compose -f getting-started/jdbc/docker-compose-bootstrap-db.yml -f getting-started/jdbc/docker-compose.yml up -d
87+
docker compose -p polaris -f getting-started/jdbc/docker-compose-bootstrap-db.yml -f getting-started/jdbc/docker-compose.yml up -d

‎getting-started/assets/cloud_providers/deploy-azure.sh‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ POSTGRES_ADDR=$(echo $CREATE_DB_RESPONSE | jq -r '.host')
3232
export QUARKUS_DATASOURCE_JDBC_URL=$(printf '%s' "jdbc:postgresql://$POSTGRES_ADDR/POLARIS")
3333
export QUARKUS_DATASOURCE_USERNAME=postgres
3434
export QUARKUS_DATASOURCE_PASSWORD=postgres
35-
echo ($QUARKUS_DATASOURCE_JDBC_URL)
35+
echo $QUARKUS_DATASOURCE_JDBC_URL
3636

3737
STORAGE_ACCOUNT_NAME="polaristest$RANDOM_SUFFIX"
3838
STORAGE_CONTAINER_NAME="polaris-test-container-$RANDOM_SUFFIX"
@@ -71,4 +71,4 @@ EOF
7171
-Dquarkus.container-image.build=true \
7272
--no-build-cache
7373

74-
docker compose -f getting-started/jdbc/docker-compose-bootstrap-db.yml -f getting-started/jdbc/docker-compose.yml up -d
74+
docker compose -p polaris -f getting-started/jdbc/docker-compose-bootstrap-db.yml -f getting-started/jdbc/docker-compose.yml up -d

‎getting-started/assets/cloud_providers/deploy-gcp.sh‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@ gcloud sql databases create POLARIS --instance=$DB_INSTANCE_NAME
3939
export QUARKUS_DATASOURCE_JDBC_URL=$(printf '%s' "jdbc:postgresql://$POSTGRES_ADDR/POLARIS")
4040
export QUARKUS_DATASOURCE_USERNAME=postgres
4141
export QUARKUS_DATASOURCE_PASSWORD=postgres
42-
echo ($QUARKUS_DATASOURCE_JDBC_URL)
42+
echo $QUARKUS_DATASOURCE_JDBC_URL
4343

4444
GCS_BUCKET_NAME="polaris-test-gcs-$RANDOM_SUFFIX"
4545
echo "GCS Bucket Name: $GCS_BUCKET_NAME"
@@ -52,4 +52,4 @@ export STORAGE_LOCATION="gs://$GCS_BUCKET_NAME/quickstart_catalog/"
5252
-Dquarkus.container-image.build=true \
5353
--no-build-cache
5454

55-
docker compose -f getting-started/jdbc/docker-compose-bootstrap-db.yml -f getting-started/jdbc/docker-compose.yml up -d
55+
docker compose -p polaris -f getting-started/jdbc/docker-compose-bootstrap-db.yml -f getting-started/jdbc/docker-compose.yml up -d

‎getting-started/spark/docker-compose.yml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,9 @@ services:
3030
POLARIS_BOOTSTRAP_CREDENTIALS: default-realm,root,s3cr3t
3131
polaris.realm-context.realms: default-realm
3232
quarkus.otel.sdk.disabled: "true"
33+
polaris.features."ALLOW_INSECURE_STORAGE_TYPES": "true"
34+
polaris.features."SUPPORTED_CATALOG_STORAGE_TYPES": "[\"FILE\",\"S3\",\"GCS\",\"AZURE\"]"
35+
polaris.readiness.ignore-severe-issues: "true"
3336
healthcheck:
3437
test: ["CMD", "curl", "http://localhost:8182/healthcheck"]
3538
interval: 10s

‎getting-started/spark/notebooks/Dockerfile‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ ENV LANGUAGE='en_US:en'
2424
COPY --chown=jovyan client /home/jovyan/client
2525
COPY --chown=jovyan regtests/requirements.txt /tmp
2626
RUN pip install -r /tmp/requirements.txt && \
27-
cd client/python && \
27+
cd client/python && poetry lock && \
2828
python3 -m poetry install && \
2929
pip install -e .
3030

‎gradle/baselibs.versions.toml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,4 +23,4 @@ idea-ext = { module = "gradle.plugin.org.jetbrains.gradle.plugin.idea-ext:gradle
2323
license-report = { module = "com.github.jk1:gradle-license-report", version = "2.9" }
2424
nexus-publish = { module = "io.github.gradle-nexus:publish-plugin", version = "2.0.0" }
2525
shadow = { module = "com.gradleup.shadow:shadow-gradle-plugin", version = "8.3.6" }
26-
spotless = { module = "com.diffplug.spotless:spotless-plugin-gradle", version = "7.0.3" }
26+
spotless = { module = "com.diffplug.spotless:spotless-plugin-gradle", version = "7.0.4" }

‎gradle/projects.main.properties‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,11 +28,11 @@ polaris-service-common=service/common
2828
polaris-quarkus-defaults=quarkus/defaults
2929
polaris-quarkus-service=quarkus/service
3030
polaris-quarkus-server=quarkus/server
31+
polaris-quarkus-distribution=quarkus/distribution
3132
polaris-quarkus-spark-tests=quarkus/spark-tests
3233
polaris-quarkus-admin=quarkus/admin
3334
polaris-quarkus-common=quarkus/common
3435
polaris-quarkus-test-commons=quarkus/test-commons
35-
polaris-quarkus-run-script=quarkus/run-script
3636
polaris-eclipselink=extension/persistence/eclipselink
3737
polaris-jpa-model=extension/persistence/jpa-model
3838
polaris-relational-jdbc=extension/persistence/relational-jdbc

0 commit comments

Comments
 (0)