Skip to content

Commit 5b14b90

Browse files
dimas-bdimas-b
committed
Merge remote-tracking branch 'apache/main' into persistence-no-sql-0
2 parents 8ee7530 + d841431 commit 5b14b90

File tree

192 files changed

+4255
-1552
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

192 files changed

+4255
-1552
lines changed

‎.github/workflows/gradle.yml‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ jobs:
4242
steps:
4343
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5
4444
- name: Set up JDK 21
45-
uses: actions/setup-java@c5195efecf7bdfc987ee8bae7a71cb8b11521c00 # v4
45+
uses: actions/setup-java@dded0888837ed1f317902acf8a20df0ad188d165 # v5
4646
with:
4747
java-version: '21'
4848
distribution: 'temurin'
@@ -83,7 +83,7 @@ jobs:
8383
steps:
8484
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5
8585
- name: Set up JDK 21
86-
uses: actions/setup-java@c5195efecf7bdfc987ee8bae7a71cb8b11521c00 # v4
86+
uses: actions/setup-java@dded0888837ed1f317902acf8a20df0ad188d165 # v5
8787
with:
8888
java-version: '21'
8989
distribution: 'temurin'
@@ -121,7 +121,7 @@ jobs:
121121
steps:
122122
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5
123123
- name: Set up JDK 21
124-
uses: actions/setup-java@c5195efecf7bdfc987ee8bae7a71cb8b11521c00 # v4
124+
uses: actions/setup-java@dded0888837ed1f317902acf8a20df0ad188d165 # v5
125125
with:
126126
java-version: '21'
127127
distribution: 'temurin'
@@ -159,7 +159,7 @@ jobs:
159159
steps:
160160
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5
161161
- name: Set up JDK 21
162-
uses: actions/setup-java@c5195efecf7bdfc987ee8bae7a71cb8b11521c00 # v4
162+
uses: actions/setup-java@dded0888837ed1f317902acf8a20df0ad188d165 # v5
163163
with:
164164
java-version: '21'
165165
distribution: 'temurin'

‎.github/workflows/helm.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -44,13 +44,13 @@ jobs:
4444
fetch-depth: 0
4545

4646
- name: Set up JDK 21
47-
uses: actions/setup-java@c5195efecf7bdfc987ee8bae7a71cb8b11521c00 # v4
47+
uses: actions/setup-java@dded0888837ed1f317902acf8a20df0ad188d165 # v5
4848
with:
4949
java-version: '21'
5050
distribution: 'temurin'
5151

5252
- name: Set up Helm
53-
uses: azure/setup-helm@b9e51907a09c216f16ebe8536097933489208112 # v4.3.0
53+
uses: azure/setup-helm@1a275c3b69536ee54be43f2070a358922e12c8d4 # v4.3.1
5454
with:
5555
version: 'v3.16.0'
5656

‎.github/workflows/nightly.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ jobs:
4848
- name: Checkout
4949
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5
5050
- name: Set up JDK 21
51-
uses: actions/setup-java@c5195efecf7bdfc987ee8bae7a71cb8b11521c00 # v4
51+
uses: actions/setup-java@dded0888837ed1f317902acf8a20df0ad188d165 # v5
5252
with:
5353
java-version: '21'
5454
distribution: 'temurin'

‎.github/workflows/python-client.yml‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -45,14 +45,11 @@ jobs:
4545
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5
4646

4747
- name: Set up JDK for Gradle
48-
uses: actions/setup-java@c5195efecf7bdfc987ee8bae7a71cb8b11521c00 # v4
48+
uses: actions/setup-java@dded0888837ed1f317902acf8a20df0ad188d165 # v5
4949
with:
5050
distribution: 'temurin'
5151
java-version: '21'
5252

53-
- name: Run regeneratePythonClient
54-
run: ./gradlew regeneratePythonClient
55-
5653
- name: Set up Python ${{ matrix.python-version }}
5754
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
5855
with:
@@ -76,3 +73,7 @@ jobs:
7673
- name: Integration Tests
7774
run: |
7875
make client-integration-test
76+
77+
- name: Run Polaris Client help maual
78+
run: |
79+
./polaris --help

‎.github/workflows/regtest.yml‎

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ jobs:
3535
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5
3636

3737
- name: Set up JDK 21
38-
uses: actions/setup-java@c5195efecf7bdfc987ee8bae7a71cb8b11521c00 # v4
38+
uses: actions/setup-java@dded0888837ed1f317902acf8a20df0ad188d165 # v5
3939
with:
4040
java-version: '21'
4141
distribution: 'temurin'
@@ -51,9 +51,6 @@ jobs:
5151
- name: Fix permissions
5252
run: mkdir -p regtests/output && chmod 777 regtests/output && chmod 777 regtests/t_*/ref/*
5353

54-
- name: Run regeneratePythonClient
55-
run: ./gradlew regeneratePythonClient
56-
5754
- name: Image build
5855
run: |
5956
./gradlew \

‎.github/workflows/spark_client_regtests.yml‎

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ jobs:
3535
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5
3636

3737
- name: Set up JDK 21
38-
uses: actions/setup-java@c5195efecf7bdfc987ee8bae7a71cb8b11521c00 # v4
38+
uses: actions/setup-java@dded0888837ed1f317902acf8a20df0ad188d165 # v5
3939
with:
4040
java-version: '21'
4141
distribution: 'temurin'
@@ -51,9 +51,6 @@ jobs:
5151
- name: Fix permissions
5252
run: mkdir -p regtests/output && chmod 777 regtests/output && chmod 777 regtests/t_*/ref/*
5353

54-
- name: Run regeneratePythonClient
55-
run: ./gradlew regeneratePythonClient
56-
5754
- name: Project build without testing
5855
env:
5956
# publishToMavenLocal causes a GH API requests, use the token for those requests

‎CHANGELOG.md‎

Lines changed: 45 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,18 @@ request adding CHANGELOG notes for breaking (!) changes and possibly other secti
2929

3030
### Highlights
3131

32+
- **HMS Federation Support**: Added support for Hive Metastore (HMS) federation, enabling integration with existing Hive metastores.
33+
34+
- **Modularized Federation**: Introduced modularized federation architecture to support multiple catalog types and improve extensibility.
35+
36+
- **External Authentication**: Added comprehensive support for external identity providers including Keycloak integration and Helm chart configuration options.
37+
38+
- **Python Client Distribution**: The Python client is now packaged and distributed as a proper Python package for easier installation and usage.
39+
40+
- **Catalog Federation CLI**: Extended the CLI with support for managing federated catalogs, making it easier to configure and operate catalog federation.
41+
42+
- **MinIO**: Added MinIO integration support with comprehensive getting started documentation.
43+
3244
### Upgrade notes
3345

3446
### Breaking changes
@@ -39,14 +51,43 @@ request adding CHANGELOG notes for breaking (!) changes and possibly other secti
3951
### New Features
4052

4153
- Added Catalog configuration for S3 and STS endpoints. This also allows using non-AWS S3 implementations.
54+
The realm-level feature flag `ALLOW_SETTING_S3_ENDPOINTS` (default: true) may be used to disable this
55+
functionality.
4256

4357
- The `IMPLICIT` authentication type enables users to create federated catalogs without explicitly
44-
providing authentication parameters to Polaris. When the authentication type is set to `IMPLICIT`,
45-
the authentication parameters are picked from the environment or configuration files.
58+
providing authentication parameters to Polaris. When the authentication type is set to `IMPLICIT`,
59+
the authentication parameters are picked from the environment or configuration files.
4660

4761
- The `DEFAULT_LOCATION_OBJECT_STORAGE_PREFIX_ENABLED` feature was added to support placing tables
4862
at locations that better optimize for object storage.
4963

64+
- The `LIST_PAGINATION_ENABLED` (default: false) feature flag can be used to enable pagination
65+
in the Iceberg REST Catalog API.
66+
67+
- The Helm chart now supports Pod Disruption Budgets (PDBs) for Polaris components. This allows users to define
68+
the minimum number of pods that must be available during voluntary disruptions, such as node maintenance.
69+
70+
- Feature configuration `PURGE_VIEW_METADATA_ON_DROP` was added to allow dropping views without purging their metadata files.
71+
72+
- Introduced S3 path-style access support for improved compatibility with S3-compatible storage systems.
73+
74+
- Enhanced Python client with integration tests and improved error handling.
75+
76+
- Introduced extensible pagination token implementation for better API performance.
77+
78+
- Added support for `s3a` scheme in addition to existing S3 schemes.
79+
80+
- Enhanced Helm chart with support for external authentication configuration and relational JDBC backend options.
81+
82+
- Added comprehensive diagnostics and monitoring capabilities throughout the system.
83+
84+
- Introduced bootstrap command options to specify custom schema files for database initialization.
85+
86+
- Added refresh credentials endpoint configuration to LoadTableResponse for AWS, Azure, and GCP. Enabling
87+
automatic storage credential refresh per table on the client side. Java client version >= 1.8.0 is required.
88+
The endpoint path is always returned when using vended credentials, but clients must enable the
89+
refresh-credentials flag for the desired storage provider.
90+
5091
### Changes
5192

5293
- Polaris Management API clients must be prepared to deal with new attributes in `AwsStorageConfigInfo` objects.
@@ -55,9 +96,8 @@ at locations that better optimize for object storage.
5596

5697
### Deprecations
5798

58-
* The property `polaris.active-roles-provider.type` is deprecated in favor of
59-
`polaris.authentication.active-roles-provider.type`. The old property is still supported, but will be removed in a
60-
future release.
99+
- The property `polaris.active-roles-provider.type` is deprecated for removal.
100+
- The `ActiveRolesProvider` interface is deprecated for removal.
61101

62102
### Fixes
63103

‎Makefile‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -134,7 +134,7 @@ client-lint: client-setup-env ## Run linting checks for Polaris client
134134
.PHONY: client-regenerate
135135
client-regenerate: client-setup-env ## Regenerate the client code
136136
@echo "--- Regenerating client code ---"
137-
@client/templates/regenerate.sh
137+
@$(ACTIVATE_AND_CD) && python3 -B generate_clients.py
138138
@echo "--- Client code regeneration complete ---"
139139

140140
.PHONY: client-unit-test

‎api/iceberg-service/build.gradle.kts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ import org.openapitools.generator.gradle.plugin.tasks.GenerateTask
2222
plugins {
2323
alias(libs.plugins.openapi.generator)
2424
id("polaris-client")
25-
alias(libs.plugins.jandex)
25+
id("org.kordamp.gradle.jandex")
2626
}
2727

2828
dependencies {

‎api/management-model/build.gradle.kts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ import org.openapitools.generator.gradle.plugin.tasks.GenerateTask
2222
plugins {
2323
alias(libs.plugins.openapi.generator)
2424
id("polaris-client")
25-
alias(libs.plugins.jandex)
25+
id("org.kordamp.gradle.jandex")
2626
}
2727

2828
dependencies {

0 commit comments

Comments
 (0)