Commit f866828
Dremio merge 2025 09 08 13 38 (apache#110)
* Integration tests for Catalog Federation (apache#2344)
Adds a Junit5 integration test for catalog federation.
* Fix merge conflict in CatalogFederationIntegrationTest (apache#2420)
apache#2344 added a new test for catalog federation, but it looks like an undetected conflict with concurrent changes related to authentication have broken the test in main.
* chore(deps): update registry.access.redhat.com/ubi9/openjdk-21-runtime docker tag to v1.23-6.1755674729 (apache#2416)
* 2334 (apache#2427)
* Fix TableIdentifier in TaskFileIOSupplier (apache#2304)
we cant just convert a `TaskEntity` to a `IcebergTableLikeEntity` as the
`getTableIdentifier()` method will not return a correct value by using
the name of the task and its parent namespace (which is empty?).
task handlers instead need to pass in the `TableIdentifier` that they
already inferred via `TaskEntity.readData`.
* Fix NPE in CreateCatalog (apache#2435)
* Doc fix: Access control page update (apache#2424)
* 2418
* 2418
* fix(deps): update dependency software.amazon.awssdk:bom to v2.32.29 (apache#2443)
* Optimize PolicyCatalog.listPolicies (apache#2370)
this is a follow-up to apache#2290
the optimization is to use `listEntities` instead of `loadEntities` when
there is `policyType` filter to apply
* Add PolarisDiagnostics field to BaseMetaStoreManager (apache#2381)
* Add PolarisDiagnostics field to BaseMetaStoreManager
the ultimate goal is removing the `PolarisCallContext` parameter from every
`PolarisMetaStoreManager` interface method, so we make steps towards
reducing its usage first.
* Add feature flag to disallow custom S3 endpoints (apache#2442)
* Add new realm-level flag: `ALLOW_SETTING_S3_ENDPOINTS` (default: true)
* Enforce in `PolarisServiceImpl.validateStorageConfig()`
Fixes apache#2436
* Deprecate ActiveRolesProvider for removal (apache#2404)
* Client: fix openapi verbose output, remove doc generate, and skip test generations (apache#2439)
* Fix various issue in client code generation
* Use logger instead of print
* Add back exclude on __pycache__ as CI is not via Makefile
* Add back exclude on __pycache__ as CI is not via Makefile
* Add user principal tag in metrics (apache#2445)
* Added API change to enable tag
* Added test
* Added production readiness check
* fix(deps): update dependency io.opentelemetry.semconv:opentelemetry-semconv to v1.36.0 (apache#2454)
* fix(deps): update dependency com.google.cloud:google-cloud-storage-bom to v2.56.0 (apache#2447)
* fix(deps): update dependency gradle.plugin.org.jetbrains.gradle.plugin.idea-ext:gradle-idea-ext to v1.3 (apache#2428)
* Build: Make jandex dependency used for index generation managed (apache#2431)
Also allows specifying the jandex index version for the build.
This is a preparation step contributing to apache#2204, once a jandex fix for reproducible builds is available.
Co-authored-by: Alexandre Dutra <adutra@apache.org>
* Built: improve reproducible archive files (apache#2432)
As part of the effort for apache#2204, this change fixes a few aspects around reproducible builds:
Some Gradle projects produce archive files, but don't get the necessary Gradle archive-tasks settings applied: one not-published project but also the tarball&zip of the distribution. This change moves the logic to the new build-plugin `polaris-reproducible`.
Another change is to have some Quarkus generated jar files adhere to the same conventions, which are constant timestamps for the zip entries and a deterministic order of the entries. That's sadly not a full fix, as the classes that are generated or instumented by Quarkus differ in each build.
Contributes to apache#2204
* Remove commons-lang3 dependency (apache#2456)
outside of tests we can replace the functionality with jdk11 and guava.
also stop using `org.assertj.core.util` as its a non-public api.
* add refresh credentials property to loadTableResult (apache#2341)
* add refresh credentials property to loadTableResult
* IcebergCatalogAdapterTest: Added test to ensure refresh credentials endpoint is included
* delegate refresh credential endpoint configuration to storage integration
* GCP: Add refresh credential properties
* fix(deps): update dependency io.opentelemetry.semconv:opentelemetry-semconv to v1.37.0 (apache#2458)
* Add Delegator to all API Implementations (apache#2434)
Per the Dev ML, implements the Delegator pattern to add Events instrumentation to all Polaris APIs.
* Prefer java.util.Base64 over commons-codec (apache#2463)
`java.util.Base64` is available since java8 and we are already using it
in a few other spots.
in a follow-up we might be able to get rid of our `commons-codec` dependency
completely.
* Service: Move tests to the right package (apache#2469)
* Update versions in runtime LICENSE and NOTICE (apache#2468)
* fix(deps): update dependency com.adobe.testing:s3mock-testcontainers to v4.8.0 (apache#2475)
* fix(deps): update dependency com.gradleup.shadow:shadow-gradle-plugin to v9.1.0 (apache#2476)
* Service: Remove hadoop-common from polaris-runtime-service (apache#2462)
* Service: Always validate allowed locations from Storage Config (apache#2473)
* Add Community Sync Meeting 20250828 (apache#2477)
* Update dependency software.amazon.awssdk:bom to v2.33.0 (apache#2483)
* Remove PolarisCallContext.getDiagServices (apache#2415)
* Remove PolarisCallContext.getDiagServices usage
* Remove diagnostics from PolarisCallContext
* Feature: Expose resetCredentials via a new reset api to allow root user to reset credentials for an existing principal with custom values (apache#2197)
* Add type-check to PolarisEntity subclass ctors (apache#2302)
currently one can freely "cast" any `PolarisEntity` to a more
specific type via their constructors.
this can lead to subtle bugs like we fixed in
a29f800
by adding type checks we discover a few more places where we need to be
more careful about how we construct new or handle existing entities.
note that we can add a check for `PolarisEntitySubType` in a followup,
but it requires more fixes currently.
* Fix CI (apache#2489)
Fix undetected merge conflict after apache#2197 + apache#2415 + apache#2434
* Use local diagnostics in TransactionWorkspaceMetaStoreManager
* Add resetCredentials to PolarisPrincipalsEventServiceDelegator
* Core: Prevent AIOOBE for negative codes in PolarisEntityType, PolarisPrivilege, ReturnStatus (apache#2490)
* feat(idgen): Start Implementation of NoSQL with the ID Generation Framework (apache#2131)
Create an ID Generation Framework.
Related to apache#650 & apache#844
Co-authored-by: Robert Stupp <snazy@snazy.de>
Co-authored-by: Dmitri Bourlatchkov <dmitri.bourlatchkov@gmail.com>
* perf(refactor): optimizing JdbcBasePersistenceImpl.listEntities (apache#2465)
- Reduced Column Selection: Only 6 columns instead of 16
- Eliminated Object Creation Overhead: Direct conversion to EntityNameLookupRecord without intermediate PolarisBaseEntity
* Add Polaris Events to Persistence (apache#1844)
* AWS CloudWatch Event Sink Implementation (apache#1965)
* Fix failing CI (apache#2498)
* Update actions/stale digest to 3a9db7e (apache#2499)
* Core: Prevent AIOOBE for negative policy codes in PredefinedPolicyType (apache#2486)
* Service: Add location tests for views (apache#2496)
* Update docker.io/jaegertracing/all-in-one Docker tag to v1.73.0 (apache#2500)
* Update dependency io.netty:netty-codec-http2 to v4.2.5.Final (apache#2495)
* Update actions/setup-python action to v6 (apache#2502)
* Update the Release Guide about the Helm Chart package (apache#2179)
* Update the Release Guide about the Helm Chart package
* Update release-guide.md
Co-authored-by: Pierre Laporte <pierre@pingtimeout.fr>
* Add missing commit message
* Whitespace
* Use Helm GPG plugin to sign the Helm chart
* Fix directories during Helm chart copy to SVN
* Add Helm index to SVN
* Use long name for svn checkout
* Ensure the Helm index is updated after the chart is moved to SVN dist release
* Do not publish any Docker image before the vote succeeds
* Typos
* Revert "Do not publish any Docker image before the vote succeeds"
This reverts commit 5617e65.
* Don't mention Helm values.yaml in the release guide as it doesn't contain version details
---------
Co-authored-by: Pierre Laporte <pierre@pingtimeout.fr>
* Update dependency com.azure:azure-sdk-bom to v1.2.38 (apache#2503)
* Update registry.access.redhat.com/ubi9/openjdk-21-runtime Docker tag to v1.23-6.1756793420 (apache#2504)
* Remove commons-codec dependency (apache#2474)
follow-up to f8ad77a
we can simply use guava instead and eliminate the extra dependency
* CLI: Remove SCRIPT_DIR and default config location to user home (apache#2448)
* Remove readInternalProperties helpers (apache#2506)
the functionality is already provided by the `PrincipalEntity`
* Add Events for Generic Table APIs (apache#2481)
This PR adds the Events instrumentation for the Generic Tables Service APIs, surrounding the default delegated call to the business logic APIs.
* Disable custom namespace locations (apache#2422)
When we create a namespace or alter its location, we must confirm that this location is within the parent location. This PR introduces introduces a check similar to the one we have for tables, where custom locations are prohibited by default. This functionality is gated behind a new behavior change flag `ALLOW_NAMESPACE_CUSTOM_LOCATION`. In addition to allowing us to revert to the old behavior, this flag allows some tests relying on arbitrarily-located namespaces to pass (such as those from upstream Iceberg).
Fixes: apache#2417
* fix for IcebergAllowedLocationTest (apache#2511)
* Remove unused config from SparkSessionBuilder (apache#2512)
Tests pass without it.
* Add Events for Policy Service APIs (apache#2479)
* Remove PolarisTestMetaStoreManager.jsonNode helper (apache#2513)
* Update dependency software.amazon.awssdk:bom to v2.33.4 (apache#2517)
* Update dependency com.nimbusds:nimbus-jose-jwt to v10.5 (apache#2514)
* Update dependency io.opentelemetry:opentelemetry-bom to v1.54.0 (apache#2515)
* Update dependency io.micrometer:micrometer-bom to v1.15.4 (apache#2519)
* Port missed OSS change
* NoSQL: adopt to updated test packages
* NoSQL: adapt to removed PolarisDiagnostics param
* NoSQL: fix libs.versions.toml
* NoSQL: include jandex plugin related changes from OSS
* NoSQL: changes for delete/set principal client-ID+secret
* Last merged commit c6176dc
---------
Co-authored-by: Pooja Nilangekar <poojan@umd.edu>
Co-authored-by: Eric Maynard <eric.maynard+oss@snowflake.com>
Co-authored-by: Mend Renovate <bot@renovateapp.com>
Co-authored-by: Yong Zheng <yongzheng0809@gmail.com>
Co-authored-by: Christopher Lambert <xn137@gmx.de>
Co-authored-by: Honah (Jonas) J. <honahx@apache.org>
Co-authored-by: Dmitri Bourlatchkov <dmitri.bourlatchkov@gmail.com>
Co-authored-by: Alexandre Dutra <adutra@apache.org>
Co-authored-by: fivetran-kostaszoumpatianos <kostas.zoumpatianos@fivetran.com>
Co-authored-by: Jason <jasonf20@gmail.com>
Co-authored-by: Adnan Hemani <adnan.h@berkeley.edu>
Co-authored-by: Yufei Gu <yufei@apache.org>
Co-authored-by: JB Onofré <jbonofre@apache.org>
Co-authored-by: fivetran-arunsuri <103934371+fivetran-arunsuri@users.noreply.github.com>
Co-authored-by: Adam Christian <105929021+adam-christian-software@users.noreply.github.com>
Co-authored-by: Artur Rakhmatulin <artur.rakhmatulin@gmail.com>
Co-authored-by: Pierre Laporte <pierre@pingtimeout.fr>1 parent ba9fa4b commit f866828
File tree
327 files changed
+8280
-1773
lines changed- .github/workflows
- api
- iceberg-service
- management-model
- management-service
- polaris-catalog-service
- build-logic
- src/main/kotlin
- publishing
- client/python
- cli
- command
- docs
- extensions/federation
- hadoop
- hive
- getting-started/telemetry
- gradle
- integration-tests/src/main/java/org/apache/polaris/service/it
- env
- ext
- test
- persistence
- eclipselink
- src
- main/java/org/apache/polaris/extension/persistence/impl/eclipselink
- test/java/org/apache/polaris/extension/persistence/impl/eclipselink
- nosql
- async
- api
- java
- vertx
- authz
- api
- impl
- spi
- store-nosql
- idgen
- api
- impl
- mocks
- spi
- nodes
- api
- impl
- spi
- store-nosql
- persistence
- api
- cdi
- common
- quarkus-distcache
- quarkus
- weld
- correctness
- db
- inmemory
- mongodb
- impl
- maintenance
- api
- impl
- retain-cel
- spi
- metastore
- src
- main/java/org/apache/polaris/persistence/nosql/metastore
- test/java/org/apache/polaris/persistence/nosql/metastore
- testextension
- types
- src
- main/java/org/apache/polaris/persistence/nosql/coretypes/principals
- test/java/org/apache/polaris/persistence/nosql/coretypes/maintenance
- realms
- api
- impl
- spi
- store-nosql
- relational-jdbc
- src
- main
- java/org/apache/polaris/persistence/relational/jdbc
- models
- resources
- h2
- postgres
- test/java/org/apache/polaris/persistence/relational/jdbc
- models
- plugins/spark/v3.5/integration
- src/intTest/java/org/apache/polaris/spark/quarkus/it
- polaris-core
- src
- main/java/org/apache/polaris/core
- auth
- config
- entity
- table
- persistence
- cache
- dao/entity
- pagination
- resolver
- transactional
- policy
- rest
- secrets
- storage
- aws
- azure
- cache
- gcp
- testFixtures/java/org/apache/polaris/core/persistence
- test/java/org/apache/polaris
- core
- entity
- persistence
- cache
- dao/entity
- policy
- storage
- aws
- azure
- cache
- service/storage
- aws
- azure
- gcp
- regtests/t_cli/src
- runtime
- admin
- src/main
- docker
- java/org/apache/polaris/admintool
- common
- defaults
- src/main/resources
- distribution
- server
- src/main/docker
- service
- src
- intTest/java/org/apache/polaris/service/it
- main/java/org/apache/polaris/service
- admin
- auth
- catalog
- common
- generic
- iceberg
- io
- policy
- config
- context
- catalog
- events
- jsonEventListener
- aws/cloudwatch
- listeners
- exception
- http
- metrics
- persistence
- quarkus/events/jsonEventListener/aws/cloudwatch
- ratelimiter
- storage
- task
- testFixtures/java/org/apache/polaris/service
- test
- java/org/apache/polaris/service
- admin
- auth
- catalog
- generic
- iceberg
- io
- policy
- distcache
- events
- jsonEventListener/aws/cloudwatch
- listeners
- it
- metrics
- ratelimiter
- task
- test
- resources/org/apache/polaris/service/events/jsonEventListener/aws/cloudwatch
- spark-tests
- src/intTest/java/org/apache/polaris/service/spark/it
- test-common
- server-templates
- site/content
- community/meetings
- in-dev/unreleased
- spec
- tools
- config-docs/site
- container-spec-helper
- minio-testcontainer
- misc-types
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
327 files changed
+8280
-1773
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
51 | 51 | | |
52 | 52 | | |
53 | 53 | | |
54 | | - | |
| 54 | + | |
55 | 55 | | |
56 | 56 | | |
57 | 57 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
25 | | - | |
| 25 | + | |
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
51 | 51 | | |
52 | 52 | | |
53 | 53 | | |
| 54 | + | |
| 55 | + | |
54 | 56 | | |
55 | 57 | | |
56 | 58 | | |
| |||
81 | 83 | | |
82 | 84 | | |
83 | 85 | | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
84 | 93 | | |
85 | 94 | | |
86 | 95 | | |
| |||
89 | 98 | | |
90 | 99 | | |
91 | 100 | | |
92 | | - | |
93 | | - | |
94 | | - | |
| 101 | + | |
| 102 | + | |
95 | 103 | | |
96 | 104 | | |
97 | 105 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
134 | 134 | | |
135 | 135 | | |
136 | 136 | | |
137 | | - | |
| 137 | + | |
138 | 138 | | |
139 | 139 | | |
140 | 140 | | |
141 | 141 | | |
142 | 142 | | |
143 | | - | |
| 143 | + | |
144 | 144 | | |
145 | 145 | | |
146 | 146 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
25 | | - | |
| 25 | + | |
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
25 | | - | |
| 25 | + | |
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
25 | | - | |
| 25 | + | |
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
25 | | - | |
| 25 | + | |
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
| 26 | + | |
26 | 27 | | |
27 | 28 | | |
28 | 29 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
17 | 22 | | |
18 | 23 | | |
19 | 24 | | |
| |||
61 | 66 | | |
62 | 67 | | |
63 | 68 | | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
0 commit comments