Dremio merge 2025 11 04 17 29 (apache#154)

* Pass AccessConfig into FileIOFactory (apache#2937)

it should not be the responsibility of the `FileIOFactory` to know how
to infer the `AccessConfig`

* Remove EclipseLink Persistence Backend (apache#2963)

as per ML decision the deprecated eclipselink backend is being removed
in the next version:

https://lists.apache.org/thread/16bj5kngf2kfhqv3noxwfm7h9wlzvhyv

* feat: Improve PolarisAdminTool default output (apache#2961)

* feat: Improve PolarisAdminTool default output

* feat: Improve PolarisAdminTool default output

* Add Polaris Community Meeting 2025-10-30 (apache#2973)

* Remove legacy management endpoints (apache#2276)

* Build/nit: cache output of `generatedMarkdownDocs` (apache#2967)

`(Java)Exec` tasks are not cacheable by default, as annotated with `@DisableCachingByDefault`.
Adding an `outputs.cacheIf { true }` enables caching on those tasks.

* Build: Helper to get effective ASF project metadata (apache#2969)

This change "bundles" the information of `AsfProject` and the `PublishingHelperExtensions`, which is what the code in `configurePom.kt` did. Bundling these objects allows other consumers, like CycloneDX SBOM generation, to access that same information without having to query remote systems (whimsey.apache.org) again.

* Alternative, concise PR template (apache#2945)

This PR proposes an alternative PR template that is much shorter, and removes all the redundant claims.

It also links to the contribution guidelines for further guidance.

* Add docs how to add `Server` header to HTTP responses (apache#2941)

* Prefer PolarisPrincipal over SecurityContext (apache#2932)

The general idea is that `SecurityContext` comes from `jakarta.ws.rs` and
there is no reason for non-REST related classes to rely on those details.
Instead, once preprocessing of a REST-request has inferred the
`PolarisPrincipal` all inner/core code should rely on only that.

Note that this simplifies a bunch of tests that had to create their own
`SecurityContext` around the principal that they wanted to use, thus
having to decide how to implement `isUserInRole` and the other methods.

* Last merged commit f934443

---------

Co-authored-by: Christopher Lambert <xn137@gmx.de>
Co-authored-by: Yong Zheng <yongzheng0809@gmail.com>
Co-authored-by: JB Onofré <jbonofre@apache.org>
Co-authored-by: Alexandre Dutra <adutra@apache.org>
Co-authored-by: fivetran-arunsuri <103934371+fivetran-arunsuri@users.noreply.github.com>

Author: 6 people

.github/pull_request_template.md

@@ -1,65 +1,12 @@
 <!--
-    Possible security vulnerabilities: STOP here and contact security@apache.org instead!
-
-    Please update the title of the PR with a meaningful message - do not leave it "empty" or "generated"
-    Please update this summary field:
-
-    The summary should cover these topics, if applicable:
-    * the motivation for the change
-    * a description of the status quo, for example the current behavior
-    * the desired behavior
-    * etc
-
-    PR checklist:
-    - Do a self-review of your code before opening a pull request
-    - Make sure that there's good test coverage for the changes included in this PR
-    - Run tests locally before pushing a PR (./gradlew check)
-    - Code should have comments where applicable. Particularly hard-to-understand
-      areas deserve good in-line documentation.
-    - Include changes and enhancements to the documentation (in site/content/in-dev/unreleased)
-    - For Work In Progress Pull Requests, please use the Draft PR feature.
-
-    Make sure to add the information BELOW this comment.
-    Everything in this comment will NOT be added to the PR description.
--->
-
-### What changes were proposed in this pull request?
-<!--
-Please clarify what changes you are proposing. The purpose of this section is to outline the changes and how this PR fixes the issue. 
-If possible, please consider writing useful notes for better and faster reviews in your PR. See the examples below.
-  1. If you refactor some codes with changing classes, showing the class hierarchy will help reviewers.
-  2. If you fix some IRC features, you can provide some references of other IRC implementations / IRC spec.
-  3. If there is design documentation, please add the link.
-  4. If there is a discussion in the mailing list, please add the link.
--->
-
-
-### Why are the changes needed?
-<!--
-Please clarify why the changes are needed. For instance,
-  1. If you propose a new API, clarify the use case for a new API.
-  2. If you fix a bug, you can clarify why it is a bug.
+📝 Describe what changes you're proposing, especially breaking or user-facing changes. 
+📖 See https://github.com/apache/polaris/blob/main/CONTRIBUTING.md for more.
 -->
 
-
-### Does this PR introduce _any_ user-facing change?
-<!--
-Note that it means *any* user-facing change including all aspects such as new features, bug fixes, or other behavior changes. Documentation-only updates are not considered user-facing changes.
-
-If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description and/or an example to show the behavior difference if possible.
-If possible, please also clarify if this is a user-facing change compared to the released Polaris versions or within the unreleased branches such as master.
-If no, write 'No'.
--->
-
-
-### How was this patch tested?
-<!--
-If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
-If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
-If tests were not added, please describe why they were not added and/or why it was difficult to add.
--->
-
-### CHANGELOG.md
-<!--
-If the changes need to be included in CHANGELOG.md, please add a line here and in CHANGELOG.md.
--->
+## Checklist
+- [ ] 🛡️ Don't disclose security issues! (contact security@apache.org)
+- [ ] 🔗 Clearly explained why the changes are needed, or linked related issues: Fixes #
+- [ ] 🧪 Added/updated tests with good coverage, or manually tested (and explained how)
+- [ ] 💡 Added comments for complex logic
+- [ ] 🧾 Updated `CHANGELOG.md` (if needed)
+- [ ] 📚 Updated documentation in `site/content/in-dev/unreleased` (if needed)

CHANGELOG.md

@@ -31,8 +31,13 @@ request adding CHANGELOG notes for breaking (!) changes and possibly other secti
 
 ### Upgrade notes
 
+- The legacy management endpoints at `/metrics` and `/healthcheck` have been removed. Please use the
+  standard management endpoints at `/q/metrics` and `/q/health` instead.
+
 ### Breaking changes
 
+- The EclipseLink Persistence implementation has been completely removed.
+
 ### New Features
 
 - Support credential vending for federated catalogs. `ALLOW_FEDERATED_CATALOGS_CREDENTIAL_VENDING` (default: true) was added to toggle this feature.

README.md

@@ -61,7 +61,6 @@ Apache Polaris is organized into the following modules:
       - `polaris-runtime-spark-tests` - Integration tests for the Polaris Spark plugin
       - `polaris-runtime-test-common` - Test utilities
   - Persistence modules:
-      - `polaris-eclipselink` - The Eclipselink implementation of the MetaStoreManager interface (deprecated for removal)
       - `polaris-relational-jdbc` - The JDBC implementation of BasePersistence to be used via AtomicMetaStoreManager
   - Extensions modules:
       - `polaris-extensions-federation-hadoop` - The Hadoop federation extension

bom/build.gradle.kts

@@ -85,7 +85,6 @@ dependencies {
 
     api(project(":polaris-core"))
 
-    api(project(":polaris-eclipselink"))
     api(project(":polaris-relational-jdbc"))
 
     api(project(":polaris-extensions-auth-opa"))

build-logic/src/main/kotlin/publishing/PublishingHelperExtension.kt

@@ -19,10 +19,13 @@
 
 package publishing
 
+import asf.AsfProject
 import java.io.File
 import javax.inject.Inject
 import org.gradle.api.Project
 import org.gradle.api.model.ObjectFactory
+import org.gradle.api.provider.Provider
+import org.gradle.internal.extensions.stdlib.capitalized
 import org.gradle.kotlin.dsl.property
 
 /**
@@ -83,3 +86,57 @@ constructor(objectFactory: ObjectFactory, project: Project) {
   fun distributionFile(ext: String): File =
     distributionDir.get().file("${baseName.get()}.$ext").asFile
 }
+
+/**
+ * Provides the effective values for an ASF project using the ASF project metadata from `AsfProject`
+ * with overrides from `PublishingHelperExtension`.
+ */
+class EffectiveAsfProject(
+  val publishingHelperExtension: PublishingHelperExtension,
+  val asfProject: AsfProject,
+) {
+  fun githubRepoName(): Provider<String> =
+    publishingHelperExtension.githubRepositoryName.orElse(asfProject.apacheId)
+
+  fun tagPrefix(): Provider<String> =
+    publishingHelperExtension.overrideTagPrefix.orElse("apache-${asfProject.apacheId}")
+
+  fun issueTracker(): Provider<String> {
+    val issuesUrl: Provider<String> =
+      codeRepoUrl().map { r -> "$r/issues" }.orElse(asfProject.bugDatabase)
+    return publishingHelperExtension.overrideIssueManagement.orElse(issuesUrl)
+  }
+
+  fun codeRepoUrl(): Provider<String> =
+    publishingHelperExtension.overrideScm.orElse(
+      githubRepoName().map { r -> "https://github.com/apache/$r" }.orElse(asfProject.repository)
+    )
+
+  fun fullName() = publishingHelperExtension.overrideName.orElse("Apache ${asfProject.name}")
+
+  fun description() = publishingHelperExtension.overrideDescription.orElse(asfProject.description)
+
+  fun projectUrl() = publishingHelperExtension.overrideProjectUrl.orElse(asfProject.website)
+
+  fun mailingList(id: String) = MailingList(id, asfProject.apacheId)
+
+  companion object {
+    fun forProject(project: Project): EffectiveAsfProject {
+      val e = project.extensions.getByType(PublishingHelperExtension::class.java)
+      val asfProject = AsfProject.memoized(project, e.asfProjectId.get())
+      return EffectiveAsfProject(e, asfProject)
+    }
+  }
+}
+
+class MailingList(val id: String, private val prj: String) {
+  fun name(): String = "${id.capitalized()} Mailing List"
+
+  fun subscribe(): String = "$id-subscribe@$prj.apache.org"
+
+  fun unsubscribe(): String = "$id-unsubscribe@$prj.apache.org"
+
+  fun post(): String = "$id@$prj.apache.org"
+
+  fun archive(): String = "https://lists.apache.org/list.html?$id@$prj.apache.org"
+}

build-logic/src/main/kotlin/publishing/configurePom.kt

@@ -19,14 +19,11 @@
 
 package publishing
 
-import asf.AsfProject
 import groovy.util.Node
 import org.gradle.api.Project
 import org.gradle.api.Task
 import org.gradle.api.artifacts.component.ModuleComponentSelector
-import org.gradle.api.provider.Provider
 import org.gradle.api.publish.maven.MavenPublication
-import org.gradle.internal.extensions.stdlib.capitalized
 
 /**
  * Configures the content of the generated `pom.xml` files.
@@ -47,8 +44,6 @@ import org.gradle.internal.extensions.stdlib.capitalized
  */
 internal fun configurePom(project: Project, mavenPublication: MavenPublication, task: Task) =
   mavenPublication.run {
-    val e = project.extensions.getByType(PublishingHelperExtension::class.java)
-
     pom {
       if (project != project.rootProject) {
         // Add the license to every pom to make it easier for downstream projects to retrieve the
@@ -75,8 +70,8 @@ internal fun configurePom(project: Project, mavenPublication: MavenPublication,
 
         task.doFirst {
           mavenPom.run {
-            val asfProject = AsfProject.memoized(project, e.asfProjectId.get())
-            val asfProjectId = asfProject.apacheId
+            val prj = EffectiveAsfProject.forProject(project)
+            val asfProjectId = prj.asfProject.apacheId
 
             organization {
               name.set("The Apache Software Foundation")
@@ -85,51 +80,41 @@ internal fun configurePom(project: Project, mavenPublication: MavenPublication,
             licenses {
               license {
                 name.set("Apache-2.0") // SPDX identifier
-                url.set(asfProject.licenseUrl)
+                url.set(prj.asfProject.licenseUrl)
               }
             }
             mailingLists {
-              e.mailingLists.get().forEach { ml ->
-                mailingList {
-                  name.set("${ml.capitalized()} Mailing List")
-                  subscribe.set("$ml-subscribe@$asfProjectId.apache.org")
-                  unsubscribe.set("$ml-unsubscribe@$asfProjectId.apache.org")
-                  post.set("$ml@$asfProjectId.apache.org")
-                  archive.set("https://lists.apache.org/list.html?$ml@$asfProjectId.apache.org")
+              prj.publishingHelperExtension.mailingLists
+                .get()
+                .map { id -> prj.mailingList(id) }
+                .forEach { ml ->
+                  mailingList {
+                    name.set(ml.name())
+                    subscribe.set(ml.subscribe())
+                    unsubscribe.set(ml.unsubscribe())
+                    post.set(ml.post())
+                    archive.set(ml.archive())
+                  }
                 }
-              }
             }
 
-            val githubRepoName: Provider<String> = e.githubRepositoryName.orElse(asfProjectId)
-            val codeRepo: Provider<String> =
-              e.overrideScm.orElse(
-                githubRepoName
-                  .map { r -> "https://github.com/apache/$r" }
-                  .orElse(asfProject.repository)
-              )
-
             scm {
-              val codeRepoString: String = codeRepo.get()
+              val codeRepoString: String = prj.codeRepoUrl().get()
               connection.set("scm:git:$codeRepoString")
               developerConnection.set("scm:git:$codeRepoString")
               url.set("$codeRepoString/tree/main")
               val version = project.version.toString()
               if (!version.endsWith("-SNAPSHOT")) {
-                val tagPrefix: String =
-                  e.overrideTagPrefix.orElse("apache-${asfProject.apacheId}").get()
+                val tagPrefix: String = prj.tagPrefix().get()
                 tag.set("$tagPrefix-$version")
               }
             }
-            issueManagement {
-              val issuesUrl: Provider<String> =
-                codeRepo.map { r -> "$r/issues" }.orElse(asfProject.bugDatabase)
-              url.set(e.overrideIssueManagement.orElse(issuesUrl))
-            }
+            issueManagement { url.set(prj.issueTracker()) }
 
-            name.set(e.overrideName.orElse("Apache ${asfProject.name}"))
-            description.set(e.overrideDescription.orElse(asfProject.description))
-            url.set(e.overrideProjectUrl.orElse(asfProject.website))
-            inceptionYear.set(asfProject.inceptionYear.toString())
+            name.set(prj.fullName())
+            description.set(prj.description())
+            url.set(prj.projectUrl())
+            inceptionYear.set(prj.asfProject.inceptionYear.toString())
 
             developers { developer { url.set("https://$asfProjectId.apache.org/community/") } }
           }

getting-started/README.md

@@ -36,10 +36,5 @@ this directory. Each example has detailed instructions.
 - [Telemetry](telemetry): An example that includes Prometheus and Jaeger to collect metrics and
   traces from Apache Polaris. This example automatically creates a `polaris_demo` catalog.
 
-- [Eclipselink](eclipselink): An example that uses an Eclipselink metastore and a Postgres
-  database. The realm is bootstrapped with the Polaris Admin tool. This example also creates a
-  `polaris_quickstart` catalog, and offers the ability to run Spark SQL and Trino queries. Finally, it shows how to
-  attach a debugger to the Polaris server.
-
 - [Keycloak](keycloak): An example that uses Keycloak as an external identity provider (IDP) for
-  authentication.
+  authentication.

getting-started/assets/cloud_providers/deploy-azure.sh

@@ -61,7 +61,7 @@ az role assignment create \
 export AZURE_TENANT_ID=$(az account show --query tenantId -o tsv)
 export STORAGE_LOCATION="abfss://$STORAGE_CONTAINER_NAME@$STORAGE_ACCOUNT_NAME.dfs.core.windows.net/quickstart_catalog"
 
-cat >> getting-started/eclipselink/trino-config/catalog/iceberg.properties << EOF
+cat >> getting-started/assets/trino-config/catalog/iceberg.properties << EOF
 fs.native-azure.enabled=true
 azure.auth-type=DEFAULT
 EOF