Skip to content

Conversation

@singhpk234
Copy link
Contributor

@singhpk234 singhpk234 commented May 28, 2025

About the change

Eclipse-link production readiness check should only run when the configured persistence type is eclipse-link but it turns out it doesn't and it creates a noise and confusion like #1684

Also add checks for JDBC production readiness

Before the change

➜  ~ docker logs jdbc-polaris-1 | grep "eclip"
INFO exec -a "java" java -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:5005 -XX:MaxRAMPercentage=80.0 -XX:+UseParallelGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -XX:+ExitOnOutOfMemoryError -cp "." -jar /deployments/quarkus-run.jar
INFO running in /deployments
2025-05-27 17:09:25,492 WARN  [org.apa.pol.ser.qua.con.ProductionReadinessChecks] [,] [,,,] (main) - :warning: The current persistence unit (jdbc:h2) is intended for tests only. Offending configuration option: 'polaris.persistence.eclipselink.configuration-file'.

After the change

➜  ~ docker logs jdbc-polaris-1 | grep "eclip"
INFO exec -a "java" java -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:5005 -XX:MaxRAMPercentage=80.0 -XX:+UseParallelGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -XX:+ExitOnOutOfMemoryError -cp "." -jar /deployments/quarkus-run.jar
INFO running in /deployments

eric-maynard
eric-maynard previously approved these changes May 28, 2025
@github-project-automation github-project-automation bot moved this from PRs In Progress to Ready to merge in Basic Kanban Board May 28, 2025
Copy link
Contributor

@flyrain flyrain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with a minor comment.

@singhpk234 singhpk234 force-pushed the fix/remove-production-readinees-check branch from a36bb1f to 6fc339f Compare May 29, 2025 00:25
@singhpk234 singhpk234 changed the title Fix Production readiness for Eclipselink Fix Production readiness for Persistence May 29, 2025
@singhpk234 singhpk234 force-pushed the fix/remove-production-readinees-check branch from 6fc339f to aa8c2d7 Compare May 29, 2025 02:51
EclipseLinkConfiguration eclipseLinkConfiguration) {
// This check should only be applicable when persistence uses EclipseLink.
MetaStoreManagerFactory metaStoreManagerFactory =
metaStoreManagerFactories.select(Identifier.Literal.of(persistenceType)).get();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we have to .select() here? The usual producer should normally yield a value if we just declared MetaStoreManagerFactory for injection, I think.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is because both @Inject as a member of this class as well as using @Any Instance and then doing .get() letter was giving ambigous bean, I wonder if these checks run before the bean is actually created from QuarkusProducers, this is the only approach worked for me.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm, this WFM :)

  @Produces
  public ProductionReadinessCheck checkJdbcUrl(EclipseLinkConfiguration eclipseLinkConfiguration,
  MetaStoreManagerFactory f) {
    System.out.println("AAA: " + f.getClass());
[...]

Output: AAA: class org.apache.polaris.service.persistence.InMemoryPolarisMetaStoreManagerFactory_ClientProxy

Copy link
Contributor Author

@singhpk234 singhpk234 May 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤦‍♂️ I didn't try just passing it in the parameter without an Annotation. i though like anyother producer it would need one, I can confirm your suggestion works, though any explanation why even with @Produces annotation we don;t need any annotation ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quarkus CDI is able to figure out bean dependencies. When a producer method is needed for a bean, its inputs are automatically injected, I suppose.

return ProductionReadinessCheck.OK;
}

if (jdbcUrl.isPresent() && jdbcUrl.get().startsWith("jdbc:h2")) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be more reliable, plus it's in sync with actual persistence usage:

The EclipseLink side is good to merge, IMHO, so if you prefer we could move this to a separate PR.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would not mind adding this: JdbcMetaStoreManagerFactory.getDatabaseType()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds good, tbh i think the h2 check is fine, though one thing i realized doing this will also check the connection is properly made to the DB and if its not then we simply fail early that your DB is misconfigured correct it !

let me make the change should be small IMHO

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point about connections. Although, I would not fail the startup in that case, maybe just produce a warning check?

Copy link
Contributor

@dimas-b dimas-b May 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, connection checks should be part of the "readiness" check.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, though i prefer failing as without persistence we can't use the application at all, let me know if you feel strongly otherwise.

@singhpk234 singhpk234 force-pushed the fix/remove-production-readinees-check branch from 3f3e083 to 1b4b38e Compare May 29, 2025 17:30
@singhpk234 singhpk234 force-pushed the fix/remove-production-readinees-check branch from 1b4b38e to e6f675a Compare May 29, 2025 17:36
Copy link
Contributor

@dimas-b dimas-b left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍 Thanks for bearing with me, @singhpk234 :)

@singhpk234 singhpk234 merged commit 81798f1 into apache:main May 30, 2025
9 checks passed
@github-project-automation github-project-automation bot moved this from Ready to merge to Done in Basic Kanban Board May 30, 2025
@singhpk234
Copy link
Contributor Author

Thank you so much for review everyone !

snazy added a commit to snazy/polaris that referenced this pull request Jun 13, 2025
* main: Update dependency org.postgresql:postgresql to v42.7.6 (apache#1697)

* main: Update helm/chart-testing-action action to v2.7.0 (apache#1700)

* main: Update gradle/actions digest to 8379f6a (apache#1696)

* main: Update medyagh/setup-minikube action to v0.0.19 (apache#1698)

* feat(metrics): Mitigate potential performance issues with realm_id tag (apache#1662)

As discussed in the ML, this PR introduces two flags to enable the inclusion of realm ID tags in API and HTTP metrics.

They are both disabled by default.

There is also a new safeguard: if the cardinality of realm IDs in HTTP metrics goes above a configurable threshold (100 by default), a warning is printed and no more HTTP metrics will be recorded. (Quarkus has a similar safeguard for URI tags in HTTP metrics.)

* Site/contributing: add recommendations for working with PRs (apache#1625)

This change updates the PR guidelines on the "Contributing" web page after [this discussion](https://lists.apache.org/thread/kfxo3cqmw3pgrpgtgqvqpwvn46yw8q7h).

Also adopt `gradlew test` to `gradlew check` in README, following the intent (all tests, incl ITs)

* main: Update dependency com.azure:azure-sdk-bom to v1.2.35 (apache#1703)

* main: Update dependency com.adobe.testing:s3mock-testcontainers to v4.4.0 (apache#1705)

* main: Update dependency boto3 to v1.38.24 (apache#1702)

* Keep generated RSA-key-pair for JWT token broker on heap (apache#1661)

Polaris allows using RSA key-paris for the JWT token broker. The recommended way is to [generate the RSA key pair](https://github.com/apache/polaris/blob/d8b862b13914d526ee147dc0e359bfc9c1e319ad/site/content/in-dev/unreleased/configuring-polaris-for-production.md?plain=1#L61-L66) and configure the location of the key files.

However, if only `polaris.authentication.token-broker.type=rsa-key-pair` but not the `public/private-key-pair` options are configured, Polaris generates those and stores them in `/tmp` using random file names (using `Files.createTempFile()`) - this happens for each (matching) realm. Each Polaris startup generates new key-pairs for each of those realms. It's practically not possible to associate the files to a realm. There is already a [production readiness check](https://github.com/apache/polaris/blob/d8b862b13914d526ee147dc0e359bfc9c1e319ad/quarkus/service/src/main/java/org/apache/polaris/service/quarkus/config/ProductionReadinessChecks.java#L118-L166) to warn users about this behavior.

Due to the issue that the files cannot be associated, those seem to be somewhat useless and bring no advantage over keeping these "ephemeral RSA key pairs" on heap. This PR changes the code to not write the key-pair to the file system and keeps these "ephemeral key pairs" on heap. Since the same code path is used for key-paris _provided_ by the user (via the `public/private-key-pair` config options), that code path now only reads those files once and not every time the private/public key is needed.

* Merge JPA module with EclipseLink Module (apache#1718)

* Create LICENSE and NOTICE for "single" distribution (apache#1694)

* Fix a failing task with the release profile (apache#1693)

* Remove unused adminDocs artifact (apache#1749)

* Production readiness for Persistence (apache#1707)

Production readiness for Persistence (apache#1707)

* Fixes for direct usage of client_secret apache#1756

When the spec was upgraded and the python client regenerated from it, clientSecret was made a password, which means calling str on it directly yields a redacted string like ******. In the initial PR to change the python client and fix regtests, some existing usage of client_secret was not changed.

* main: Update dependency org.junit:junit-bom to v5.13.0 (apache#1760)

* main: Update dependency org.testcontainers:testcontainers-bom to v1.21.1 (apache#1748)

* fix: Improve reliability of metrics tests (apache#1763)

CI sometimes fails with errors like "http_server_requests_seconds not found"
in the reported metrics.

This looks like a race between the Quarkus metrics producer and the
tests asking for these metrics.

This change adds a time-limited retry loop until the expected metrics
are available, before proceeding with other assertions.

Note: in normal cases the loop finishes fast because the metrics are
available. The two-minute timeout would apply only when the expected
metrics fail to be produced at all.

* Fix test_spark_credentials_s3_exception_on_metadata_file_deletion (apache#1759)

* Regenerate bundled spec & Regenerate Python client (apache#1751)

I ran these commands from main:

```
redocly bundle spec/polaris-catalog-service.yaml -o spec/generated/bundled-polaris-catalog-service.yaml
./gradlew regeneratePythonClient
```

I didn't realize before that some Python types are generated form the bundled spec, so some of the fixes from apache#1347 didn't get properly applied before.

* main: Update dependency boto3 to v1.38.27 (apache#1714)

* NoSQL: bump Weld/Junit5 (fixes a bug that surfaces w/ JUnit 5.13)

* NoSQL: Let some more tests leverage Jandex

* Info: Last merged commit b7aac72

---------

Co-authored-by: Mend Renovate <bot@renovateapp.com>
Co-authored-by: Alexandre Dutra <adutra@users.noreply.github.com>
Co-authored-by: Yufei Gu <yufei@apache.org>
Co-authored-by: JB Onofré <jbonofre@apache.org>
Co-authored-by: Prashant Singh <35593236+singhpk234@users.noreply.github.com>
Co-authored-by: Eric Maynard <eric.maynard+oss@snowflake.com>
Co-authored-by: Dmitri Bourlatchkov <dmitri.bourlatchkov@gmail.com>
Co-authored-by: gh-yzou <167037035+gh-yzou@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants