Persistence plugins in default build artifacts

[collado-mike]

Should we include persistence plugins in the default build output? Originally, we included eclipselink in the build output of the shadow jar that is used in the Docker image. This is nice for anyone that wants to configure the default docker image to use eclipselink instead of the in-memory PolarisMetaStoreManager. That was accidentally removed in #53 . It's easy enough to fix that, but I wanted to bring up the question of whether or not we should, by default, include the extension project(s) in the default build output.

The intention of the extension builds is to support multiple options for plugins which, possibly for licensing reasons, would not be required dependencies (hibernate was one use case). If we were to add support for other persistence plugins, we may have to deal with conflicting dependencies, as everything is lumped into the same shadow jar.

On the other hand, not shipping a persistent metastore plugin makes the default Docker image somewhat useless for production. Though, eclipselink's requirement on a jar that contains the persistence.xml file also makes the default Docker image unusable for production anyhow, so...

Thoughts?

[MonkeyCanCode]

Compared Polaris to Nessie, Nessie supportes a list of plug-able backend metastore out of box (https://projectnessie.org/nessie-latest/configuration/#version-store-settings with setting nessie.version.store.type). So an end-user can select from a list of supported backends and run with it.

Currently even if we enabled eclipselink by default (and add h2 lib back for dependency), it is still pretty useless imo for production. For anyone to use it, they will still need to modify the gradle dependencies to include the driver jars they will needed and test out if that driver will work or not which is a bit inconvenience and making reproduce issues a bit more tedious (e.g. assuming user A is using driver A and user B is using driver Z to connect to MySQL and user A reported there are issues with connections etc. it will be hard for user B to reproduce the issue as the issue can be from driver as well as diff configures settings supported by various drivers). Then people will end up running their own forks for their prod workload.

Then for licensing purpose, can we not avoid it by pulling down dependencies on runtime or use Apache licensed driver lib? For example, for Spark to connect to MySQL, a end-user will need to either pass in arguments to pull those dependencies before the job started or copy the jar into their image during build.

[snazy]

(Nessie maintainer here) TL;DR the approach between the extension model/shadowJar is different from what Nessie technically does. Nessie is built upon a "key value store first" approach (BigTable, Dynamo, Mongo, Rocks, "in memory", but also JDBC). And Nessie benefits a lot from all-the-things Quarkus brings. For h2/pgsql/etc Nessie uses JDBC directly, there's no ORM mapper like Eclipselink or Hibernate. The approach allows us to bundle all database libs in a single distribution and users can configure the backing storage (database).

Not sure if using "plain boilerplate JDBC" is an option, the operations do not look that complex from a brief look.

[MonkeyCanCode]

Very good to know. When using Nessie with Quarkus, it makes a lot of things easier such as how to configure the db properties and quickly changed to a diff backend for additional experiments.

[flyrain]

I think it is important to make the default build close to production ready as much as possible. We don't have to include every options, but at least a default ORM layer.

In case of driver's licensing restriction, pulling at runtime or picking a Apache licensed one could be a solution as @MonkeyCanCode suggested. We can use a deployment script or any orchestrator to pull the driver.

[collado-mike]

I'm inclined to agree with the following caveats

1. as @snazy mentioned, a pure JDBC or at least "lightweight" JDBC layer with few dependencies would be preferable and limit classpath conflicts
2. Given the Eclipselink requirement to package the persistence.xml into a jar, the user has to add new jar dependencies anyway

With those in mind, is it sufficient to provide an almost "production ready" image with clear instructions for the last mile (this is necessary already to some degree to configure the persistence layer)?

TBH, I'm fine either way here, but I think it's good to tackle the question now before any releases go out to maven or dockerhub or whatever. Once we start packaging eclipselink with everything else, we'll have to continue doing so even if other extensions are added later on.

[snazy]

Since users have to do some packaging/re-configuration anyway, I think it's safer to not include eclipselink by default. It still leaves the option to add it later, but the other way around is much more difficult.

[flyrain]

Given the Eclipselink requirement to package the persistence.xml into a jar, the user has to add new jar dependencies anyway

Given the Eclipselink limitation, it makes sense to me to leave it out now.

[TomerHeber]

I recommend moving the eclipse link extension to a different repo.
Add it to maven and then document how to use the extension.

[snazy]

It doesn't necessarily need to be a different repo. The extension can still be released (as a Maven artifact), it's rather that the extension is not in a "default" distribution.

[collado-mike]

I merged #114 as a result of this. Closed the discussion.