Epic: small footprint container usable for development, testing and production purposes

[poikilotherm]

IMHO this is an epic, not a single story.

This issue is successor to #5187 and closes it.

It is intended to serve as a base for solutions or make life easier in:

• Running integration tests for more than just API #5068 - Running integration tests for more than just API
• installing Dataverse under Payara 5 #4172 - Payara 5 Upgrade
• Refactoring of S3AccessIO driver #5072 - Refactoring of S3 storage driver
• Docker for production #4665 - Docker in production and projects related to that (IQSS/dataverse-docker and the like)
• Get Dataverse running on OpenShift (Docker and Kubernetes) #4040 - OpenShift / Kubernetes deployments

This is blocked by some stuff and relies on some prior art to be done:

• Dependency Housekeeping and how developers will benefit from it #5288 - Dependency Housekeeping Done!
• Stripping AWS Mail component with TrueZIP blocks packaging for docker images #5274 - Stripping AWS dependency and TrueZIP from POM
• Dependency Housekeeping, Round 2 #5360 - Dependency Housekeeping, second round
• As an admin and developer, I'd benefit from a simplified configuration mechanism #5293 - Simplify configuration of the application.
• Proposal: make persistent EJB timers non-persistent or not rely on database #5345 - Make EJB timers non-persistent to drop database requirement.
• Bootstrapping config on first deployment #5361 - Bootstraping empty instance
• Eating 100% CPU when setting com.sun.enterprise.server.logging.GFFileHandler.logtoFile=false payara/Payara#3506 and payara/docker-payaraserver-full#72 - Logging to console eats 1 CPU

Things to consider:

• Add a HEALTCHECK CMD that uses a to-be-built API endpoint reporting on health status?

---

Vision / Proposal

Currently, when running integration tests or deploying Dataverse to Docker/Kubernetes, only kind of heavyweight solutions exist with the DockerAIO for IT tests and most (all?) Docker/Kubernetes/OpenShift approaches relying on the installer script.

I encourage the following vision:

1. Build a new image directly from a Maven target, needing only a dev env plus Docker installed and running (obsolete once img support is in place...)
2. Make this image as small as possible with an application server only, add dependencies and the application.
3. Anything else lives in other containers, following the micro services credo.
4. Make the application container itself stateless, also following the micro services credo. (This does not affect the use of volumes/...)
5. Make the configuration a breeze - don't use the install script inside the container. Instead provide options to get the configuration inside from external sources.

To get there I suggest using:

• fabric8io/docker-maven-plugin
• payara/server-full Docker image
• Separate Maven profile to build this as long as the "old way" is still used broadly.

Things to keep in mind:

1. Ideally this is based on Payara 5, not 4.
2. Let users still use the "old" WAR file approach in parallel! Somebody might rely on that. (That's why I killed As a developer and sysadmin, I would enjoy the benefits of using Java MicroProfile based deployments with an application server like Payara Micro or Thorntail. #5187)
3. Let the configuration ways currently know to all users still work. Somebody might rely on that!

Give it a shot! (Testing)

To test, just have Docker, Maven, Git and Java installed.
Then do:

git clone https://github.com/poikilotherm/dataverse -b 5292-small-container
mvn -Pcontainer clean package docker:build docker:run -DskipTests

Please keep in mind that this is a feature branch. If you already have a cloned dataverse repo, you might better off using:

git remote add poikilotherm https://github.com/poikilotherm/dataverse
git fetch poikilotherm 5292-small-container
git checkout -b 5292-small-container

I regularly update this feature branch to be based on the latest develop. This involves rebasing, which will let your local branch be diverged. In that case, simply use git reset --hard poikilotherm/5292-small-container after a fetch.

[poikilotherm]

@pdurbin and other: initial work on the building part has been to my feature branch.

Right now this will (of course) not work. The Postgres driver is missing right now and the config part (see #5293) has to be addressed first.

As I wrote in the commit message, the upstream container project is in some parts not very responsive/active (see here, here, here and here).

Will try my best to get things upstream, but maybe better fork and try to get this upstream later:

• Fix the missing signal handling including gracefull shutdowns
• Get inspiration from the stuff that @shoeper did
• Adress more issues from upstream.

Also wondering if automated builds and security scans from quay.io could be interesting for this.

[poikilotherm]

I am currently working on an updated payara image, see https://github.com/poikilotherm/docker-payaraserver-full/tree/refactor . Will try to get my work upstream, just a few hours ago they merged stuff :-D

Since the upstream merged a lot of stuff cleaning up most of the issues (see here), I switched back to use those. Some issues still exist with the init system.

[poikilotherm]

I just opened PR payara/docker-payaraserver-full#61 and hope things will get merged.

[poikilotherm]

Current niceness blockers:

• It would be a great relief not to include the bootstrap shell scripts but make Dataverse do it themself. Bootstrapping config on first deployment #5361, scripts for rebuilding a dev environment (with sample data) #7256
• It would so great to not fiddle with the Postgres driver, but include it in the WAR. Include Postgres JDBC driver in pom.xml #6819
• It would be so nice not to create a tedious init script configuring Dataverse on container boot. This is true for multiple resources.
  A start to be made with the DB connection, using @DataSourceDefinition, see Include Postgres JDBC driver in pom.xml #6819. JMS etc to follow.
• No more fiddling with password aliases by using secrets via MicroProfile Config. As a sysadmin I want to use MicroProfile Config API to configure my installation #7000, As an admin and developer, I'd benefit from a simplified configuration mechanism #5293
• No more fiddling with configuration parsing during startup by using MicroProfile Config. As a sysadmin I want to use MicroProfile Config API to configure my installation #7000
• Some other, smaller things to do...

But maybe we can just deal with it for now. Jenkinsfiles missing. Solr image missing.

[poikilotherm]

There is still plenty of stuff to do. Yet this issue is quite dated and it's starting to become messy. Closing this for now.

Note, that there is traction at https://github.com/gdcc/dataverse/tree/develop+ct, https://github.com/gdcc/dataverse-kubernetes and much MPCONFIG stuff going on.