feat(ct-base): switch /docroot to /dv and add volumes IQSS#8932

- Instead of a /docroot, add a more generic /dv which is owned by payara:payara
  and can be used to either store data in a single volume using subfolders or
  use subfolders with different backing volumes. Anyway, data is not written
  to overlay FS this way. (As long as an app image points to this location)
- Also define /secrets and /dumps as volumes, so data flowing into these
  locations is again not added to the overlay FS (which might cause severe
  damage in case of heap dumps!)
- Document the different locations in the base image guide.
- Remove the /docroot workaround for uploaded files. This will be solved at application
  level (either by moving the workaround there) or IQSS#8983

doc/sphinx-guides/source/container/base-image.rst

@@ -218,7 +218,16 @@ Locations
 +++++++++
 
 This environment variables represent certain locations and might be reused in your scripts etc.
-These variables aren't meant to be reconfigurable and reflect state in the filesystem layout!
+All of these variables aren't meant to be reconfigurable and reflect state in the filesystem layout!
+
+**Writeable at build time:**
+
+The overlay filesystem of Docker and other container technologies is not meant to be used for any performance IO.
+You should avoid *writing* data anywhere in the file tree at runtime, except for well known locations with mounted
+volumes backing them (see below).
+
+The locations below are meant to be written to when you build a container image, either this base or anything
+building upon it. You can also use these for references in scripts, etc.
 
 .. list-table::
     :align: left
@@ -245,10 +254,35 @@ These variables aren't meant to be reconfigurable and reflect state in the files
     * - ``DEPLOY_DIR``
       - ``${HOME_DIR}/deployments``
       - Any EAR or WAR file, exploded WAR directory etc are autodeployed on start
-    * - ``DOCROOT_DIR``
-      - ``/docroot``
-      - Mount a volume here to store i18n language bundle files, sitemaps, images for Dataverse collections, logos,
-        custom themes and stylesheets, etc here. You might need to replicate this data or place on shared file storage.
+    * - ``DOMAIN_DIR``
+      - ``${PAYARA_DIR}/glassfish`` ``/domains/${DOMAIN_NAME}``
+      - Path to root of the Payara domain applications will be deployed into. Usually ``${DOMAIN_NAME}`` will be ``domain1``.
+
+
+**Writeable at runtime:**
+
+The locations below are defined as `Docker volumes <https://docs.docker.com/storage/volumes/>`_ by the base image.
+They will by default get backed by an "anonymous volume", but you can (and should) bind-mount a host directory or
+named Docker volume in these places to avoid data loss, gain performance and/or use a network file system.
+
+**Notes:**
+1. On Kubernetes you still need to provide volume definitions for these places in your deployment objects!
+2. You should not write data into these locations at build time - it will be shadowed by the mounted volumes!
+
+.. list-table::
+    :align: left
+    :width: 100
+    :widths: 10 10 50
+    :header-rows: 1
+
+    * - Env. variable
+      - Value
+      - Description
+    * - ``STORAGE_DIR``
+      - ``/dv``
+      - This place is writeable by the Payara user, making it usable as a place to store research data, customizations
+        or other. Images inheriting the base image should create distinct folders here, backed by different
+        mounted volumes.
     * - ``SECRETS_DIR``
       - ``/secrets``
       - Mount secrets or other here, being picked up automatically by
@@ -258,10 +292,6 @@ These variables aren't meant to be reconfigurable and reflect state in the files
       - ``/dumps``
       - Default location where heap dumps will be stored (see above).
         You should mount some storage here (disk or ephemeral).
-    * - ``DOMAIN_DIR``
-      - ``${PAYARA_DIR}/glassfish`` ``/domains/${DOMAIN_NAME}``
-      - Path to root of the Payara domain applications will be deployed into. Usually ``${DOMAIN_NAME}`` will be ``domain1``.
-
 
 
 Exposed Ports

modules/container-base/src/main/docker/Dockerfile

@@ -1,4 +1,4 @@
-# Copyright 2019 Forschungszentrum Jülich GmbH
+# Copyright 2022 Forschungszentrum Jülich GmbH
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
@@ -38,7 +38,7 @@ ENV PAYARA_DIR="${HOME_DIR}/appserver" \
     SCRIPT_DIR="${HOME_DIR}/scripts" \
     CONFIG_DIR="${HOME_DIR}/config" \
     DEPLOY_DIR="${HOME_DIR}/deployments" \
-    DOCROOT_DIR="/docroot" \
+    STORAGE_DIR="/dv" \
     SECRETS_DIR="/secrets" \
     DUMPS_DIR="/dumps" \
     PASSWORD_FILE="${HOME_DIR}/passwordFile" \
@@ -73,17 +73,19 @@ ARG GID=1000
 USER root
 WORKDIR /
 SHELL ["/bin/bash", "-euo", "pipefail", "-c"]
+# Mark these directories as mutuable data containers to avoid cluttering the images overlayfs at runtime.
+VOLUME ${STORAGE_DIR} ${SECRETS_DIR} ${DUMPS_DIR}
 RUN <<EOF
     # Create pathes
     mkdir -p "${HOME_DIR}" "${PAYARA_DIR}" "${DEPLOY_DIR}" "${CONFIG_DIR}" "${SCRIPT_DIR}"
-    mkdir -p "${DOCROOT_DIR}" "${SECRETS_DIR}" "${DUMPS_DIR}"
+    mkdir -p "${STORAGE_DIR}" "${SECRETS_DIR}" "${DUMPS_DIR}"
     # Create user
     addgroup --gid ${GID} payara
     adduser --system --uid ${UID} --no-create-home --shell /bin/bash --home "${HOME_DIR}" --gecos "" --ingroup payara payara
     echo payara:payara | chpasswd
     # Set permissions
     chown -R payara: "${HOME_DIR}"
-    chown -R payara: "${DOCROOT_DIR}" "${SECRETS_DIR}" "${DUMPS_DIR}"
+    chown -R payara: "${STORAGE_DIR}" "${SECRETS_DIR}" "${DUMPS_DIR}"
 EOF
 
 ARG JATTACH_VERSION="v2.1"
@@ -211,15 +213,6 @@ RUN <<EOF
         "${PAYARA_DIR}/glassfish/domains/${DOMAIN_NAME}/logs"
 EOF
 
-# Make docroot of Payara reside in higher level directory for easier targeting
-# Due to gdcc/dataverse-kubernetes#177: create the generated pathes so they are
-# writeable by us. TBR with gdcc/dataverse-kubernetes#178.
-RUN <<EOF
-    rm -rf "${DOMAIN_DIR}"/docroot
-    ln -s "${DOCROOT_DIR}" "${DOMAIN_DIR}"/docroot
-    mkdir -p "${DOMAIN_DIR}"/generated/jsp/dataverse
-EOF
-
 # Set the entrypoint to tini (as a process supervisor)
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 # This works because we add ${SCRIPT_DIR} to $PATH above!