runner.singularity: Allow writes to HOME on the container filesystem

Our recent upgrade of Snakemake in our container image¹ resulted in an
fatal error when used with the Singularity runtime, e.g.:

    OSError: [Errno 30] Read-only file system: '/home/runner'

We first observed this in periodic CI for Nextstrain CLI, but were able
to reproduce it outside CI.  The error comes from Snakemake inside the
container (newly) trying to `mkdir -p` paths under HOME and represents
two separate-but-intertwined bugs: the first being the read-only
container filesystem preventing creation under HOME, and the second that
the value of HOME from outside the container (e.g. /home/runner in CI)
is leaking into the container.

The read-only filesystem previously wasn't an issue (and wasn't noticed)
because writes were limited to paths bind-mounted read-write from
outside the container (e.g. /nextstrain/build).  Writing to the
container filesystem has always been allowed in the Docker runtime, so
we come to parity with it using --writable-tmpfs, which requires
Singularity 3.0.0.

The HOME leak arose thru a intersection of behaviours.  Our container
image sets HOME=/nextstrain², but this is a *default* value that can be
overridden at container launch time.  With our Docker runtime, it's
never overridden.  Singularity, however, automatically forwards the
user's HOME value into the container, even though we use --no-home to
disable mounting the user's home at the same path.  The forwarded value
thus override our image's default.  Providing an explicitly empty HOME
value via the --home option allows the image default to apply, as
expected.

Resolves: <#274>

¹ <nextstrain/docker-base#136>
² <https://github.com/nextstrain/docker-base/blob/1b0d1998/Dockerfile#L390>

doc/installation.rst

@@ -193,7 +193,7 @@ dependencies as validated versions are already bundled into a container image
 by the Nextstrain team.
 
 Run ``nextstrain setup singularity`` to get started.
-Singularity version 2.6.0 or newer is required.
+Singularity version 3.0.0 or newer is required.
 
 Note that the Singularity project forked into two separate projects in late
 2021: `SingularityCE`_ under `Sylabs`_ and `Apptainer`_ under the `Linux

nextstrain/cli/runner/singularity.py

@@ -43,7 +43,7 @@
              or "docker://nextstrain/base"
 
 
-SINGULARITY_MINIMUM_VERSION = "2.6.0"
+SINGULARITY_MINIMUM_VERSION = "3.0.0"
 
 SINGULARITY_CONFIG_ENV = {
     # Store image caches in our runtime root instead of ~/.singularity/…
@@ -89,7 +89,29 @@
     # ¹ <https://docs.sylabs.io/guides/latest/user-guide/singularity_and_docker.html#docker-like-compat-flag>
     # ² <https://docs.sylabs.io/guides/latest/user-guide/oci_runtime.html#oci-mode>
     "--contain",
+
+    # Don't mount anything at all at the container's value of HOME.  This is
+    # necesary because --compat includes --containall which includes --contain
+    # which makes HOME in the container an empty temporary directory.
+    # --no-home is available since 2.6.0.
     "--no-home",
+
+    # Singularity really wants to default HOME inside the container to the
+    # value from outside the container, thus ignoring the value set by the
+    # upstream Docker image which is only used as a default by the Singularity
+    # image.  Singularity forbids using --env to directly override HOME, so
+    # instead we use --home <src>:<dst> with two empty values.  <src> doesn't
+    # apply because we use --no-home, and setting <dst> to an empty value
+    # allows the container's default to apply (thus avoiding hardcoding it
+    # here).
+    "--home", ":",
+
+    # Allow writes to the image filesystem, discarded at container exit, à la
+    # Docker.  Snakemake, for example, needs to be able to write to HOME
+    # (/nextstrain).
+    "--writable-tmpfs",
+
+    # Don't copy entire host environment.  We forward our own hostenv.
     "--cleanenv",
 
     # Since we use --no-home above, avoid warnings about not being able to cd