wip! bug fix

runner.singularity: Ensure HOME is taken from the image's default

Singularity had been setting HOME to the user's HOME outside of the
container which meant the image's default of /nextstrain wasn't getting
used.  This apparently wasn't an issue until we upgraded Snakemake¹ and
it started expecting to be able to write files under HOME.  HOME was
practically guaranteed *not* to exist in the container because we
disable mounting of the user's homedir with --no-home.

We first observed this bug (alongside another, to be addressed next) in
CI as:

    OSError: [Errno 30] Read-only file system: '/home/runner'

The error comes from inside the container, but that homedir path comes
from outside (on the CI runner).  Since it isn't mounted due to
--no-home, it's a non-existent path on the image filesystem.  That could
be ok since Snakemake will create the missing parts (without a boundary!
orz), but it isn't ok in practice since the image filesystem is by
default read-only (and the next bug to address).

Note that even though our Docker image sets HOME=/nextstrain, the
conversion to a Singularity image makes that a *default* value, e.g.
HOME=${HOME:-/nextstrain}.  It's unsurprising this semantic shift is a
source of a subtle bugs!

Resolves: <#274>

¹ <nextstrain/docker-base#136>

doc/installation.rst

@@ -193,7 +193,7 @@ dependencies as validated versions are already bundled into a container image
 by the Nextstrain team.
 
 Run ``nextstrain setup singularity`` to get started.
-Singularity version 2.6.0 or newer is required.
+Singularity version 3.0.0 or newer is required.
 
 Note that the Singularity project forked into two separate projects in late
 2021: `SingularityCE`_ under `Sylabs`_ and `Apptainer`_ under the `Linux

nextstrain/cli/runner/singularity.py

@@ -43,7 +43,7 @@
              or "docker://nextstrain/base"
 
 
-SINGULARITY_MIN_VERSION = "2.6.0"
+SINGULARITY_MIN_VERSION = "3.0.0"
 
 SINGULARITY_CONFIG_ENV = {
     # Store image caches in our runtime root instead of ~/.singularity/…
@@ -89,7 +89,29 @@
     # ¹ <https://docs.sylabs.io/guides/latest/user-guide/singularity_and_docker.html#docker-like-compat-flag>
     # ² <https://docs.sylabs.io/guides/latest/user-guide/oci_runtime.html#oci-mode>
     "--contain",
+
+    # Allow writes to the image filesystem, discarded at container exit, à la
+    # Docker.  Snakemake, for example, needs to be able to write to HOME
+    # (/nextstrain).
+    "--writable-tmpfs",
+
+    # Don't mount anything at all at the container's value of HOME.  This is
+    # necesary because --compat includes --containall which includes --contain
+    # which makes HOME in the container an empty temporary directory.
+    # --no-home is available since 2.6.0.
     "--no-home",
+
+    # Singularity really wants to default HOME inside the container to the
+    # value from outside the container, thus ignoring the value set by the
+    # upstream Docker image which is only used as a default by the Singularity
+    # image.  Singularity forbids using --env to directly override HOME, so
+    # instead we use --home <src>:<dst> with two empty values.  <src> doesn't
+    # apply because we use --no-home, and setting <dst> to an empty value
+    # allows the container's default to apply (thus avoiding hardcoding it
+    # here).
+    "--home", ":",
+
+    # Don't copy entire host environment.  We forward our own hostenv.
     "--cleanenv",
 
     # Since we use --no-home above, avoid warnings about not being able to cd