Skip to content

Commit

Permalink
runner.singularity: Support for build's --cpus and --memory options
Browse files Browse the repository at this point in the history
Only available since Singularity 3.10.0 and on systems with cgroups v2.¹
With older Singularity versions, these options will continue to be
ignored (as before this change).  If the Singularity version is new
enough but the system doesn't support cgroups v2 (or it is
administratively disabled in the Singularity config), then Singularity
will error when these options are used.

¹ <https://docs.sylabs.io/guides/latest/user-guide/cgroups.html#requirements-linux-cgroups>
  • Loading branch information
tsibley committed May 25, 2023
1 parent 91e079e commit cfbdda7
Show file tree
Hide file tree
Showing 3 changed files with 15 additions and 3 deletions.
3 changes: 2 additions & 1 deletion doc/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -193,7 +193,8 @@ dependencies as validated versions are already bundled into a container image
by the Nextstrain team.

Run ``nextstrain setup singularity`` to get started.
Singularity version 3.0.0 or newer is required.
Singularity version 3.0.0 or newer is required, but we recommend at least
version 3.10.0 or newer when possible.

Note that the Singularity project forked into two separate projects in late
2021: `SingularityCE`_ under `Sylabs`_ and `Apptainer`_ under the `Linux
Expand Down
4 changes: 2 additions & 2 deletions nextstrain/cli/command/build.py
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ def register_parser(subparser):
parser.add_argument(
"--cpus",
help = "Number of CPUs/cores/threads/jobs to utilize at once. "
"Limits containerized (Docker, AWS Batch) builds to this amount. "
"Limits containerized (Docker, Singularity, AWS Batch) builds to this amount. "
"Informs Snakemake's resource scheduler when applicable. "
"Informs the AWS Batch instance size selection. "
"By default, no constraints are placed on how many CPUs are used by a build; "
Expand All @@ -63,7 +63,7 @@ def register_parser(subparser):
"--memory",
help = "Amount of memory to make available to the build. "
"Units of b, kb, mb, gb, kib, mib, gib are supported. "
"Limits containerized (Docker, AWS Batch) builds to this amount. "
"Limits containerized (Docker, Singularity, AWS Batch) builds to this amount. "
"Informs Snakemake's resource scheduler when applicable. "
"Informs the AWS Batch instance size selection. ",
metavar = "<quantity>",
Expand Down
11 changes: 11 additions & 0 deletions nextstrain/cli/runner/singularity.py
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,17 @@ def run(opts, argv, working_volume = None, extra_env = {}, cpus: int = None, mem
# Change the default working directory if requested
*(("--pwd", "/nextstrain/%s" % working_volume.name) if working_volume else ()),

# Set resource limits if any. These options are equivalent to Docker's
# and are first available in Singularity 3.10.0. The lower-level
# --apply-cgroups option first available in 3.9.0 can do the same
# things, but the version difference seems marginally useful and
# there's more implementation overhead required to use --apply-cgroups.
*(["--cpus", str(cpus)]
if cpus is not None and singularity_version_at_least("3.10.0") else []),

*(["--memory", str(memory)]
if memory is not None and singularity_version_at_least("3.10.0") else []),

str(image_path(image)),
*argv,
], extra_env)
Expand Down

0 comments on commit cfbdda7

Please sign in to comment.