diff --git a/docs/getting_started/developer-guide.rst b/docs/getting_started/developer-guide.rst index b67131dbe..c99e873ab 100644 --- a/docs/getting_started/developer-guide.rst +++ b/docs/getting_started/developer-guide.rst @@ -203,7 +203,7 @@ Wrapper Script Singularity HPC allows exposure of two kinds of wrapper scripts: -1. A global level wrapper intended to replace aliases. E.g., if an alias "samtools" is typically a direct container call, enabling a wrapper will generate an executable script "samtools" in a bin associated with the container, added to the path, to call instead. This is desired for workflow tools that want to run scripts. This global script is defined in settings.yml and described in the user guide. +1. A global level wrapper intended to replace aliases. E.g., if an alias "samtools" is typically a direct container call, enabling a wrapper will generate an executable script "samtools" in a "bin" directory associated with the container, added to the path, to call instead. This is desired when MPI ("mpirun") or scheduler (e.g. "srun" with Slurm) utilities are needed to run the scripts. This global script is defined in settings.yml and described in the user guide. 2. A container level wrapper that is specific to a container, described here. For container specific scripts, you can add sections to a ``container.yaml`` to specify the script (and container type) @@ -233,13 +233,14 @@ the custom container script "fork": .. code-block:: console - $ tree containers/vanessa/salad/ - containers/vanessa/salad/ + $ tree modules/vanessa/salad/ + modules/vanessa/salad/ └── latest - ├── bin - │   ├── fork - │   └── salad - └── vanessa-salad-latest-sha256:e8302da47e3200915c1d3a9406d9446f04da7244e4995b7135afd2b79d4f63db.sif + ├── 99-shpc.sh + ├── bin + │   ├── fork + │   └── salad + └── module.lua If we disable all wrapper scripts, the bin directory would not exist. If we set the default wrapper scripts for singularity and docker in settings.yml and left enable to true, we would only see "fork." diff --git a/docs/getting_started/user-guide.rst b/docs/getting_started/user-guide.rst index d2e5b4c45..304c2a927 100644 --- a/docs/getting_started/user-guide.rst +++ b/docs/getting_started/user-guide.rst @@ -201,6 +201,21 @@ variable replacement. A summary table of variables is included below, and then f * - test_shell - The shell used for the test.sh file - /bin/bash + * - wrapper_shell + - The shell used for wrapper scripts + - /bin/bash + * - wrapper_scripts:enabled + - enable or disable generation of wrapper scripts, instead of module aliases + - false + * - wrapper_scripts:docker + - The name of the generic wrapper script template for docker + - docker.sh + * - wrapper_scripts:podman + - The name of the generic wrapper script template for podman + - docker.sh + * - wrapper_scripts:singularity + - The name of the generic wrapper script template for singularity + - singularity.sh * - namespace - Set a default module namespace that you want to install from. - null @@ -451,7 +466,7 @@ Wrapper Scripts Singularity HPC allows for global definition of wrapper scripts, meaning that instead of writing a module alias to run a container for some given alias, we generate a wrapper script of the same name instead. Since the settings.yml is global, all wrapper scripts defined here are specific to replacing aliases. -Container-specific scripts you'll want to include in the container.yaml, described in the developer docs. Let's take a look at the settings: +Container-specific scripts you'll want to include in the container.yaml are described in the developer docs. Let's take a look at the settings: .. code-block:: yaml @@ -474,14 +489,14 @@ Container-specific scripts you'll want to include in the container.yaml, describ Since different container technologies might expose different environment variables (e.g., ``SINGULARITY_OPTS`` vs ``PODMAN_OPTS``) they are organized above based on the container technology. If you want to customize the wrapper script, simply replace the relative paths -above (e.g., ``singularity.sh`` with an absolute path to a file that will be used instead. For global alias scripts such as these, +above (e.g., ``singularity.sh``) with an absolute path to a file that will be used instead. For global alias scripts such as these, Singularity HPC will look for: 1. An absolute path first, if found is used first. 2. Then a script name in the shpc/main/wrappers directory Here is an example of using wrapper scripts for the "python" container, which doesn't have container specific wrappers. What you see -is the one entrypoint, "python" being placed in a bin that the module will see instead of defining the alias. +is the one entrypoint, `python`, being placed in a "bin" subdirectory that the module will see instead of defining the alias. .. code-block:: console @@ -533,7 +548,7 @@ Where are wrapper scripts stored? ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Since we don't allow overlap -of the name of an alias wrapper script (e.g., ``bin/python``) as a wrapper to a python entrypoint) from a custom container wrapper script (e.g., a wrapper script with name "python" under a container.yaml) we can keep them both in the modules directory. If you see a need to put them elsewhere please let us kno. +of the name of an alias wrapper script (e.g., ``bin/python`` as a wrapper to a python entrypoint) from a custom container wrapper script (e.g., a wrapper script with name "python" under a container.yaml) we can keep them both in the modules directory. If you see a need to put them elsewhere please let us know. .. _getting_started-commands: diff --git a/shpc/main/modules/templates/docs.md b/shpc/main/modules/templates/docs.md index 35eb75e0d..1a30cfb20 100644 --- a/shpc/main/modules/templates/docs.md +++ b/shpc/main/modules/templates/docs.md @@ -47,7 +47,7 @@ You can use tab for auto-completion of module names or commands that are provide ### Commands -When you install this module, you'll be able to load it to make the following commands accessible. +When you install this module, you will be able to load it to make the following commands accessible. Examples for both Singularity, Podman, and Docker (container technologies supported) are included. #### {|module_name|}-run: @@ -61,15 +61,15 @@ $ docker run --rm {% if settings.bindpaths %}-v {{ settings.bindpaths }} {% endi #### {|module_name|}-shell: ```bash -$ singularity settings.singularity_shell -s {{ settings.singularity_shell }} {% if settings.bindpaths %}-B {{ settings.bindpaths }} {% endif %} -$ podman run --it --rm --entrypoint {{ settings.singularity_shell }} {% if settings.bindpaths %}-v {{ settings.bindpaths }} {% endif %} -v ${PWD} -w ${PWD} -$ docker run --it --rm --entrypoint {{ settings.singularity_shell }} {% if settings.bindpaths %}-v {{ settings.bindpaths }} {% endif %} -v ${PWD} -w ${PWD} +$ singularity shell -s {{ settings.singularity_shell }} {% if settings.bindpaths %}-B {{ settings.bindpaths }} {% endif %} +$ podman run --it --rm --entrypoint {{ settings.podman_shell }} {% if settings.bindpaths %}-v {{ settings.bindpaths }} {% endif %} -v ${PWD} -w ${PWD} +$ docker run --it --rm --entrypoint {{ settings.docker_shell }} {% if settings.bindpaths %}-v {{ settings.bindpaths }} {% endif %} -v ${PWD} -w ${PWD} ``` #### {|module_name|}-exec: ```bash -$ singularity exec -s {{ settings.singularity_shell }} {% if settings.bindpaths %}-B {{ settings.bindpaths }} {% endif %} "$@" +$ singularity exec {% if settings.bindpaths %}-B {{ settings.bindpaths }} {% endif %} "$@" $ podman run --it --rm --entrypoint "" {% if settings.bindpaths %}-v {{ settings.bindpaths }} {% endif %} -v ${PWD} -w ${PWD} "$@" $ docker run --it --rm --entrypoint "" {% if settings.bindpaths %}-v {{ settings.bindpaths }} {% endif %} -v ${PWD} -w ${PWD} "$@" ``` @@ -118,7 +118,7 @@ $ docker run --rm {% if settings.bindpaths %}-v {{ settings.bindpaths }}{% endif In the above, the `` directive will reference an actual container provided by the module, for the version you have chosen to load. An environment file in the module folder will also be bound. Note that although a container -might provide custom commands, every container exposes unique exec, settings.singularity_shell, run, and +might provide custom commands, every container exposes unique exec, shell, run, and inspect aliases. For anycommands above, you can export: - SINGULARITY_OPTS: to define custom options for singularity (e.g., --debug)