Update spack stack create ctr: make specs configurable, remove warn…

…ings, add CI builds (#805)

* Update submodule pointer for spack
* Update configs/containers: all container recipes, extra specs directory with different sets of specs, updated README.md
* Remove invalid/unnecessary settings from container yamls
* CI tests for building containers
* Update documentation for building containers

.github/workflows/ubuntu-ci-containers-x86_64.yaml

@@ -0,0 +1,80 @@
+name: ubuntu-ci-container-x86_64-build
+on:
+  pull_request:
+    # pull request to develop with label
+    branches: [develop]
+    types: [labeled]
+  schedule:
+    - cron: '0 8 * * *'
+  workflow_dispatch:
+    inputs:
+      container:
+        description: 'Container template (input requred, no default value).'
+        required: true
+      specs:
+        description: 'Which specs to add to the template  (input requred, no default value).'
+        required: true
+
+defaults:
+  run:
+    shell: bash
+    #shell: bash -leo pipefail {0}
+
+jobs:
+  ubuntu-ci-container-x86_64-build:
+    # For PRs only if label matches, and for workflow_dispatch and schedule events
+    if: ${{ github.event.label.name == 'container-ci' }} || ${{ github.event_name == 'workflow_dispatch' }} || ${{ github.event_name == 'schedule' }}
+    runs-on: [ubuntu-ci-x86_64]
+
+    steps:
+      - name: checkout
+        uses: actions/checkout@v3
+        with:
+          submodules: true
+
+      # Ensure that ubuntu can run docker
+      - name: ubuntu-docker
+        run: |
+          echo "Checking if user ubuntu can run docker"
+          if id -nG ubuntu | grep -qw docker; then
+            echo "User ubuntu already belongs to group docker"
+          else
+            echo "Adding user ubuntu to group docker"
+            sudo gpasswd -a ubuntu docker
+            newgrp docker
+            docker run hello-world
+          fi
+
+      - name: create-ctr
+        run: |
+          source ./setup.sh
+
+          # Get day of week to set default container for scheduled builds
+          DOW=$(date +%u)
+          # Monday is 1 ... Sunday is 7
+          if [[ $DOW == 1 || $DOW == 4 ]]; then
+            export CONTAINER=${{ inputs.container || 'docker-ubuntu-clang-mpich' }}
+            export SPECS=${{ inputs.specs || 'jedi-ci' }}
+          elif [[ $DOW == 2 || $DOW == 5 ]]; then
+            export CONTAINER=${{ inputs.container || 'docker-ubuntu-gcc-openmpi' }}
+            export SPECS=${{ inputs.specs || 'jedi-ci' }}
+          elif [[ $DOW == 3 || $DOW == 6 ]]; then
+            export CONTAINER=${{ inputs.container || 'docker-ubuntu-intel-impi' }}
+            export SPECS=${{ inputs.specs || 'jedi-ci' }}
+          else
+            # Day 7: The Sabbath of rest
+            echo "Pruning all docker images"
+            docker system prune -a
+            exit 0
+          fi
+
+          export ENVDIR=$PWD/envs/${CONTAINER}
+          spack stack create ctr --container ${CONTAINER} --specs ${SPECS}
+
+          cd ${ENVDIR}
+          # mapl doesn't build with mpich - https://github.com/JCSDA/spack-stack/issues/608
+          if [[ '${CONTAINER}' == 'docker-ubuntu-clang-mpich' ]]; then
+              sed -i 's/- mapl@2.35.2/#- mapl@2.35.2/g' spack.yaml
+          fi
+          spack containerize > Dockerfile
+          docker build -t ${SPECS}-${SPECS} .

configs/containers/README.md

@@ -1,29 +1,6 @@
 ## Overview
 
-To avoid hardcoding specs in the generic container recipes, we keep the specs list empty (`specs: []`) and manually add the specs for the particular spack-stack release and application as listed below, *after* running `spack stack create ctr`.
-
-### spack-stack-1.5.0 / skylab-6.0.0 containers for fv3-jedi and mpas-jedi (but not for ufs-jedi)
-```
-  specs: [base-env@1.0.0, jedi-base-env@1.0.0 ~fftw, ewok-env@1.0.0, jedi-fv3-env@1.0.0,
-    jedi-mpas-env@1.0.0, bacio@2.4.1, bison@3.8.2, bufr@12.0.0, ecbuild@3.7.2, eccodes@2.27.0, ecflow@5,
-    eckit@1.24.4, ecmwf-atlas@0.34.0 +trans ~fftw, fiat@1.2.0, ectrans@1.2.0 ~fftw, eigen@3.4.0,
-    fckit@0.11.0, fms@release-jcsda, g2@3.4.5, g2tmpl@1.10.0, gftl-shared@1.5.0,
-    gsibec@1.1.3, hdf@4.2.15, hdf5@1.14.0, ip@4.3.0, jasper@2.0.32, jedi-cmake@1.4.0,
-    libpng@1.6.37, nccmp@1.9.0.1, netcdf-c@4.9.2, netcdf-cxx4@4.3.1,
-    netcdf-fortran@4.6.0, nlohmann-json@3.10.5, nlohmann-json-schema-validator@2.1.0,
-    parallelio@2.5.10, parallel-netcdf@1.12.2, py-eccodes@1.4.2, py-f90nml@1.4.3,
-    py-gitpython@3.1.27, py-h5py@3.7.0, py-numpy@1.22.3,
-    py-pandas@1.5.3, py-pip, py-pyyaml@5.4.1, py-scipy@1.9.3, py-shapely@1.8.0, py-xarray@2022.3.0,
-    sp@2.3.3, udunits@2.2.28, w3nco@2.4.1, w3emc@2.10.0, nco@5.0.6, esmf@8.4.2, mapl@2.35.2,
-    yafyaml@0.5.1, zlib@1.2.13, zstd@1.5.2, odc@1.4.6, shumlib@macos_clang_linux_intel_port,
-    awscli@1.27.84, py-globus-cli@3.16.0]
-    # Notes:
-    # 1. Remove mapl@2.35.2 from clang/mpich container, because mapl doesn't work with mpich@4
-    # 2. Don't build CRTM by default so that it gets built in the JEDI bundles
-    # 3. Comment out for now until build problems are solved
-    #    https://github.com/jcsda/spack-stack/issues/522
-    #    py-mysql-connector-python@8.0.32
-```
+The list of specs to be installed in the containers is defined in `specs.yaml` and included in the three container recipes. In the future, we will provide the option to specify different specs as a command line argument.
 
 ### Create an AMI on AWS EC2 to build docker containers
 

configs/containers/docker-ubuntu-clang-mpich.yaml

@@ -1,7 +1,6 @@
 spack:
   concretizer:
     unify: true
-  view: false
 
   config:
     checksum: false
@@ -110,7 +109,7 @@ spack:
       os: ubuntu:20.04
       spack:
         url: https://github.com/jcsda/spack
-        ref: spack-stack-1.5.0
+        ref: jcsda_emc_spack_stack
         resolve_sha: false
 
     # Whether or not to strip binaries

configs/containers/docker-ubuntu-gcc-openmpi.yaml

@@ -1,7 +1,6 @@
 spack:
   concretizer:
     unify: true
-  view: false
 
   config:
     checksum: false
@@ -98,7 +97,7 @@ spack:
       os: ubuntu:20.04
       spack:
         url: https://github.com/jcsda/spack
-        ref: spack-stack-1.5.0
+        ref: jcsda_emc_spack_stack
         resolve_sha: false
 
     # Whether or not to strip binaries

configs/containers/docker-ubuntu-intel-impi.yaml

@@ -1,7 +1,6 @@
 spack:
   concretizer:
     unify: true
-  view: false
 
   config:
     checksum: false
@@ -115,7 +114,7 @@ spack:
       os: ubuntu:20.04
       spack:
         url: https://github.com/jcsda/spack
-        ref: spack-stack-1.5.0
+        ref: jcsda_emc_spack_stack
         resolve_sha: false
 
     # Whether or not to strip binaries

configs/containers/specs/jedi-ci.yaml

@@ -0,0 +1,20 @@
+  ### spack-stack-1.5.0 / skylab-6.0.0 containers for fv3-jedi and mpas-jedi (but not for ufs-jedi)
+  specs: [base-env@1.0.0, jedi-base-env@1.0.0 ~fftw, ewok-env@1.0.0, jedi-fv3-env@1.0.0,
+    jedi-mpas-env@1.0.0, bacio@2.4.1, bison@3.8.2, bufr@12.0.0, ecbuild@3.7.2, eccodes@2.27.0, ecflow@5,
+    eckit@1.24.4, ecmwf-atlas@0.34.0 +trans ~fftw, fiat@1.2.0, ectrans@1.2.0 ~fftw, eigen@3.4.0,
+    fckit@0.11.0, fms@release-jcsda, g2@3.4.5, g2tmpl@1.10.0, gftl-shared@1.5.0,
+    gsibec@1.1.3, hdf@4.2.15, hdf5@1.14.0, ip@4.3.0, jasper@2.0.32, jedi-cmake@1.4.0,
+    libpng@1.6.37, nccmp@1.9.0.1, netcdf-c@4.9.2, netcdf-cxx4@4.3.1,
+    netcdf-fortran@4.6.0, nlohmann-json@3.10.5, nlohmann-json-schema-validator@2.1.0,
+    parallelio@2.5.10, parallel-netcdf@1.12.2, py-eccodes@1.4.2, py-f90nml@1.4.3,
+    py-gitpython@3.1.27, py-h5py@3.7.0, py-numpy@1.22.3,
+    py-pandas@1.5.3, py-pip, py-pyyaml@5.4.1, py-scipy@1.9.3, py-shapely@1.8.0, py-xarray@2022.3.0,
+    sp@2.3.3, udunits@2.2.28, w3nco@2.4.1, w3emc@2.10.0, nco@5.0.6, esmf@8.4.2, mapl@2.35.2,
+    yafyaml@0.5.1, zlib@1.2.13, zstd@1.5.2, odc@1.4.6, shumlib@macos_clang_linux_intel_port,
+    awscli@1.27.84, py-globus-cli@3.16.0]
+    # Notes:
+    # 1. Remove mapl@2.35.2 from clang/mpich container, because mapl doesn't work with mpich@4
+    # 2. Don't build CRTM by default so that it gets built in the JEDI bundles
+    # 3. Comment out for now until build problems are solved
+    #    https://github.com/jcsda/spack-stack/issues/522
+    #    py-mysql-connector-python@8.0.32

doc/source/BuildingContainers.rst

@@ -8,7 +8,7 @@ Creating Containers
 Container recipes
 =================
 
-Container recipes are self-contained except that at container creation time (``spack stack create ctr ...``) package information from the common ``packages.yaml`` (or a manually provided version) is added to the ``packages:`` section, and specs are added in the section ``specs`` as defined in the template (default is an empty template with no specs.) Container recipes are defined in ``configs/containers``. Currently available container recipes can be found in ``configs/containers/``.
+Container recipes are self-contained except that at container creation time (``spack stack create ctr ...``) package information from the common ``packages.yaml`` (or a manually provided version) is added to the ``packages:`` section, and specs are added in the section ``specs`` from the specs file provided as an argument. Container recipes are defined in ``configs/containers``, and specs are defined in ``configs/containers/specs``.
 
 **WORK IN PROGRESS**
 
@@ -17,22 +17,19 @@ Container recipes are self-contained except that at container creation time (``s
 Container build example
 -----------------------
 
-In this example, a container is created with an empty template, and specs are added manually. It is also possible to start with a different template, but it is important to know that container builds do not allow for multiple versions of the same package (e.g., ``fms@2022.01`` and ``fms@release-jcsda``), therefore not all templates will work (one can remove certain specs from the build, as long as this does not impact the usability of the container).
+It is important to know that container builds do not allow for multiple versions of the same package (e.g., ``fms@2022.01`` and ``fms@release-jcsda``), which needs to be taken into account when creating the specs in ``configs/containers/specs/*.yaml``.
 
 .. code-block:: console
 
    # See a list of preconfigured containers
    spack stack create ctr -h
 
    # Create container spack definition (spack.yaml) in directory envs/<container-config>
-   spack stack create ctr docker-ubuntu-gcc-openmpi --template=empty
+   spack stack create ctr container=docker-ubuntu-gcc-openmpi --specs=jedi-ci
 
    # Descend into container environment directory
    cd envs/docker-ubuntu-gcc-openmpi
 
-   # Edit config file and add the required specs in section "specs:"
-   emacs spack.yaml
-
    # Docker: create Dockerfile and build container
    # See section "container" in spack.yaml for additional information
    spack containerize > Dockerfile

doc/source/SpackStackExtension.rst

@@ -27,9 +27,9 @@ The full list of options for creating containers is:
 
 .. code-block:: console
 
-   spack stack create ctr [--template TEMPLATE] [--name NAME] [--dir DIR] [--overwrite] [--packages PACKAGES] container
+   spack stack create ctr [--container CONTAINER] [--dir DIR] [--overwrite] [--specs NAME] 
 
-``TEMPLATE`` is identical to creating environments, and ``container`` corresponds to a pre-defined container recipe (see :numref:`Section %s <BuildingContainers>`). For all other options, consult the output of ``spack stack create ctr -h``.
+``CONTAINER`` corresponds to a pre-defined container recipe and ``SPECS`` to a list of packages to be installed (see :numref:`Section %s <BuildingContainers>`). For all other options, consult the output of ``spack stack create ctr -h``.
 
 Following a successful creation of an environment (not a container), and the generation of the ``spack`` ``lmod/tcl`` module files via ``spack module [lmod|tcl] refresh``, the meta modules for compiler, Python interpreter and MPI library are generated with