Skip to content

Latest commit

 

History

History
81 lines (50 loc) · 5.51 KB

README.md

File metadata and controls

81 lines (50 loc) · 5.51 KB
Raw

model-deployment-template

A template repository for the deployment of spack-based models.

Note

Feel free to replace this README with information on the model once the TODOs have been ticked off.

Things TODO to get your model deployed

Settings

Repository Settings

Branch protections should be set up on main and the special backport/*.* branches, which are used for backporting of fixes to major releases (the YEAR.MONTH portion of the YEAR.MONTH.MINOR version) of models.

Repository Secrets/Variables

There are a few secrets and variables that must be set at the repository level.

Repository Secrets
  • BUILD_DB_CONNECTION_STR: A postgresql connection url to the release provenance database
  • GH_COMMIT_CHECK_TOKEN: GitHub Token that allows workflows to run based on workflow-authored commits (in the case where a user uses !bump commands in PRs that bumps the version of the model)
Repository Variables
  • BUILD_DB_PACKAGES: List of spack packages that are model components that will be uploaded to the release provenance database
  • NAME: which corresponds to the model name - which is usually the repository name
  • CONFIG_VERSIONS_SCHEMA_VERSION: Version of the config/versions.json schema used in this repository
  • SPACK_YAML_SCHEMA_VERSION: Version of the ACCESS-NRI-style spack.yaml schema used in this repository

Environment Secrets/Variables

GitHub Environments are sets of variables and secrets that are used specifically to deploy software, and hence have more security requirements for their use.

Currently, we have two Environments per deployment target - one for Release and one for Prerelease. Our current list of deployment targets and Environments can be found in this deployment configuration file in build-cd.

In order to deploy to a given deployment target:

  • Environments with the name of the deployment target must be created in this repository and have the associated secrets/variables set (see below)
  • There must be a Prerelease Environment associated with the Release Environment. For example, if we are deploying to SUPERCOMPUTER, we require Environments with the names SUPERCOMPUTER, SUPERCOMPUTER Prerelease.

When setting the environment up, remember to require sign off by a member of ACCESS-NRI when deploying as a Release.

Regarding the secrets and variables that must be created:

Environment Secrets
  • HOST: The deployment location SSH Host
  • HOST_DATA: The deployment location SSH Host for data transfer (may be the same as HOST)
  • SSH_KEY: A SSH Key that allows access to the above HOST/HOST_DATA
  • USER: A Username to login to the above HOST/HOST_DATA
Environment Variables
  • DEPLOYMENT_TARGET: Name of the deployment target for logging purposes
  • SPACK_INSTALLS_ROOT_LOCATION: Path to the directory that contains all versions of a deployment of spack. For example, if /some/apps/spack is the SPACK_INSTALLS_ROOT_LOCATION, that directory will contain directories like 0.20, 0.21, 0.22, which in turn contain an install of spack, spack-packages and spack-config
  • SPACK_YAML_LOCATION: Path to a directory that will contain the spack.yaml from this repository during deployment
  • (Optional) SPACK_INSTALL_PARALLEL_JOBS: Explicit number of parallel jobs for the installation of the given model. Must be either of the form --jobs N or unset (for the default --jobs 16).

File Modifications

In .github/workflows

  • Reminder that these workflows use vars.NAME (as well as inherit the above environment secrets) and hence these must be set.
  • If the name of the root SBD for the model (in spack-packages) is different from the model name (for example, ACCESS-ESM1.5s root SBD is access-esm1p5), you must uncomment and set the jobs.[pr-ci|pr-comment].with.root-sbd line to the appropriate SBD name.

In config/versions.json

  • .spack must be given a version. For example, it will clone the associated releases/VERSION branch of ACCESS-NRI/spack if you give it VERSION.
  • .spack-packages should also have a CalVer-compliant tag as the version. See the associated repo for a list of available tags.

In spack.yaml

There are a few TODOs for the spack.yaml:

  • spack.specs: Set the root SBD as the only element of spack.specs. This must also have an @git.YEAR.MONTH.MINOR version as it is the version of the entire deployment (and indeed will be a tag in this repository).
  • spack.packages.*: In this section, you can specify the versions and variants of dependencies. Note that the first element of the spack.packages.*.require must be only a version. Variants and other configuration can be done on subsequent lines.
  • spack.packages.all: Can set configuration for all packages. For example, the compiler used, or the target architecture.
  • spack.modules.default.tcl.include: List of package names that will be explicitly included and available to module load.
  • spack.modules.default.tcl.projections: For included modules, you must set the name of the module to be the same as the spack.packages.*.require[0] version, without the @git..