[INSTALL]: g2tmpl/1.12.0

[Hang-Lei-NOAA]

Package name

g2tmpl

Package version/tag

1.12.0

Build options

current

Installation timeframe

As soon as possible.
Already tested on Hera.

Other information

Hera, Orion, Jet, Hercules, Gaea, S4 under spack-stack-1.6.0. Build under UPP template.

Locations:

• Hera: /scratch1/NCEPDEV/nems/role.epic/spack-stack/spack-stack-1.6.0/envs/upp-addon-env/install/modulefiles/Core
• Jet: /mnt/lfs4/HFIP/hfv3gfs/role.epic/spack-stack/spack-stack-1.6.0/envs/upp-addon-env/install/modulefiles/Core
• Orion: /work/noaa/epic/role-epic/spack-stack/orion/spack-stack-1.6.0/envs/upp-addon-env/install/modulefiles/Core
• Hercules: /work/noaa/epic/role-epic/spack-stack/hercules/spack-stack-1.6.0/envs/upp-addon-env/install/modulefiles/Core
• Gaea: /autofs/ncrc-svm1_proj/epic/spack-stack/spack-stack-1.6.0/envs/upp-addon-env/install/modulefiles/Core

WCOSS2

• Check this box if and only if your package should be installed on WCOSS2 Cactus and Dogwood (all spack-stack packages will be installed on Acorn). If not, you may disregard the rest of the items below and submit this request.

WCOSS2: General questions

No response

WCOSS2: Installation and testing

No response

WCOSS2: Technical & security review list

• The code is mature, stable, and production ready
• The code is does not and cannot use the internet, and does not contain URLs (http, https, ftp, etc.) except in comments
• The package does not contain prebuilt binary files that have not been approved by NCO security review
• The code has no publicly disclosed cybersecurity vulnerabilities and exposures (search https://cve.mitre.org/cve/)
• The code is not prohibited by DHS, DOC, NOAA, or NWS
• The code comes from a trusted source. Trusted sources include other NWS, NOAA, or DOC, agencies, or other Federal agencies that operate at a FISMA high or equivalent level. Additionally, trusted sources could be third-party agencies through which there is an existing SLA on file (such as RedHat).
• The code is actively maintained and supported (it continues to get updates, patches, etc.)
• The code is not maintained by a private entity operating in a foreign country (if it is, make a note below)
• There is sufficient documentation to support maintenance
• There are no known security vulnerabilities or weaknesses
• Installing and running the code does not require privileged processes/users
• There are no software dependencies that are unapproved or have security concerns (if there are, make a note below)
• There are no concerns related to SA, SI, and SC NIST control families

WCOSS2: Additional comments

No response

[WenMeng-NOAA]

Two UPP PRs have depended on the installation of g2tmpl/1.12.0 on R&D platforms to meet RRFS scientific code freezing deadline at the beginning of May. I could assist with on verifying g2tmpl in UPP testing. Thanks!

[AlexanderRichert-NOAA]

I've created a subset of the spack-stack 1.6.0 unified env on Hera with g2tmpl@1.12.0: /scratch1/NCEPDEV/nems/role.epic/spack-stack/spack-stack-1.6.0/envs/upp-addon-env/install/modulefiles/Core
Use that instead of the unified-env-rocky8 environment; please verify that it works and in the meantime I'll be working on Orion and Jet.

[WenMeng-NOAA]

I've created a subset of the spack-stack 1.6.0 unified env on Hera with g2tmpl@1.12.0: /scratch1/NCEPDEV/nems/role.epic/spack-stack/spack-stack-1.6.0/envs/upp-addon-env/install/modulefiles/Core Use that instead of the unified-env-rocky8 environment; please verify that it works and in the meantime I'll be working on Orion and Jet.

@AlexanderRichert-NOAA Can you install prod_util/2.1.1 under the subset of the spack-stack 1.6.0?

[AlexanderRichert-NOAA]

Yes, done

[WenMeng-NOAA]

@AlexanderRichert-NOAA Your subset of the spack-stack 1.6.0 works well in UPP. I confirm that expected changes from UPP PR #917 and #929 are generated with g2tmpl/1.12.0. Could you please prioritize the installation on Hera, Jet, Orion, eventually propagate it to Hercules, Gaea and S4? Thanks!

[AlexanderRichert-NOAA]

@climbfuji would you be able to do this on S4, or do you know someone who could? Basically, under spack-stack-1.6.0:

• spack stack create env --site orion --template empty --name upp-addon-env --modify-pkg g2tmpl --upstream /data/prod/jedi/spack-stack/spack-stack-1.6.0/envs/unified-env/install/
• spack edit g2tmpl to add version("1.12.0", sha256="44272be7bde8da05565255a8ecdbd080c659d7f0669e356e1c8fef6bac05e723")
• use an empty template and add:
  • upp-env ^g2tmpl@1.12.0 %intel
  • prod-util@2.1.1 %intel
  • ip %intel
  • grib-util %intel
  • wgrib2 %intel
• the rest as usual for a chained env

[AlexanderRichert-NOAA]

@WenMeng-NOAA I've done all but S4, which I don't have access to. Do you want Acorn as well?

[WenMeng-NOAA]

@WenMeng-NOAA I've done all but S4, which I don't have access to. Do you want Acorn as well?

@AlexanderRichert-NOAA Where can I find g2tmpl/1.12.0 on Hera? Under /scratch1/NCEPDEV/nems/role.epic/spack-stack/spack-stack-1.6.0/envs/unified-env-rocky8/install/modulefiles/Core, or other path?
I loaded the spack-stack 1.6.0 as following, but only see g2tmpl/1.10.2

module use /scratch1/NCEPDEV/nems/role.epic/spack-stack/spack-stack-1.6.0/envs/unified-env-rocky8/install/modulefiles/Core
module load stack-intel/2021.5.0
module load stack-intel-oneapi-mpi/2021.5.1

[AlexanderRichert-NOAA]

Replace 'unified-env-rocky8' with 'upp-addon-env'

[WenMeng-NOAA]

Replace 'unified-env-rocky8' with 'upp-addon-env'

@AlexanderRichert-NOAA So we should use the subset 'upp-addon-env' (not 'unified-env-rocky8') of the spack-stack 1.6.0 to load g2tmp/1.12.0 on all R&D platforms?

[AlexanderRichert-NOAA]

And all of the other packages, yes. Only upp-addon-env should be in your $MODULEPATH.

[climbfuji]

@climbfuji would you be able to do this on S4, or do you know someone who could? Basically, under spack-stack-1.6.0:

• spack stack create env --site orion --template empty --name upp-addon-env --modify-pkg g2tmpl --upstream /data/prod/jedi/spack-stack/spack-stack-1.6.0/envs/unified-env/install/

• spack edit g2tmpl to add version("1.12.0", sha256="44272be7bde8da05565255a8ecdbd080c659d7f0669e356e1c8fef6bac05e723")

• use an empty template and add:

  • upp-env ^g2tmpl@1.12.0 %intel
  • prod-util@2.1.1 %intel
  • ip %intel

• the rest as usual for a chained env

Yes I can do this later today - sorry for the wait

[climbfuji]

Done on S4. Who would be able to test if I got it all right?

[AlexanderRichert-NOAA]

@WenMeng-NOAA I added wgrib2 and grib-util on all systems but Orion (not accessible right now) and S4.

[WenMeng-NOAA]

Done on S4. Who would be able to test if I got it all right?

I don't have S4 access, but might ask for someone at EMC do an UPP build testing.

[WenMeng-NOAA]

Done on S4. Who would be able to test if I got it all right?

@climbfuji Innocent Souopgui confirmed that your installation working in UPP on s4.
Thanks for installation. @AlexanderRichert-NOAA @climbfuji

[climbfuji]

Thanks for checking. Closing as done.

[natalie-perlin]

@climbfuji would you be able to do this on S4, or do you know someone who could? Basically, under spack-stack-1.6.0:

• spack stack create env --site orion --template empty --name upp-addon-env --modify-pkg g2tmpl --upstream /data/prod/jedi/spack-stack/spack-stack-1.6.0/envs/unified-env/install/

• spack edit g2tmpl to add version("1.12.0", sha256="44272be7bde8da05565255a8ecdbd080c659d7f0669e356e1c8fef6bac05e723")

• use an empty template and add:

  • upp-env ^g2tmpl@1.12.0 %intel
  • prod-util@2.1.1 %intel
  • ip %intel
  • grib-util %intel
  • wgrib2 %intel

• the rest as usual for a chained env

What does it mean "use an empty template"?

I'm currently installing on NOAA CSPs. I've added the following
to
./envs/upp-addon-env/spack.yaml:
specs:

    - upp-env ^g2tmpl@1.12.0 %intel
    - prod-util@2.1.1  %intel
    - ip %intel

Concretization step does not go as expected though, and spack.lock is not created.
The error message:

[upp-addon-env] [root@NOAA-AWS:/contrib/spack-stack/spack-stack-1.6.0/envs/upp-addon-env]# spack concretize 2>&1 | tee log.upp-addon-env.concretize
==> Error: concretization failed for the following reasons:

   1. Cannot satisfy 'g2tmpl@1.12.0'
   2. Cannot satisfy 'g2tmpl@1.12.0'
        required because upp-env ^g2tmpl@1.12.0%intel requested from CLI 

If spack.yaml is changed to:

  specs:
    - g2tmpl@1.12.0 %intel
    - prod-util@2.1.1  %intel
    - ip %intel

it still produces the error:

[upp-addon-env] [root@NOAA-AWS:/contrib/spack-stack/spack-stack-1.6.0/envs/upp-addon-env]# spack concretize 2>&1 | tee log.upp-addon-env.concretize.3
==> Error: concretization failed for the following reasons:

   1. Cannot satisfy 'g2tmpl@1.12.0'
   2. Cannot satisfy 'g2tmpl@1.12.0'
        required because g2tmpl@1.12.0%intel requested from CLI 

Does the packages need to be listed in a different location that spack.yaml?
./common/packages.yaml (a template?) is not empty, it contains all the packages for the spack-stack-1.6.0. Do I need to replace g2tmpl version listed in ./common/packages.yaml, which was 1.10.2?

The environment was created with the following:
spack stack create env --site noaa-aws --template empty --name upp-addon-env --modify-pkg g2tmpl --upstream /contrib/spack-stack/spack-stack-1.6.0/envs/unified-env/install

[climbfuji]

Empty template means --template empty as you did. Following the steps shown above worked for me on S4, not sure what is different on your system. @AlexanderRichert-NOAA @ulmononian please help with this, thanks.

[ulmononian]

@natalie-perlin what does your spack.yaml look like? you can always use a different template and then remove all unneeded compilers, specs, etc.. maybe concretize is having trouble because the spack.yaml is not configured properly.

[natalie-perlin]

Used another test environment created in an identical way to verify steps. The spack.yaml created:

# spack-stack hash: 0c2ff3b
# spack hash: 595a31b
spack:
  concretizer:
    unify: when_possible

  config:
    install_tree:
      root: $env/install
  modules:
    default:
      roots:
        lmod: $env/install/modulefiles
        tcl: $env/install/modulefiles

  view: false
  include:
  - site
  - common

  specs: []
  upstreams:
    spack-stack-1.6.0-unified-env:
      install_tree: /contrib/spack-stack/spack-stack-1.6.0/envs/unified-env/install
  repos: [$env/envrepo]

A ./upp-addon-env/common/packages.yaml that is created is identical to that from the ./unified-env/common/packages.yaml

Still trying out various solutions. Please let me know if something in these steps seem to be out of order!..

[natalie-perlin]

@ulmononian -
Concretization steps completed properly when attempted to repeat the steps, and used the following modified spack.yaml as shown below. It is being install-ed on AWS now.

# spack-stack hash: 0c2ff3b
# spack hash: 595a31b
spack:
  concretizer:
    unify: when_possible

  config:
    install_tree:
      root: $env/install
  modules:
    default:
      roots:
        lmod: $env/install/modulefiles
        tcl: $env/install/modulefiles

  view: false
  include:
  - site
  - common

  specs:
    - upp-env ^g2tmpl@1.12.0 %intel
    - prod-util@2.1.1  %intel
    - ip %intel
   
  upstreams:
     spack-stack-1.6.0-unified-env:
       install_tree: /contrib/spack-stack/spack-stack-1.6.0/envs/unified-env/install
  repos: [$env/envrepo]