spack.yaml: depend on generic tracers Spack components #82

harshula · 2024-10-07T04:50:18Z

Opening this PR to create a pre-release build of ACCESS-OM2 with Spack-built generic tracers components for testing.

github-actions · 2024-10-07T05:14:53Z

The model version in the spack.yaml has not been updated.
Either update it manually, or comment the following to have it updated and committed automatically:

!bump major for feature releases
!bump minor for bugfixes

harshula · 2024-10-07T09:15:49Z

!bump major

github-actions · 2024-10-07T09:16:06Z

❌ Failed to bump version or commit changes, see https://github.com/ACCESS-NRI/ACCESS-OM2/actions/runs/11212418716 ❌

CodeGat · 2024-10-07T10:25:21Z

Token has expired. Will regenerate one tomorrow...considering making it have a longer lifespan 😅

aidanheerdegen · 2024-10-07T22:21:36Z

Token has expired. Will regenerate one tomorrow...considering making it have a longer lifespan 😅

Would be good to have a mechanism to remind us when things like this are expiring. Can we generate tokens through an API and set through an API?

github-actions · 2024-10-07T22:23:06Z

🚀 Deploying access-om2 2024.10.0 as prerelease pr82-4

Details and usage instructions

This access-om2 model will be deployed as:

2024.10.0 as a Release (when merged).
pr82-4 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-om2/pr82-4

where the binaries shall be on your $PATH.
This Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-om2- environment.

🛠️ Using: spack 0.22, spack-packages development, spack-config 2024.07.05

Details

It will be deployed using:

access-nri/spack on branch 0.22
access-nri/spack-packages version development
access-nri/spack-config version 2024.07.05

If this is not what was expected, commit changes to config/versions.json.

aidanheerdegen

Just a question at this point.

This is fine for testing, but just flagging we have some fork management discussions/decisons to make for those new components before merging.

aidanheerdegen · 2024-10-07T22:47:10Z

spack.yaml

          cice5: '{name}/2023.10.19'
-          mom5: '{name}/2023.11.09'
+          mom5: '{name}/development'


This made me wonder, would we have namespace clash issues with modules in prerelease if two PRs tried to create modules using the same tag or name?

ping @CodeGat for visibility

I suspect so, now that you mention it...!

Shall we do development_2024.10.08 or dev_2024.10.08?

dev_2024.10.08 is sufficiently descriptive and a little less clunky.

For the future: could we isolate modules into separate namespaces, use the PR name as well?

How do we do that? Just change the name of the module, or something fancy with modules that I don't know?

github-actions · 2024-10-07T23:57:19Z

🚀 Deploying access-om2 2024.10.0 as prerelease pr82-5

Details and usage instructions

This access-om2 model will be deployed as:

2024.10.0 as a Release (when merged).
pr82-5 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-om2/pr82-5

where the binaries shall be on your $PATH.
This Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-om2- environment.

🛠️ Using: spack 0.22, spack-packages development, spack-config 2024.07.05

Details

It will be deployed using:

access-nri/spack on branch 0.22
access-nri/spack-packages version development
access-nri/spack-config version 2024.07.05

If this is not what was expected, commit changes to config/versions.json.

github-actions · 2024-10-08T00:58:46Z

🚀 Deploying access-om2 2024.10.0 as prerelease pr82-6

Details and usage instructions

This access-om2 model will be deployed as:

2024.10.0 as a Release (when merged).
pr82-6 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-om2/pr82-6

where the binaries shall be on your $PATH.
This Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-om2- environment.

🛠️ Using: spack 0.22, spack-packages development, spack-config 2024.10.03

Details

It will be deployed using:

access-nri/spack on branch 0.22
access-nri/spack-packages version development
access-nri/spack-config version 2024.07.05

If this is not what was expected, commit changes to config/versions.json.

CodeGat · 2024-10-08T02:10:55Z

Note: the Prerelease succeeded (and is usable), the workflow failed to collect some metadata as the updates to spack-config move directories around. This PR fixes that for future runs: ACCESS-NRI/build-cd#144

github-actions · 2024-10-09T12:07:25Z

🚀 Deploying access-om2 2024.10.0 as prerelease pr82-7

Details and usage instructions

This access-om2 model will be deployed as:

2024.10.0 as a Release (when merged).
pr82-7 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-om2/pr82-7

where the binaries shall be on your $PATH.
This Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-om2-pr82-7 environment.

🛠️ Using: spack 0.22, spack-packages development, spack-config 2024.10.03

Details

It will be deployed using:

access-nri/spack on branch 0.22
access-nri/spack-packages version development
access-nri/spack-config version 2024.10.03

If this is not what was expected, commit changes to config/versions.json.

harshula · 2024-10-09T22:56:40Z

Hi @chrisb13 , @anton-seaice , @dougiesquire , This is ready for testing.

dougiesquire · 2024-10-09T23:10:23Z

Hi @chrisb13 , @anton-seaice , @dougiesquire , This is ready for testing.

I probably won't do any testing while on leave, but if anyone wants to try run this they can use this config. You'll obviously need to modify the config.yaml to use the prerelease module.

chrisb13 · 2024-10-11T03:24:26Z

Following @dougiesquire suggestion, we've ran a test using this branch (commit 12761ac56fc9c7fa2c80db17f1b0d8c337724ea2). With small modifications to the config.yaml (thanks @jo-basevi!).

The test ran successfully for five years and produced output from ACCESS-OM2 with WOMBAT legacy in generic tracers.

@pearseb also checked the output. NPP screenshot below.

aidanheerdegen · 2024-10-11T04:54:44Z

With small modifications to the config.yaml (thanks @jo-basevi!).

Can we get those modifications into this PR?

The test ran successfully for five years and produced output from ACCESS-OM2 with WOMBAT legacy in generic tracers.

Nice. Was this up for debate?

chrisb13 · 2024-10-14T03:49:08Z

Can we get those modifications into this PR?

The modifications were small but also made to the config repo' (here) so not sure how they'd be incorporated into this PR..?

aidanheerdegen · 2024-10-14T05:52:13Z

Oh sorry, I misspoke. As you quite rightly point out the config repo isn't part of this PR, so ignore me. But you can push changes to that branch, or create a new branch with those changes, to make it easier to test.

…55_ryf_wombatlite (12761ac) for this ACCESS-NRI/ACCESS-OM2#82.

chrisb13 · 2024-10-17T06:55:54Z

But you can push changes to that branch, or create a new branch with those changes, to make it easier to test.

Ok as discussed and above, now on a new branch on my own repo', here:
chrisb13/access-om2-configs@e41c923
Do I put in a pull request? (Does it need to get onto the https://github.com/ACCESS-NRI/access-om2-configs ?)

dougiesquire · 2024-10-17T07:00:34Z

@chrisb13 feel free to just push to the dougiesquire/1deg_jra55_ryf_wombatlite branch (or submit a PR to it)

… tracers in #82

dougiesquire · 2024-10-21T10:13:47Z

@chrisb13, I've just hijacked outdated PR #76 and pushed changes to build a prerelease that hopefully helps with testing:

Prerelease pr76-26 uses git submodule generic tracers (and MOM5/src/shared FMS and git submodule mocsy)
Prerelease pr82-7 uses "spackified" generic tracers (and ACCESS "spackified" FMS and mocsy)

Everything else should be the same between the prereleases.

We want to check whether these two prereleases give the same answers and performance. I've updated my ACCESS-OM2 1deg_jra55_ryf_wombatlite test config to use the pr76-26 prerelease. This will hopefully run as is and then this line can be changed to use the pr82-7 prerelease.

chrisb13 · 2024-10-24T05:49:25Z

@dougiesquire, thanks for clarifying.

I think there's something odd about pr76-26 as a payu setup fails to find the ocean executable..

[cyb561.gadi-login-05: 1deg_jra55_ryf4_dougie]$ payu setup
laboratory path:  /scratch/p73/cyb561/access-om2
binary path:  /scratch/p73/cyb561/access-om2/bin
input path:  /scratch/p73/cyb561/access-om2/input
work path:  /scratch/p73/cyb561/access-om2/work
archive path:  /scratch/p73/cyb561/access-om2/archive
Metadata and UUID generation is disabled. Experiment name used for archival: 1deg_jra55_ryf4_dougie
payu: Found modules in /opt/Modules/v4.3.0
Loading access-om2/pr76-26
  Loading requirement: cice5/2023.10.19 libaccessom2/2023.10.26 mom5/2024.08.23
Loading input manifest: manifests/input.yaml
Loading restart manifest: manifests/restart.yaml
Loading exe manifest: manifests/exe.yaml
Setting up atmosphere
Setting up ocean
Traceback (most recent call last):
  File "/g/data/vk83/apps/payu/1.1.5/bin/payu", line 10, in <module>
    sys.exit(parse())
  File "/g/data/vk83/apps/payu/1.1.5/lib/python3.10/site-packages/payu/cli.py", line 49, in parse
    run_cmd(**args)
  File "/g/data/vk83/apps/payu/1.1.5/lib/python3.10/site-packages/payu/subcommands/setup_cmd.py", line 28, in runcmd
    expt.setup(force_archive=force_archive)
  File "/g/data/vk83/apps/payu/1.1.5/lib/python3.10/site-packages/payu/experiment.py", line 439, in setup
    model.setup()
  File "/g/data/vk83/apps/payu/1.1.5/lib/python3.10/site-packages/payu/models/mom.py", line 66, in setup
    super(Mom, self).setup()
  File "/g/data/vk83/apps/payu/1.1.5/lib/python3.10/site-packages/payu/models/model.py", line 316, in setup
    raise FileNotFoundError(
FileNotFoundError: Executable for ocean model not found on path: /scratch/p73/cyb561/access-om2/bin/fms_ACCESS-OM.x

Working backwards our two commits (yours and mine) are very similar with changes only to the config.yaml (both our commits have the same parent):

Swapping pr76-26 for pr82-7 removes the above error.

Have you been able to run pr76-26?

anton-seaice · 2024-10-24T06:01:22Z

For reasons that are not clear to me there is an fms build named fms_ACCESS-ESM.x in the access-om2/pr76-26 module. So it looks like the binary might be there with the wrong name ? Or the wrong variant was built ?

dougiesquire · 2024-10-24T09:52:46Z

Have you been able to run pr76-26?

@chrisb13 I hadn't tried but trying just now I get the same issue.

Something has gone wrong with the environment module for pr76-26 and it includes MOM binaries from an ACCESS-ESM1.6 prerelease (??). I've pinged the release team for help in pr76.

chrisb13 · 2024-11-07T00:55:55Z

We want to check whether these two prereleases give the same answers

@dougiesquire. An update is here

github-actions · 2024-11-14T03:08:16Z

🚀 Deploying access-om2 2024.10.0 as prerelease pr82-9

Details and usage instructions

This access-om2 model will be deployed as:

2024.10.0 as a Release (when merged).
pr82-9 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-om2/pr82-9

where the binaries shall be on your $PATH.
This Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-om2- environment.

🛠️ Using: spack 0.22, spack-packages development, spack-config 2024.11.11

Details

It will be deployed using:

access-nri/spack on branch 0.22
access-nri/spack-packages version development
access-nri/spack-config version 2024.11.11

If this is not what was expected, commit changes to config/versions.json.

CodeGat · 2024-11-14T03:15:53Z

@harshula note the errors with rebasing on Model Deployment Repositories. The environment pr82-9 will not reflect the changes accurately.

* "spack": "0.22", * "spack-packages": "development",

… 2024.10.0

* This change moves us closer to upstream behaviour. * Remove all the duplication of spack-config's common/modules.yaml.

github-actions · 2024-11-14T11:49:08Z

🚀 Deploying access-om2 2024.10.0 as prerelease pr82-8

Details and usage instructions

This access-om2 model will be deployed as:

2024.10.0 as a Release (when merged).
pr82-8 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-om2/pr82-8

where the binaries shall be on your $PATH.
This Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-om2- environment.

🛠️ Using: spack 0.22, spack-packages development, spack-config 2024.11.11

Details

It will be deployed using:

access-nri/spack on branch 0.22
access-nri/spack-packages version development
access-nri/spack-config version 2024.11.11

If this is not what was expected, commit changes to config/versions.json.

CodeGat · 2024-11-15T01:18:30Z

Hey all, due to:

The update to Prerelease versioning (in which we now use number of deployments rather than number of commits, which allows rebasing), and
The update to Projections in the spack.yaml

Some of the spack environment names may clash for future deployments in this PR. If possible, can we move this to a new PR? Happy to help out if needed.

harshula requested review from chrisb13, aidanheerdegen, dougiesquire, CodeGat and anton-seaice October 7, 2024 04:50

harshula self-assigned this Oct 7, 2024

harshula force-pushed the componentize-generic-tracers branch from 33ff4bd to 7e7a627 Compare October 7, 2024 05:14

CodeGat temporarily deployed to Gadi Prerelease October 7, 2024 22:23 — with GitHub Actions Inactive

aidanheerdegen reviewed Oct 7, 2024

View reviewed changes

CodeGat had a problem deploying to Gadi Prerelease October 7, 2024 23:57 — with GitHub Actions Failure

CodeGat had a problem deploying to Gadi Prerelease October 8, 2024 00:58 — with GitHub Actions Failure

harshula force-pushed the componentize-generic-tracers branch from fb44950 to 799674f Compare October 9, 2024 12:06

harshula temporarily deployed to Gadi Prerelease October 9, 2024 12:07 — with GitHub Actions Inactive

chrisb13 added a commit to chrisb13/access-om2-configs that referenced this pull request Oct 17, 2024

tiny changes made to config.yaml to run test of dougiesquire/1deg_jra…

e41c923

…55_ryf_wombatlite (12761ac) for this ACCESS-NRI/ACCESS-OM2#82.

dougiesquire added a commit that referenced this pull request Oct 21, 2024

Changes to allow direct comparison to build with "spackified" generic…

17a539a

… tracers in #82

CodeGat removed their request for review October 30, 2024 05:07

harshula mentioned this pull request Nov 1, 2024

Spack packages built with different specs have the same modulefile name ACCESS-NRI/build-cd#157

Closed

chrisb13 mentioned this pull request Nov 7, 2024

Test pre-release for MOM5 with generic tracer WOMBAT #76

Closed

harshula force-pushed the componentize-generic-tracers branch from 799674f to ec5e27f Compare November 14, 2024 03:02

harshula had a problem deploying to Gadi Prerelease November 14, 2024 03:08 — with GitHub Actions Failure

CodeGat and others added 8 commits November 14, 2024 22:47

Updated config/versions.json to 3-0-0 (#83)

09306f9

spack.yaml: depend on generic tracers Spack components

ab5c4e7

versions.json: use the latest versions

b36577f

* "spack": "0.22", * "spack-packages": "development",

Removed spack-config from versions.json, bumped access-om2 version to…

fb6afb5

… 2024.10.0

Updated mom5 projection to 'development'

d76279d

Added comments, reset spack-packages to origin/development on target

ce06e8f

spack.yaml: Use tag dev_2024.08.14 instead of branch name

41ad699

Append a hash to the SPR modulefile names

8d7c601

* This change moves us closer to upstream behaviour. * Remove all the duplication of spack-config's common/modules.yaml.

harshula force-pushed the componentize-generic-tracers branch from 174cbac to 8d7c601 Compare November 14, 2024 11:48

harshula temporarily deployed to Gadi Prerelease November 14, 2024 11:49 — with GitHub Actions Inactive

harshula closed this Nov 15, 2024

harshula mentioned this pull request Nov 18, 2024

spack.yaml: depend on generic tracers Spack components #86

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

spack.yaml: depend on generic tracers Spack components #82

spack.yaml: depend on generic tracers Spack components #82

harshula commented Oct 7, 2024

github-actions bot commented Oct 7, 2024

harshula commented Oct 7, 2024

github-actions bot commented Oct 7, 2024

CodeGat commented Oct 7, 2024

aidanheerdegen commented Oct 7, 2024

github-actions bot commented Oct 7, 2024

aidanheerdegen left a comment

aidanheerdegen Oct 7, 2024

CodeGat Oct 7, 2024

harshula Oct 8, 2024

aidanheerdegen Oct 8, 2024

aidanheerdegen Oct 8, 2024

CodeGat Oct 8, 2024

github-actions bot commented Oct 7, 2024

github-actions bot commented Oct 8, 2024 •

edited by CodeGat

Loading

CodeGat commented Oct 8, 2024 •

edited

Loading

github-actions bot commented Oct 9, 2024 •

edited by CodeGat

Loading

harshula commented Oct 9, 2024

dougiesquire commented Oct 9, 2024 •

edited

Loading

chrisb13 commented Oct 11, 2024

aidanheerdegen commented Oct 11, 2024

chrisb13 commented Oct 14, 2024

aidanheerdegen commented Oct 14, 2024

chrisb13 commented Oct 17, 2024

dougiesquire commented Oct 17, 2024

dougiesquire commented Oct 21, 2024

chrisb13 commented Oct 24, 2024 •

edited

Loading

anton-seaice commented Oct 24, 2024

dougiesquire commented Oct 24, 2024 •

edited

Loading

chrisb13 commented Nov 7, 2024 •

edited

Loading

github-actions bot commented Nov 14, 2024

CodeGat commented Nov 14, 2024

github-actions bot commented Nov 14, 2024

CodeGat commented Nov 15, 2024

spack.yaml: depend on generic tracers Spack components #82

spack.yaml: depend on generic tracers Spack components #82

Conversation

harshula commented Oct 7, 2024

github-actions bot commented Oct 7, 2024

harshula commented Oct 7, 2024

github-actions bot commented Oct 7, 2024

CodeGat commented Oct 7, 2024

aidanheerdegen commented Oct 7, 2024

github-actions bot commented Oct 7, 2024

aidanheerdegen left a comment

Choose a reason for hiding this comment

aidanheerdegen Oct 7, 2024

Choose a reason for hiding this comment

CodeGat Oct 7, 2024

Choose a reason for hiding this comment

harshula Oct 8, 2024

Choose a reason for hiding this comment

aidanheerdegen Oct 8, 2024

Choose a reason for hiding this comment

aidanheerdegen Oct 8, 2024

Choose a reason for hiding this comment

CodeGat Oct 8, 2024

Choose a reason for hiding this comment

github-actions bot commented Oct 7, 2024

github-actions bot commented Oct 8, 2024 • edited by CodeGat Loading

CodeGat commented Oct 8, 2024 • edited Loading

github-actions bot commented Oct 9, 2024 • edited by CodeGat Loading

harshula commented Oct 9, 2024

dougiesquire commented Oct 9, 2024 • edited Loading

chrisb13 commented Oct 11, 2024

aidanheerdegen commented Oct 11, 2024

chrisb13 commented Oct 14, 2024

aidanheerdegen commented Oct 14, 2024

chrisb13 commented Oct 17, 2024

dougiesquire commented Oct 17, 2024

dougiesquire commented Oct 21, 2024

chrisb13 commented Oct 24, 2024 • edited Loading

anton-seaice commented Oct 24, 2024

dougiesquire commented Oct 24, 2024 • edited Loading

chrisb13 commented Nov 7, 2024 • edited Loading

github-actions bot commented Nov 14, 2024

CodeGat commented Nov 14, 2024

github-actions bot commented Nov 14, 2024

CodeGat commented Nov 15, 2024

github-actions bot commented Oct 8, 2024 •

edited by CodeGat

Loading

CodeGat commented Oct 8, 2024 •

edited

Loading

github-actions bot commented Oct 9, 2024 •

edited by CodeGat

Loading

dougiesquire commented Oct 9, 2024 •

edited

Loading

chrisb13 commented Oct 24, 2024 •

edited

Loading

dougiesquire commented Oct 24, 2024 •

edited

Loading

chrisb13 commented Nov 7, 2024 •

edited

Loading