Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update path for M2_REPLAY_ANA_LOCATION #377

Merged
merged 5 commits into from
Dec 2, 2022

Conversation

sanAkel
Copy link
Collaborator

@sanAkel sanAkel commented Nov 29, 2022

This PR updates:

  • M2_REPLAY_ANA_LOCATION
  • REPLAY_FILE in AGCM.rc.tmpl

The reason for this update is because the current:
M2_REPLAY_ANA_LOCATION ➡️ /discover/nobackup/projects/gmao/share/gmao_ops/verification/MERRA-2/ana/ which only goes up to Y2021/M01. The files are NOT up to date.

Per @rlucches (via @danholdaway), a good alternative that is kept current is:
/discover/nobackup/projects/gmao/merra2/data/ana/MERRA2_all which is what this PR provides.

This update has been tested.

@sanAkel sanAkel requested a review from a team as a code owner November 29, 2022 23:47
@sanAkel sanAkel requested review from sdrabenh and mathomp4 November 29, 2022 23:47
@sanAkel sanAkel self-assigned this Nov 29, 2022
@sanAkel sanAkel added bugfix This pull request fixes a bug enhancement New feature or request labels Nov 29, 2022
@rlucches
Copy link

rlucches commented Nov 30, 2022 via email

@sanAkel
Copy link
Collaborator Author

sanAkel commented Nov 30, 2022

You can use this directory instead: /discover/nobackup/projects/gmao/merra2/data/ana/MERRA2_all/

@rlucches, this is precisely what this PR updates our scripts to use. And I have tested to make sure it works.
So I think all is good. In any case @sdrabenh and/or @mathomp4 whom I requested to review- would double check.

Thanks!

AGCM.rc.tmpl Outdated
@@ -273,7 +273,7 @@ RECORD_REF_TIME: >>>REFTIME<<< >>>FCSTIME<<<
#M2 REPLAY_ANA_EXPID: MERRA-2
#M2 REPLAY_ANA_LOCATION: @M2_REPLAY_ANA_LOCATION
#M2 REPLAY_MODE: Regular
#M2 REPLAY_FILE: ana/Y%y4/M%m2/MERRA-2.ana.eta.%y4%m2%d2_%h2z.nc4
#M2 REPLAY_FILE: /discover/nobackup/projects/gmao/merra2/data/ana/MERRA2_all/Y%y4/M%m2/MERRA2.ana.eta.%y4%m2%d2_%h2z.nc4
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sanAkel I'm not sure this needs to be a full path. I want to say the model uses REPLAY_ANA_LOCATION to prepend to REPLAY_FILE. So this probably should only be...uh...maybe:

#M2 REPLAY_FILE Y%y4/M%m2/MERRA2.ana.eta.%y4%m2%d2

Dunno. Probably needs testing.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: It's done this way because the files are in a different place at NAS...and I now realize we probably need to do something at NAS for this. I'll talk with @bena-nasa ...

Copy link
Collaborator Author

@sanAkel sanAkel Nov 30, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mathomp4 The "prepend" did not work for me. I tested withOUT full path (in REPLAY_FILE) and at runtime it was looking for files like: Y%y4/M%m2/MERRA2.ana.eta.%y4%m2%d2_%h2z.nc4. My test was at NCCS.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sanAkel Hmm. Well need to to figure it out otherwise we break Replay at NAS. I'll try and see if I can figure it out.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mathomp4 line 2641 of /discover/nobackup/sakella/test1/slurm-66111020.out may help!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay. I think it's done and seems to work. Apparently we need REPLAY_FILE to start with ana/

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh. And I pushed the changes. Give them a try when you can.

@mathomp4 mathomp4 added 0 diff The changes in this pull request have verified to be zero-diff with the target branch. Contingent - DNA Do Not Approve (DNA). These changes are contingent on other PRs labels Nov 30, 2022
@mathomp4
Copy link
Member

I've added a blocker label until @bena-nasa and I can fix up the files at NAS to work with this new location paradigm.

Copy link
Collaborator Author

@sanAkel sanAkel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mathomp4 I tried, it works with your changes.

@bena-nasa
Copy link
Collaborator

All the data should be in the new place at NAS with symlinks in the place the data was sitting so as to not break any backwards compatibility with older tags

@github-actions
Copy link

github-actions bot commented Dec 1, 2022

Label error. Requires exactly 0 of: Contingent - DNA, Needs Lead Approval, Contingent -- Do Not Approve. Found: enhancement, bugfix, 0 diff, Contingent - DNA

@github-actions
Copy link

github-actions bot commented Dec 2, 2022

Label error. Requires exactly 0 of: Contingent - DNA, Needs Lead Approval, Contingent -- Do Not Approve. Found: enhancement, bugfix, 0 diff, Contingent - DNA

@mathomp4 mathomp4 removed the Contingent - DNA Do Not Approve (DNA). These changes are contingent on other PRs label Dec 2, 2022
@mathomp4 mathomp4 self-requested a review December 2, 2022 16:30
Copy link
Member

@mathomp4 mathomp4 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approve!

@sdrabenh sdrabenh merged commit e8a40a7 into develop Dec 2, 2022
@sdrabenh sdrabenh deleted the update/sanAkel/Replay-MERRA-2_path branch December 2, 2022 16:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0 diff The changes in this pull request have verified to be zero-diff with the target branch. bugfix This pull request fixes a bug enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants