Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove need for a ice_prod dependency check script #2809

Merged

Conversation

EricSinsky-NOAA
Copy link
Contributor

@EricSinsky-NOAA EricSinsky-NOAA commented Aug 6, 2024

Description

The main purpose of this PR is to remove the need for an ice_prod dependency check script ush/check_ice_netcdf.sh. The original purpose of the ice_prod dependency check script is to check for special case dependencies where ( cyc + FHMIN ) % FHOUT_ICE )) =! 0 (more details on this issue can be found in issue #2674 ). A bugfix for these special cases is expected to come from a PR in the ufs-weather-model.

Resolves #2721
Refs #2721, #2674

Type of change

  • New feature (adds functionality)

Change characteristics

  • Is this a breaking change (a change in existing functionality)? NO
  • Does this change require a documentation update? NO
  • Does this change require an update to any of the following submodules? NO

How has this been tested?

  • Clone, built and tested on WCOSS2.
  • In the WCOSS2 test, FHOUT_OCN and FHOUT_ICE was set to 24 and the GEFS CI case was successfully tested (202103232 12Z).
  • TODO: Perform test cases with FHOUT_OCN and FHOUT_ICE set to 6.

Checklist

  • Any dependent changes have been merged and published
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • I have made corresponding changes to the documentation if necessary

EricSinsky-NOAA and others added 22 commits June 3, 2024 15:16
The linters.yaml has been updated so that shellnorms can run whenever a push occurs to EricSinsky-NOAA/develop.
The changes to linters.yaml have been reverted.
The check_ice_netcdf dependency has been removed for the ice_prod task. The rocoto age datadep dependency has been restored for the ice_prod task.
The check_ice_netcdf script has been deleted.
A function has been added to postdet to calculate a CICE average from 6-hour averaged CICE forecast data.
The cdo utility is used to perform the time averaging in the cice_avg function.
The FHOUT_ICE_6h has been removed and has been replaced with the already defined variable FHOUT_ICE.
ush/forecast_postdet.sh Fixed Show fixed Hide fixed
ush/forecast_postdet.sh Fixed Show fixed Hide fixed
ush/forecast_predet.sh Fixed Show fixed Hide fixed
ush/forecast_predet.sh Fixed Show fixed Hide fixed
echo "SUB ${FUNCNAME[0]}: No further averaging needed for CICE"
else
echo "SUB ${FUNCNAME[0]}: Averaging output data for CICE"
CDO=/apps/prod/hpc-stack/intel-19.1.3.304/cdo/1.9.8/bin/cdo
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we choose to use the cdo utility for this, it may need to be added somewhere in jobs/JGLOBAL_FORECAST. This line is just a temporary workaround for the cdo utility being undefined in the forecast task environment.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The cdo binary directory is already added to the path by the cdo module. You should be able to just call it with cdo without this.

Copy link
Contributor

@WalterKolczynski-NOAA WalterKolczynski-NOAA Aug 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NB: We have it in the base modulefile used for most jobs, but the model modulefile isn't loading cdo, so it would need to be added to load_ufswm_modules.sh.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @WalterKolczynski-NOAA. Will module load cdo in load_ufswm_modules.sh.

parm/config/gefs/config.fcst Fixed Show fixed Hide fixed
parm/config/gefs/config.fcst Fixed Show fixed Hide fixed
parm/config/gfs/config.fcst Fixed Show fixed Hide fixed
parm/config/gfs/config.fcst Fixed Show fixed Hide fixed
@emcbot emcbot added CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress CI-Orion-Running **Bot use only** CI testing on Orion for this PR is in-progress and removed CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera CI-Orion-Building **Bot use only** CI testing is cloning/building on Orion labels Sep 20, 2024
@emcbot
Copy link

emcbot commented Sep 20, 2024

Experiment C48_ATM FAILED on Orion in Build# 2 with error logs:

/work2/noaa/stmp/CI/ORION/2809/RUNTESTS/COMROOT/C48_ATM_6c8548e3/logs/2021032312/gfsgenesis.log
/work2/noaa/stmp/CI/ORION/2809/RUNTESTS/COMROOT/C48_ATM_6c8548e3/logs/2021032312/gfstracker.log

Follow link here to view the contents of the above file(s): (link)

@emcbot emcbot added CI-Orion-Failed **Bot use only** CI testing on Orion for this PR has failed and removed CI-Orion-Running **Bot use only** CI testing on Orion for this PR is in-progress labels Sep 20, 2024
@emcbot
Copy link

emcbot commented Sep 20, 2024

Experiment C48_ATM FAILED on Orion in Build# 2 in
/work2/noaa/stmp/CI/ORION/2809/RUNTESTS/EXPDIR/C48_ATM_6c8548e3

@emcbot
Copy link

emcbot commented Sep 20, 2024

Experiment C48_S2SWA_gefs FAILED on Orion in Build# 2 in
/work2/noaa/stmp/CI/ORION/2809/RUNTESTS/EXPDIR/C48_S2SWA_gefs_6c8548e3

@emcbot
Copy link

emcbot commented Sep 20, 2024

Experiment C48_S2SW FAILED on Orion in Build# 2 with error logs:

/work2/noaa/stmp/CI/ORION/2809/RUNTESTS/COMROOT/C48_S2SW_6c8548e3/logs/2021032312/gfsgenesis.log
/work2/noaa/stmp/CI/ORION/2809/RUNTESTS/COMROOT/C48_S2SW_6c8548e3/logs/2021032312/gfstracker.log

Follow link here to view the contents of the above file(s): (link)

@emcbot
Copy link

emcbot commented Sep 20, 2024

Experiment C48_S2SW FAILED on Orion in Build# 2 in
/work2/noaa/stmp/CI/ORION/2809/RUNTESTS/EXPDIR/C48_S2SW_6c8548e3

@emcbot
Copy link

emcbot commented Sep 20, 2024

Experiment C96C48_hybatmDA FAILED on Orion in Build# 2 with error logs:

/work2/noaa/stmp/CI/ORION/2809/RUNTESTS/COMROOT/C96C48_hybatmDA_6c8548e3/logs/2021122100/gfsgenesis.log
/work2/noaa/stmp/CI/ORION/2809/RUNTESTS/COMROOT/C96C48_hybatmDA_6c8548e3/logs/2021122100/gfstracker.log

Follow link here to view the contents of the above file(s): (link)

@emcbot
Copy link

emcbot commented Sep 20, 2024

Experiment C96C48_hybatmDA FAILED on Orion in Build# 2 in
/work2/noaa/stmp/CI/ORION/2809/RUNTESTS/EXPDIR/C96C48_hybatmDA_6c8548e3

@WalterKolczynski-NOAA
Copy link
Contributor

Ignore the Orion failure. We already know what that is.

@WalterKolczynski-NOAA WalterKolczynski-NOAA removed the CI-Orion-Failed **Bot use only** CI testing on Orion for this PR has failed label Sep 20, 2024
@emcbot
Copy link

emcbot commented Sep 21, 2024

Experiment C96_atm3DVar FAILED on Orion in Build# 2 with error logs:

/work2/noaa/stmp/CI/ORION/2809/RUNTESTS/COMROOT/C96_atm3DVar_6c8548e3/logs/2021122100/gfsgenesis.log
/work2/noaa/stmp/CI/ORION/2809/RUNTESTS/COMROOT/C96_atm3DVar_6c8548e3/logs/2021122100/gfstracker.log

Follow link here to view the contents of the above file(s): (link)

@emcbot emcbot added the CI-Orion-Failed **Bot use only** CI testing on Orion for this PR has failed label Sep 21, 2024
@emcbot
Copy link

emcbot commented Sep 21, 2024

Experiment C96_atm3DVar FAILED on Orion in Build# 2 in
/work2/noaa/stmp/CI/ORION/2809/RUNTESTS/EXPDIR/C96_atm3DVar_6c8548e3

@emcbot emcbot added CI-Orion-Failed **Bot use only** CI testing on Orion for this PR has failed and removed CI-Orion-Failed **Bot use only** CI testing on Orion for this PR has failed labels Sep 21, 2024
@emcbot
Copy link

emcbot commented Sep 21, 2024

CI Failed on Orion in Build# 2
Built and ran in directory /work2/noaa/stmp/CI/ORION/2809


Experiment C48_ATM_6c8548e3 Terminated with 0
FAIL
FAIL tasks failed and 2 dead at Fri Sep 20 01:04:13 PM CDT 2024
Experiment C48_ATM_6c8548e3 Terminated: *FAIL*
Error logs:
/work2/noaa/stmp/CI/ORION/2809/RUNTESTS/COMROOT/C48_ATM_6c8548e3/logs/2021032312/gfsgenesis.log
/work2/noaa/stmp/CI/ORION/2809/RUNTESTS/COMROOT/C48_ATM_6c8548e3/logs/2021032312/gfstracker.log
Experiment C48_S2SWA_gefs_6c8548e3 Terminated with 0 tasks failed and 0 dead at Fri Sep 20 01:18:23 PM CDT 2024
Experiment C48_S2SWA_gefs_6c8548e3 Terminated: *UNAVAILABLE*
Experiment C48_S2SW_6c8548e3 Terminated with 0
FAIL
FAIL tasks failed and 2 dead at Fri Sep 20 01:53:16 PM CDT 2024
Experiment C48_S2SW_6c8548e3 Terminated: *FAIL*
Error logs:
/work2/noaa/stmp/CI/ORION/2809/RUNTESTS/COMROOT/C48_S2SW_6c8548e3/logs/2021032312/gfsgenesis.log
/work2/noaa/stmp/CI/ORION/2809/RUNTESTS/COMROOT/C48_S2SW_6c8548e3/logs/2021032312/gfstracker.log
Experiment C96C48_hybatmDA_6c8548e3 Terminated with 0
FAIL
FAIL tasks failed and 2 dead at Fri Sep 20 03:42:50 PM CDT 2024
Experiment C96C48_hybatmDA_6c8548e3 Terminated: *FAIL*
Error logs:
/work2/noaa/stmp/CI/ORION/2809/RUNTESTS/COMROOT/C96C48_hybatmDA_6c8548e3/logs/2021122100/gfsgenesis.log
/work2/noaa/stmp/CI/ORION/2809/RUNTESTS/COMROOT/C96C48_hybatmDA_6c8548e3/logs/2021122100/gfstracker.log
Experiment C96_atm3DVar_6c8548e3 Terminated with 0
FAIL
FAIL tasks failed and 2 dead at Fri Sep 20 10:13:54 PM CDT 2024
Experiment C96_atm3DVar_6c8548e3 Terminated: *FAIL*
Error logs:
/work2/noaa/stmp/CI/ORION/2809/RUNTESTS/COMROOT/C96_atm3DVar_6c8548e3/logs/2021122100/gfsgenesis.log
/work2/noaa/stmp/CI/ORION/2809/RUNTESTS/COMROOT/C96_atm3DVar_6c8548e3/logs/2021122100/gfstracker.log

@emcbot emcbot added CI-Wcoss2-Passed **Bot use only** CI testing on WCOSS for this PR has completed successfully and removed CI-Wcoss2-Running **Bot use only** CI testing on WCOSS for this PR is in-progress labels Sep 21, 2024
@emcbot
Copy link

emcbot commented Sep 21, 2024

All CI Test Cases Passed on Wcoss2:

Experiment C48_ATM_6c8548e3 *** SUCCESS *** at 09/20/24 05:14:08 PM
Experiment C48_S2SW_6c8548e3 *** SUCCESS *** at 09/20/24 05:35:11 PM
Experiment C96C48_hybatmDA_6c8548e3 *** SUCCESS *** at 09/20/24 06:28:31 PM
Experiment C96C48_hybatmaerosnowDA_6c8548e3 *** SUCCESS *** at 09/20/24 07:00:28 PM
Experiment C96C48_ufs_hybatmDA_6c8548e3 *** SUCCESS *** at 09/20/24 07:56:23 PM
Experiment C96_atm3DVar_extended_6c8548e3 *** SUCCESS *** at 09/21/24 03:14:31 AM

@WalterKolczynski-NOAA WalterKolczynski-NOAA removed the CI-Orion-Failed **Bot use only** CI testing on Orion for this PR has failed label Sep 21, 2024
@emcbot emcbot added CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully and removed CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress labels Sep 21, 2024
@emcbot
Copy link

emcbot commented Sep 21, 2024

CI Passed on Hera in Build# 3
Built and ran in directory /scratch1/NCEPDEV/global/CI/2809


Experiment C48mx500_3DVarAOWCDA_6c8548e3 Completed 2 Cycles: *SUCCESS* at Fri Sep 20 20:11:45 UTC 2024
Experiment C48_ATM_6c8548e3 Completed 1 Cycles: *SUCCESS* at Fri Sep 20 22:20:29 UTC 2024
Experiment C48_S2SW_6c8548e3 Completed 1 Cycles: *SUCCESS* at Fri Sep 20 22:51:44 UTC 2024
Experiment C48_S2SWA_gefs_6c8548e3 Completed 1 Cycles: *SUCCESS* at Sat Sep 21 00:00:03 UTC 2024
Experiment C96_atm3DVar_6c8548e3 Completed 3 Cycles: *SUCCESS* at Sat Sep 21 07:37:18 UTC 2024
Experiment C96C48_hybatmDA_6c8548e3 Completed 3 Cycles: *SUCCESS* at Sat Sep 21 08:07:51 UTC 2024
Experiment C96C48_ufs_hybatmDA_6c8548e3 Completed 2 Cycles: *SUCCESS* at Sat Sep 21 11:54:22 UTC 2024
Experiment C96C48_hybatmaerosnowDA_6c8548e3 Completed 3 Cycles: *SUCCESS* at Sat Sep 21 11:54:55 UTC 2024

@WalterKolczynski-NOAA WalterKolczynski-NOAA merged commit fe57bb4 into NOAA-EMC:develop Sep 23, 2024
10 of 11 checks passed
@EricSinsky-NOAA EricSinsky-NOAA deleted the feature/issue_2721 branch September 23, 2024 12:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully CI-Hercules-Passed **Bot use only** CI testing on Hercules for this PR has completed successfully CI-Wcoss2-Passed **Bot use only** CI testing on WCOSS for this PR has completed successfully
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update dependency check for ice netcdf file for GEFS
4 participants