Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update ufs model to version from 10-12-23 #1933

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
08ed404
update model hash to latest
JessicaMeixner-NOAA Sep 26, 2023
3703472
update configure files based on model updates to reg test
JessicaMeixner-NOAA Sep 26, 2023
cc3c1c1
updates to wcoss2 for module conflict errors
JessicaMeixner-NOAA Sep 27, 2023
46a9502
temporary updates to not link files that caused issues with aerosols
JessicaMeixner-NOAA Sep 29, 2023
22da06c
update model version (temporary branch) to use stack 1.5
JessicaMeixner-NOAA Oct 2, 2023
20e8de3
update for gocart new updates
JessicaMeixner-NOAA Oct 2, 2023
9630e5f
revert back to ufs develop branch
JessicaMeixner-NOAA Oct 3, 2023
684f1c4
update wave jobs to new modules
JessicaMeixner-NOAA Oct 3, 2023
fae62fc
Merge remote-tracking branch 'EMC/develop' into feature/updateufsstack15
JessicaMeixner-NOAA Oct 3, 2023
b3e111d
add coupled script temporarily
JessicaMeixner-NOAA Oct 3, 2023
a2de2fb
revert wcoss2 updates finish backing out 1.5 updates and just got to top
JessicaMeixner-NOAA Oct 3, 2023
4412f58
add wgrib2
JessicaMeixner-NOAA Oct 4, 2023
2b93041
Merge branch 'feature/updateufsstack15' of github.com:JessicaMeixner-…
JessicaMeixner-NOAA Oct 4, 2023
408bc97
update modules for wcoss2
JessicaMeixner-NOAA Oct 5, 2023
8430844
update for init cfp job on wcoss2
JessicaMeixner-NOAA Oct 5, 2023
9eeeade
Merge branch 'NOAA-EMC:develop' into feature/updateufsstack15
JessicaMeixner-NOAA Oct 12, 2023
a9169aa
use latest UFS
JessicaMeixner-NOAA Oct 12, 2023
7608b80
update fcst and waveinit modules
JessicaMeixner-NOAA Oct 12, 2023
fb96b7e
load extra module for wcoss2 wave jobs in rocoto script
JessicaMeixner-NOAA Oct 12, 2023
4c672eb
update UFS to latest
JessicaMeixner-NOAA Oct 14, 2023
6a61d8a
move linked gocart files to a copy statement at the end of run
JessicaMeixner-NOAA Oct 14, 2023
c8b0702
Merge remote-tracking branch 'EMC/develop' into updateUFS101223
JessicaMeixner-NOAA Oct 14, 2023
2ed31de
update copying of gocart files
JessicaMeixner-NOAA Oct 16, 2023
a34e88a
debugged output for gocart
JessicaMeixner-NOAA Oct 17, 2023
0d26726
Merge branch 'NOAA-EMC:develop' into updateUFS101223
JessicaMeixner-NOAA Oct 17, 2023
91c2fc6
remove temporary file
JessicaMeixner-NOAA Oct 17, 2023
981f64d
Merge branch 'updateUFS101223' of github.com:JessicaMeixner-NOAA/glob…
JessicaMeixner-NOAA Oct 17, 2023
fc9b4b3
Update ush/forecast_postdet.sh
JessicaMeixner-NOAA Oct 19, 2023
5333b5d
implementing review suggestions
JessicaMeixner-NOAA Oct 19, 2023
b7eae82
updates to clean up module load and create function to reduce possibi…
JessicaMeixner-NOAA Oct 20, 2023
803555d
update pnt resources and remove temp file
JessicaMeixner-NOAA Oct 20, 2023
ac28b5e
Update jobs/rocoto/waveprep.sh
JessicaMeixner-NOAA Oct 23, 2023
69157b5
Update ush/load_ufswm_modules.sh
JessicaMeixner-NOAA Oct 23, 2023
9a0b156
Update Externals.cfg
JessicaMeixner-NOAA Oct 23, 2023
1cbc829
Update ush/load_ufswm_modules.sh
JessicaMeixner-NOAA Oct 23, 2023
eb9459c
updating NETCDF as suggested by Walter for GEFS test
JessicaMeixner-NOAA Oct 23, 2023
9649540
Merge branch 'updateUFS101223' of github.com:JessicaMeixner-NOAA/glob…
JessicaMeixner-NOAA Oct 23, 2023
224b204
add export UTILROOT=${prod_util_ROOT}
JessicaMeixner-NOAA Oct 23, 2023
6acdf8b
update efcs in the same way fcst job was updated
JessicaMeixner-NOAA Oct 24, 2023
2db41ab
Merge remote-tracking branch 'EMC/develop' into updateUFS101223
JessicaMeixner-NOAA Oct 24, 2023
b49e0f3
pad point job with extra time to avoid out of time errors
JessicaMeixner-NOAA Oct 24, 2023
0fad02f
Merge remote-tracking branch 'EMC/develop' into updateUFS101223
JessicaMeixner-NOAA Oct 26, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Externals.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ protocol = git
required = True

[UFS]
tag = 4d05445
tag = 68050e5
local_path = sorc/ufs_model.fd
repo_url = https://github.com/ufs-community/ufs-weather-model.git
protocol = git
Expand Down
1 change: 1 addition & 0 deletions env/WCOSS2.env
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ elif [[ "${step}" = "preplandobs" ]]; then

elif [[ "${step}" = "waveinit" ]] || [[ "${step}" = "waveprep" ]] || [[ "${step}" = "wavepostsbs" ]] || [[ "${step}" = "wavepostbndpnt" ]] || [[ "${step}" = "wavepostbndpntbll" ]] || [[ "${step}" = "wavepostpnt" ]]; then

export USE_CFP="YES"
if [[ "${step}" = "waveprep" ]] && [[ "${CDUMP}" = "gfs" ]]; then export NTASKS=${NTASKS_gfs} ; fi
export wavempexec="${launcher} -np"
export wave_mpmd=${mpmd_opt}
Expand Down
20 changes: 3 additions & 17 deletions jobs/rocoto/efcs.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,23 +5,9 @@ source "${HOMEgfs}/ush/preamble.sh"
###############################################################
# Source FV3GFS workflow modules
#. ${HOMEgfs}/ush/load_fv3gfs_modules.sh
#status=$?
#[[ ${status} -ne 0 ]] && exit ${status}

# TODO: clean this up
source "${HOMEgfs}/ush/detect_machine.sh"
set +x
source "${HOMEgfs}/ush/module-setup.sh"
module use "${HOMEgfs}/sorc/ufs_model.fd/tests"
module load modules.ufs_model.lua
# Workflow needs utilities from prod_util (setPDY.sh, ndate, etc.)
module load prod_util
if [[ "${MACHINE_ID}" = "wcoss2" ]]; then
module load cray-pals
fi
module list
unset MACHINE_ID
set_trace
. ${HOMEgfs}/ush/load_ufswm_modules.sh
status=$?
[[ ${status} -ne 0 ]] && exit ${status}

export job="efcs"
export jobid="${job}.$$"
Expand Down
63 changes: 4 additions & 59 deletions jobs/rocoto/fcst.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,65 +3,11 @@
source "${HOMEgfs}/ush/preamble.sh"

###############################################################
# Source FV3GFS workflow modules
# Source UFS Weather Model workflow modules
#. ${HOMEgfs}/ush/load_fv3gfs_modules.sh
#status=$?
#[[ ${status} -ne 0 ]] && exit ${status}

# TODO: clean this up
source "${HOMEgfs}/ush/detect_machine.sh"
set +x
source "${HOMEgfs}/ush/module-setup.sh"
if [[ "${MACHINE_ID}" != "noaacloud" ]]; then
module use "${HOMEgfs}/sorc/ufs_model.fd/tests"
module load modules.ufs_model.lua
module load prod_util
fi

if [[ "${MACHINE_ID}" = "wcoss2" ]]; then
module load cray-pals
fi
if [[ "${MACHINE_ID}" = "hera" ]]; then
module use "/scratch2/NCEPDEV/ensemble/save/Walter.Kolczynski/modulefiles/core"
module load "miniconda3/4.6.14"
module load "gfs_workflow/1.0.0"
# TODO: orion and wcoss2 will be uncommented when they are ready. This comment block will be removed in the next PR
#elif [[ "${MACHINE_ID}" = "orion" ]]; then
# module use "/home/rmahajan/opt/global-workflow/modulefiles/core"
# module load "python/3.7.5"
# module load "gfs_workflow/1.0.0"
#elif [[ "${MACHINE_ID}" = "wcoss2" ]]; then
# module load "python/3.7.5"
fi
if [[ "${MACHINE_ID}" == "noaacloud" ]]; then
if [[ "${PW_CSP:-}" = "aws" ]]; then

# TODO: This can be cleaned-up; most of this is a hack for now.
module use "/contrib/spack-stack/envs/ufswm/install/modulefiles/Core"
module load "stack-intel"
module load "stack-intel-oneapi-mpi"
module use -a "/contrib/spack-stack/miniconda/modulefiles/miniconda/"
module load "py39_4.12.0"
module load "ufs-weather-model-env/1.0.0"
export NETCDF="/contrib/spack-stack/miniconda/apps/miniconda/py39_4.12.0"
# TODO: Are there plans for EPIC to maintain this package or should GW provide support?
export UTILROOT="/contrib/global-workflow/NCEPLIBS-prod_util"
export PATH="${PATH}:/contrib/global-workflow/bin"
ndate_path="$(command -v ndate)"
export NDATE="${ndate_path}"
fi
fi

module list
unset MACHINE_ID
set_trace

###############################################################
# exglobal_forecast.py requires the following in PYTHONPATH
# This will be moved to a module load when ready
wxflowPATH="${HOMEgfs}/ush/python:${HOMEgfs}/ush/python/wxflow/src:${HOMEgfs}/ush/python/pygfs"
PYTHONPATH="${PYTHONPATH:+${PYTHONPATH}:}${wxflowPATH}"
export PYTHONPATH
. ${HOMEgfs}/ush/load_ufswm_modules.sh
status=$?
[[ ${status} -ne 0 ]] && exit ${status}

export job="fcst"
export jobid="${job}.$$"
Expand All @@ -71,5 +17,4 @@ export jobid="${job}.$$"
${HOMEgfs}/jobs/JGLOBAL_FORECAST
status=$?


exit ${status}
3 changes: 2 additions & 1 deletion jobs/rocoto/waveinit.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@ source "${HOMEgfs}/ush/preamble.sh"
###############################################################
echo
echo "=============== START TO SOURCE FV3GFS WORKFLOW MODULES ==============="
. ${HOMEgfs}/ush/load_fv3gfs_modules.sh
#. ${HOMEgfs}/ush/load_fv3gfs_modules.sh
. ${HOMEgfs}/ush/load_ufswm_modules.sh
status=$?
[[ ${status} -ne 0 ]] && exit ${status}

Expand Down
3 changes: 2 additions & 1 deletion jobs/rocoto/wavepostbndpnt.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@ source "${HOMEgfs}/ush/preamble.sh"
###############################################################
echo
echo "=============== START TO SOURCE FV3GFS WORKFLOW MODULES ==============="
. ${HOMEgfs}/ush/load_fv3gfs_modules.sh
#. ${HOMEgfs}/ush/load_fv3gfs_modules.sh
. ${HOMEgfs}/ush/load_ufswm_modules.sh
status=$?
[[ ${status} -ne 0 ]] && exit ${status}

Expand Down
3 changes: 2 additions & 1 deletion jobs/rocoto/wavepostbndpntbll.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@ source "${HOMEgfs}/ush/preamble.sh"
###############################################################
echo
echo "=============== START TO SOURCE FV3GFS WORKFLOW MODULES ==============="
. ${HOMEgfs}/ush/load_fv3gfs_modules.sh
#. ${HOMEgfs}/ush/load_fv3gfs_modules.sh
. ${HOMEgfs}/ush/load_ufswm_modules.sh
status=$?
[[ ${status} -ne 0 ]] && exit ${status}

Expand Down
3 changes: 2 additions & 1 deletion jobs/rocoto/wavepostpnt.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@ source "${HOMEgfs}/ush/preamble.sh"
###############################################################
echo
echo "=============== START TO SOURCE FV3GFS WORKFLOW MODULES ==============="
. ${HOMEgfs}/ush/load_fv3gfs_modules.sh
#. ${HOMEgfs}/ush/load_fv3gfs_modules.sh
. ${HOMEgfs}/ush/load_ufswm_modules.sh
status=$?
[[ ${status} -ne 0 ]] && exit ${status}

Expand Down
3 changes: 2 additions & 1 deletion jobs/rocoto/wavepostsbs.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@ source "${HOMEgfs}/ush/preamble.sh"

###############################################################
# Source FV3GFS workflow modules
. ${HOMEgfs}/ush/load_fv3gfs_modules.sh
#. ${HOMEgfs}/ush/load_fv3gfs_modules.sh
. ${HOMEgfs}/ush/load_ufswm_modules.sh
status=$?
[[ ${status} -ne 0 ]] && exit ${status}

Expand Down
3 changes: 2 additions & 1 deletion jobs/rocoto/waveprep.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@ source "${HOMEgfs}/ush/preamble.sh"
###############################################################
echo
echo "=============== START TO SOURCE FV3GFS WORKFLOW MODULES ==============="
. ${HOMEgfs}/ush/load_fv3gfs_modules.sh
#. ${HOMEgfs}/ush/load_fv3gfs_modules.sh
. ${HOMEgfs}/ush/load_ufswm_modules.sh
status=$?
[[ ${status} -ne 0 ]] && exit ${status}

Expand Down
2 changes: 1 addition & 1 deletion parm/config/gefs/config.base.emc.dyn
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ export VERBOSE="YES"
export KEEPDATA="NO"
export CHGRP_RSTPROD="@CHGRP_RSTPROD@"
export CHGRP_CMD="@CHGRP_CMD@"
export NCDUMP="${NETCDF}/bin/ncdump"
export NCDUMP="${NETCDF:-}/bin/ncdump"
export NCLEN="${HOMEgfs}/ush/getncdimlen"

# Machine environment, jobs, and other utility scripts
Expand Down
2 changes: 1 addition & 1 deletion parm/config/gfs/config.resources
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,7 @@ elif [[ ${step} = "wavepostbndpntbll" ]]; then

elif [[ ${step} = "wavepostpnt" ]]; then

export wtime_wavepostpnt="01:30:00"
export wtime_wavepostpnt="04:00:00"
export npe_wavepostpnt=200
export nth_wavepostpnt=1
npe_node_wavepostpnt=$(echo "${npe_node_max} / ${nth_wavepostpnt}" | bc)
Expand Down
2 changes: 2 additions & 0 deletions parm/ufs/chem/CAP.rc
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ REPORT_THROUGHPUT: .false.

USE_SHMEM: 0

GridType: Cubed-Sphere

MAPL_ENABLE_TIMERS: NO
MAPL_ENABLE_MEMUTILS: NO
PRINTSPEC: 0 # (0: OFF, 1: IMPORT & EXPORT, 2: IMPORT, 3: EXPORT)
Expand Down
2 changes: 2 additions & 0 deletions parm/ufs/chem/GOCART2G_GridComp.rc
Original file line number Diff line number Diff line change
Expand Up @@ -39,3 +39,5 @@ PASSIVE_INSTANCES_NI:
aerosol_monochromatic_optics_wavelength_in_nm_from_LUT: 470 550 670 870
wavelengths_for_profile_aop_in_nm: 470 550 670 870 # must be included in LUT
wavelengths_for_vertically_integrated_aop_in_nm: 470 550 670 870 # must be included in LUT

use_threads: .TRUE.
11 changes: 8 additions & 3 deletions parm/ufs/nems.configure.cpld.IN
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ runSeq::
MED med_phases_post_atm
ICE -> MED :remapMethod=redist
MED med_phases_post_ice
MED med_phases_ocnalb_run
MED med_phases_prep_ocn_accum
@
OCN -> MED :remapMethod=redist
Expand All @@ -89,12 +90,10 @@ MED_attributes::
ATM_model = @[atm_model]
ICE_model = @[ice_model]
OCN_model = @[ocn_model]
history_n = 0
history_option = nhours
history_ymd = -999
coupling_mode = @[CPLMODE]
history_tile_atm = @[ATMTILESIZE]
pio_rearranger = box
ocean_albedo_limit = @[ocean_albedo_limit]
::
ALLCOMP_attributes::
ScalarFieldCount = 2
Expand All @@ -114,4 +113,10 @@ ALLCOMP_attributes::
stop_n = @[FHMAX]
stop_option = nhours
stop_ymd = -999
orb_eccen = 1.e36
orb_iyear = 2000
orb_iyear_align = 2000
orb_mode = fixed_year
orb_mvelp = 1.e36
orb_obliq = 1.e36
::
11 changes: 8 additions & 3 deletions parm/ufs/nems.configure.cpld_aero.IN
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,7 @@ runSeq::
MED med_phases_post_atm
ICE -> MED :remapMethod=redist
MED med_phases_post_ice
MED med_phases_ocnalb_run
MED med_phases_prep_ocn_accum
@
OCN -> MED :remapMethod=redist
Expand All @@ -101,12 +102,10 @@ MED_attributes::
ATM_model = @[atm_model]
ICE_model = @[ice_model]
OCN_model = @[ocn_model]
history_n = 0
history_option = nhours
history_ymd = -999
coupling_mode = @[CPLMODE]
history_tile_atm = @[ATMTILESIZE]
pio_rearranger = box
ocean_albedo_limit = @[ocean_albedo_limit]
::
ALLCOMP_attributes::
ScalarFieldCount = 2
Expand All @@ -126,4 +125,10 @@ ALLCOMP_attributes::
stop_n = @[FHMAX]
stop_option = nhours
stop_ymd = -999
orb_eccen = 1.e36
orb_iyear = 2000
orb_iyear_align = 2000
orb_mode = fixed_year
orb_mvelp = 1.e36
orb_obliq = 1.e36
::
11 changes: 8 additions & 3 deletions parm/ufs/nems.configure.cpld_aero_outerwave.IN
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,7 @@ runSeq::
MED med_phases_post_atm
ICE -> MED :remapMethod=redist
MED med_phases_post_ice
MED med_phases_ocnalb_run
MED med_phases_prep_ocn_accum
MED med_phases_prep_wav_accum
@
Expand All @@ -121,12 +122,10 @@ MED_attributes::
ICE_model = @[ice_model]
OCN_model = @[ocn_model]
WAV_model = @[wav_model]
history_n = 0
history_option = nhours
history_ymd = -999
coupling_mode = @[CPLMODE]
history_tile_atm = @[ATMTILESIZE]
pio_rearranger = box
ocean_albedo_limit = @[ocean_albedo_limit]
::
ALLCOMP_attributes::
ScalarFieldCount = 2
Expand All @@ -146,4 +145,10 @@ ALLCOMP_attributes::
stop_n = @[FHMAX]
stop_option = nhours
stop_ymd = -999
orb_eccen = 1.e36
orb_iyear = 2000
orb_iyear_align = 2000
orb_mode = fixed_year
orb_mvelp = 1.e36
orb_obliq = 1.e36
::
12 changes: 9 additions & 3 deletions parm/ufs/nems.configure.cpld_aero_wave.IN
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,7 @@ runSeq::
MED med_phases_post_ice
WAV -> MED :remapMethod=redist
MED med_phases_post_wav
MED med_phases_ocnalb_run
MED med_phases_prep_ocn_accum
@
OCN -> MED :remapMethod=redist
Expand All @@ -121,11 +122,10 @@ MED_attributes::
ICE_model = @[ice_model]
OCN_model = @[ocn_model]
WAV_model = @[wav_model]
history_n = 0
history_option = nhours
history_ymd = -999
coupling_mode = @[CPLMODE]
history_tile_atm = @[ATMTILESIZE]
pio_rearranger = box
ocean_albedo_limit = @[ocean_albedo_limit]
::
ALLCOMP_attributes::
ScalarFieldCount = 2
Expand All @@ -145,4 +145,10 @@ ALLCOMP_attributes::
stop_n = @[FHMAX]
stop_option = nhours
stop_ymd = -999
orb_eccen = 1.e36
orb_iyear = 2000
orb_iyear_align = 2000
orb_mode = fixed_year
orb_mvelp = 1.e36
orb_obliq = 1.e36
::
11 changes: 8 additions & 3 deletions parm/ufs/nems.configure.cpld_outerwave.IN
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@ runSeq::
MED med_phases_post_atm
ICE -> MED :remapMethod=redist
MED med_phases_post_ice
MED med_phases_ocnalb_run
MED med_phases_prep_ocn_accum
MED med_phases_prep_wav_accum
@
Expand All @@ -109,12 +110,10 @@ MED_attributes::
ICE_model = @[ice_model]
OCN_model = @[ocn_model]
WAV_model = @[wav_model]
history_n = 0
history_option = nhours
history_ymd = -999
coupling_mode = @[CPLMODE]
history_tile_atm = @[ATMTILESIZE]
pio_rearranger = box
ocean_albedo_limit = @[ocean_albedo_limit]
::
ALLCOMP_attributes::
ScalarFieldCount = 2
Expand All @@ -134,4 +133,10 @@ ALLCOMP_attributes::
stop_n = @[FHMAX]
stop_option = nhours
stop_ymd = -999
orb_eccen = 1.e36
orb_iyear = 2000
orb_iyear_align = 2000
orb_mode = fixed_year
orb_mvelp = 1.e36
orb_obliq = 1.e36
::
Loading