Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stochastic physics clean and updates to CA #832

Merged
merged 34 commits into from
Sep 30, 2021
Merged
Show file tree
Hide file tree
Changes from 32 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
f8ff87a
updates for cleaned up ca code
pjpegion Sep 8, 2021
a9bafe3
add ca restart test
pjpegion Sep 10, 2021
a16f951
add ca_restart
pjpegion Sep 22, 2021
fc52bc3
sync with develop
pjpegion Sep 22, 2021
e1aadfb
add ca_restart to rt.conf
pjpegion Sep 22, 2021
5e5b148
update stochastic_physics submodule
pjpegion Sep 23, 2021
fd4f196
address code review changes
pjpegion Sep 24, 2021
a12dac7
updated submodules
pjpegion Sep 24, 2021
c79869f
change rcell to 0.1 for RT
pjpegion Sep 24, 2021
6bd07d0
update FV3 submodule
pjpegion Sep 24, 2021
9d02429
updated stochastic_phyiscs
pjpegion Sep 25, 2021
be37ca9
remove blank or commented lines
pjpegion Sep 27, 2021
4aff8e5
update gitmodules to point to my fork
pjpegion Sep 27, 2021
00bd110
update gitmodules again
pjpegion Sep 27, 2021
2c1e16a
update stochastic_physics submodule
pjpegion Sep 28, 2021
b0eb712
make addtions to MOM_input_templates consistant with style
pjpegion Sep 28, 2021
9517f6e
update MOM6 submodule
pjpegion Sep 28, 2021
12d302f
add resolution dependent rcell for ca, change restart logic and add …
pjpegion Sep 29, 2021
9ba6cf5
fix typo in default_vars.sh
pjpegion Sep 29, 2021
1c975d1
update to develop
pjpegion Sep 29, 2021
1903c0f
remove redundant copies in Diag container
pjpegion Sep 29, 2021
b3b9a68
update BL_DATE to 20210929
pjpegion Sep 29, 2021
3773b3e
RT JOBS PASSED: hera.gnu. Log file uploaded.
BrianCurtis-NOAA Sep 30, 2021
2dad8e5
RT JOBS PASSED: hera.intel. Log file uploaded.
BrianCurtis-NOAA Sep 30, 2021
7cdec95
RT JOBS PASSED: cheyenne.gnu. Log file uploaded.
BrianCurtis-NOAA Sep 30, 2021
6c820f6
RT JOBS PASSED: gaea.intel. Log file uploaded.
BrianCurtis-NOAA Sep 30, 2021
66c2ba7
RT JOBS PASSED: cheyenne.intel. Log file uploaded.
BrianCurtis-NOAA Sep 30, 2021
0635d8e
add jet.intel log
DeniseWorthen Sep 30, 2021
0cd2d37
add wcoss_dell_p3 log
DeniseWorthen Sep 30, 2021
20fdeeb
add orion.intel log
DeniseWorthen Sep 30, 2021
44210fe
add wcoss_cray log
DeniseWorthen Sep 30, 2021
21ce83c
Merge branch 'new_stoch' of https://github.com/pjpegion/ufs-weather-m…
DeniseWorthen Sep 30, 2021
9696b61
fix utest script for restarts
pjpegion Sep 30, 2021
d62f71c
revert submodules
pjpegion Sep 30, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[submodule "FV3"]
path = FV3
url = https://github.com/NOAA-EMC/fv3atm
branch = develop
url = https://github.com/pjpegion/fv3atm
branch = new_stoch
[submodule "NEMS"]
path = NEMS
url = https://github.com/NOAA-EMC/NEMS
Expand All @@ -12,8 +12,8 @@
branch = develop
[submodule "stochastic_physics"]
path = stochastic_physics
url = https://github.com/noaa-psd/stochastic_physics
branch = master
url = https://github.com/pjpegion/stochastic_physics
branch = unit_tests_orion
[submodule "CMakeModules"]
path = CMakeModules
url = https://github.com/NOAA-EMC/CMakeModules
Expand All @@ -28,8 +28,8 @@
branch = emc/develop
[submodule "MOM6"]
path = MOM6-interface/MOM6
url = https://github.com/NOAA-EMC/MOM6
branch = dev/emc
url = https://github.com/pjpegion/MOM6
branch = ocn_stoch_july2021
[submodule "CICE"]
path = CICE-interface/CICE
url = https://github.com/NOAA-EMC/CICE
Expand Down
5 changes: 5 additions & 0 deletions MOM6-interface/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@ target_include_directories(mom6_obj PRIVATE $<BUILD_INTERFACE:${CMAKE_CURRENT_BI
target_include_directories(mom6_obj PRIVATE $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/MOM6/config_src/memory/dynamic_nonsymmetric>
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/MOM6/src/framework>)
target_link_libraries(mom6_obj PRIVATE fms
esmf
stochastic_physics
NetCDF::NetCDF_Fortran)
# OpenMP is disabled in MOM6
#if(OpenMP_Fortran_FOUND)
Expand Down Expand Up @@ -58,6 +60,7 @@ target_include_directories(mom6 PUBLIC $<BUILD_INTERFACE:${CMAKE_CURRENT_BINARY_
target_include_directories(mom6 PUBLIC $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/MOM6/config_src/memory/dynamic_nonsymmetric>
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/MOM6/src/framework>)
target_link_libraries(mom6 PUBLIC fms
stochastic_physics
esmf
NetCDF::NetCDF_Fortran)
# OpenMP is disabled in MOM6
Expand All @@ -77,6 +80,8 @@ if(MOM6SOLO)
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/MOM6/src/framework>)
target_link_libraries(mom6solo PRIVATE mom6_obj
fms
stochastic_physics
esmf
NetCDF::NetCDF_Fortran)
# OpenMP is disabled in MOM6
#if(OpenMP_Fortran_FOUND)
Expand Down
1 change: 1 addition & 0 deletions MOM6-interface/mom6_files.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -254,6 +254,7 @@ list(APPEND mom6_src_files
MOM6/config_src/external/ODA_hooks/ocean_da_core.F90
MOM6/config_src/external/ODA_hooks/ocean_da_types.F90
MOM6/config_src/external/ODA_hooks/write_ocean_obs.F90
MOM6/config_src/external/OCEAN_stochastic_phyiscs/MOM_stochastics.F90

MOM6/config_src/infra/FMS1/MOM_coms_infra.F90
MOM6/config_src/infra/FMS1/MOM_constants.F90
Expand Down
1 change: 1 addition & 0 deletions cmake/configure_apps.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ endif()

if(APP MATCHES "^(NG-GODAS|NG-GODAS-NEMSDATM)$")
set(CMEPS ON CACHE BOOL "Enable CMEPS" FORCE)
set(STOCH_PHYS ON CACHE BOOL "Enable Stochastic Physics" FORCE)
set(FMS ON CACHE BOOL "Enable FMS" FORCE)
set(MOM6 ON CACHE BOOL "Enable MOM6" FORCE)
set(CICE6 ON CACHE BOOL "Enable CICE6" FORCE)
Expand Down
2 changes: 1 addition & 1 deletion stochastic_physics
Submodule stochastic_physics updated 69 files
+4 −20 CMakeLists.txt
+0 −780 atmosphere_stub.F90
+164 −206 cellular_automata_global.F90
+100 −103 cellular_automata_sgs.F90
+0 −40 compile_standalone
+0 −17 compile_standalone_ca
+0 −12 compns_stochy.F90
+0 −273 dezouv_stochy.f
+0 −271 dozeuv_stochy.f
+0 −95 epslon_stochy.f
+0 −536 fftpack_stochy.f
+0 −54 four_to_grid_stochy.F
+0 −1,300 fv_control_stub.F90
+0 −93 get_lats_node_a_stochy.f
+0 −84 get_ls_node_stochy.f
+88 −147 get_stochy_pattern.F90
+0 −89 getcon_lag_stochy.f
+0 −273 getcon_spectral.F90
+0 −115 glats_stochy.f
+0 −179 gozrineo_stochy.f
+0 −250 initialize_spectral_mod.F90
+7 −14 mersenne_twister.F90
+54 −5 mpi_wrapper.F90
+0 −11 num_parthds_stochy.f
+0 −288 pln2eo_stochy.f
+0 −0 plumes.F90
+30 −0 random_numbers.F90
+0 −196 setlats_a_stochy.f
+0 −134 setlats_lag_stochy.f
+0 −269 spectral_layout.F90
+2,366 −0 spectral_transforms.F90
+0 −262 standalone_ca.F90
+0 −335 standalone_stochy.F90
+0 −92 standalone_stochy_module.F90
+15 −34 stochastic_physics.F90
+45 −70 stochy_data_mod.F90
+9 −43 stochy_internal_state_mod.F90
+2 −2 stochy_namelist_def.F90
+9 −14 stochy_patterngenerator.F90
+0 −278 sumfln_stochy.f
+392 −0 unit_tests/atmosphere_stub.F90
+106 −0 unit_tests/compare_ca_output.F90
+162 −0 unit_tests/compare_output.F90
+1 −0 unit_tests/compile_compare.sh
+1 −0 unit_tests/compile_compare_ca.sh
+51 −0 unit_tests/compile_standalone.hera_gnu
+46 −0 unit_tests/compile_standalone.hera_intel
+53 −0 unit_tests/compile_standalone_ca.hera.gnu
+45 −0 unit_tests/compile_standalone_ca.hera.intel
+351 −0 unit_tests/fv_arrays_stub.F90
+1,166 −0 unit_tests/fv_control_stub.F90
+1,387 −0 unit_tests/fv_mp_stub_mod.F90
+72 −0 unit_tests/input.nml.ca_template
+149 −0 unit_tests/input.nml.lnd1
+72 −0 unit_tests/input.nml.noise
+83 −0 unit_tests/input.nml.template
+149 −0 unit_tests/module-setup.sh
+26 −0 unit_tests/modules.hera.intel
+35 −0 unit_tests/modules.orion.intel
+26 −0 unit_tests/modules.stoch
+28 −0 unit_tests/modules.stoch_gnu
+27 −0 unit_tests/modules.stoch_gnu_dbg
+69 −0 unit_tests/run_ca.sh
+45 −0 unit_tests/run_standalone.sh
+102 −0 unit_tests/run_unit_tests.sh
+134 −0 unit_tests/run_unit_tests_ca.sh
+376 −0 unit_tests/standalone_ca.F90
+367 −0 unit_tests/standalone_stochy.F90
+303 −239 update_ca.F90
264 changes: 145 additions & 119 deletions tests/RegressionTests_cheyenne.gnu.log

Large diffs are not rendered by default.

1,033 changes: 529 additions & 504 deletions tests/RegressionTests_cheyenne.intel.log

Large diffs are not rendered by default.

1,017 changes: 521 additions & 496 deletions tests/RegressionTests_gaea.intel.log

Large diffs are not rendered by default.

262 changes: 144 additions & 118 deletions tests/RegressionTests_hera.gnu.log

Large diffs are not rendered by default.

1,221 changes: 635 additions & 586 deletions tests/RegressionTests_hera.intel.log

Large diffs are not rendered by default.

1,065 changes: 551 additions & 514 deletions tests/RegressionTests_jet.intel.log

Large diffs are not rendered by default.

1,031 changes: 528 additions & 503 deletions tests/RegressionTests_orion.intel.log

Large diffs are not rendered by default.

709 changes: 368 additions & 341 deletions tests/RegressionTests_wcoss_cray.log

Large diffs are not rendered by default.

1,078 changes: 547 additions & 531 deletions tests/RegressionTests_wcoss_dell_p3.log

Large diffs are not rendered by default.

23 changes: 23 additions & 0 deletions tests/default_vars.sh
Original file line number Diff line number Diff line change
Expand Up @@ -600,8 +600,16 @@ export CDMBWD_c192='0.23,1.5,1.0,1.0'
export CDMBWD_c384='1.1,0.72,1.0,1.0'
export CDMBWD_c768='4.0,0.15,1.0,1.0'

# CA tunable parameter
export RCELL_c48=0.1
export RCELL_c96=0.1
export RCELL_c192=0.2
export RCELL_c384=0.72
export RCELL_c768=0.72

# set default
export CDMBWD=${CDMBWD_c96}
export RCELL=${RCELL_c96}

# PBL
export SATMEDMF=.false.
Expand Down Expand Up @@ -840,6 +848,11 @@ export MOM6_ALLOW_LANDMASK_CHANGES=False
# MOM6 IAU
export MOM_IAU=False
export MOM_IAU_HRS=6
# MOM6 stochastics
export DO_OCN_SPPT=False
export PERT_EPBL=False
export OCN_SPPT=-999.
export EPBL=-999.

# CICE6 defaults; 1 degree
export NPROC_ICE=12
Expand Down Expand Up @@ -969,6 +982,11 @@ export CHLCLIM=seawifs_1998-2006_smoothed_2X.nc
# MOM6 IAU
export MOM_IAU=False
export MOM_IAU_HRS=6
# MOM6 stochastics
export DO_OCN_SPPT=False
export PERT_EPBL=False
export OCN_SPPT=-999.
export EPBL=-999.
# this must be set False for restart repro
export MOM6_REPRO_LA=False
# since coupling_interval_slow is set to DT_THERM, this should be always be false
Expand Down Expand Up @@ -1092,6 +1110,11 @@ export CHLCLIM=seawifs_1998-2006_smoothed_2X.nc
# MOM6 IAU
export MOM_IAU=False
export MOM_IAU_HRS=6
# MOM6 stochastics
export DO_OCN_SPPT=False
export PERT_EPBL=False
export OCN_SPPT=-999.
export EPBL=-999.
# this must be set False for restart repro
export MOM6_REPRO_LA=False
# since coupling_interval_slow is set to DT_THERM, this should be always be false
Expand Down
20 changes: 12 additions & 8 deletions tests/fv3_conf/control_run.IN
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ else
if [ $MODEL_INITIALIZATION = false ]; then
#read_inc and iau tests restart from fh=24
CURRDIR=`pwd`
FILEDIR=${CURRDIR}/../control${SUFFIX}/RESTART/
FILEDIR=${CURRDIR}/../${DEP_RUN}${SUFFIX}/RESTART/
else
#for warm start from initialiation time
FILEDIR=@[INPUTDATA_ROOT]/${inputdir}/INPUT_L127_GDAS
Expand All @@ -53,18 +53,22 @@ else
cp ${FILEDIR}/phy_data.* INPUT/.
cp ${FILEDIR}/sfc_data.* INPUT/.
else
#restart test start from fh=12
rsync -arv ${FILEDIR} INPUT/
cp ${FILEDIR}/20210322.180000.* INPUT/.
#restart test start from fh FHROT
cp -r ../${DEP_RUN}${SUFFIX}/RESTART/${RESTART_FILE_PREFIX}.* ./INPUT
rm -f INPUT/fv_core.res.*
rm -f INPUT/fv_srf_wnd.res.*
rm -f INPUT/fv_tracer.res.*
rm -f INPUT/phy_data.*
rm -f INPUT/sfc_data.*
cp ${FILEDIR}/../INPUT/grid_spec.nc INPUT/.
cp ${FILEDIR}/../INPUT/*_grid.tile*.nc INPUT/.
cp ${FILEDIR}/../INPUT/oro_data.tile*.nc INPUT/.
cd INPUT
for RFILE in 20210322.180000.*; do
for RFILE in ../${DEP_RUN}${SUFFIX}/RESTART/${RESTART_FILE_PREFIX}.*; do
[ -e $RFILE ] || exit 1
mv $RFILE ${RFILE#20210322.180000.}
RFILE_OLD=$(basename $RFILE)
RFILE_NEW="${RFILE_OLD//${RESTART_FILE_PREFIX}./}"
cp $RFILE INPUT/${RFILE_NEW}
done
cd ..
fi

fi
Expand Down
150 changes: 0 additions & 150 deletions tests/fv3_conf/control_stochy_run.IN

This file was deleted.

5 changes: 5 additions & 0 deletions tests/parm/MOM_input_template_025
Original file line number Diff line number Diff line change
Expand Up @@ -858,6 +858,11 @@ SEA_ICE_RIGID_MASS = 100.0 ! [kg m-2] default = 1000.0
LIQUID_RUNOFF_FROM_DATA = @[MOM6_RIVER_RUNOFF] ! [Boolean] default = False
! If true, allows liquid river runoff to be specified via
! the data_table using the component name 'OCN'.
! === module ocean_stochastics ===
DO_SPPT = @[DO_OCN_SPPT] ! [Boolean] default = False
! If true perturb the diabatic tendencies in MOM_diabadic_driver
PERT_EPBL = @[PERT_EPBL] ! [Boolean] default = False
! If true perturb the KE dissipation and destruction in MOM_energetic_PBL
! === module MOM_restart ===
RESTART_CHECKSUMS_REQUIRED = False
! === module MOM_file_parser ===
5 changes: 5 additions & 0 deletions tests/parm/MOM_input_template_050
Original file line number Diff line number Diff line change
Expand Up @@ -912,6 +912,11 @@ SEA_ICE_RIGID_MASS = 100.0 ! [kg m-2] default = 1000.0
LIQUID_RUNOFF_FROM_DATA = @[MOM6_RIVER_RUNOFF] ! [Boolean] default = False
! If true, allows liquid river runoff to be specified via
! the data_table using the component name 'OCN'.
! === module ocean_stochastics ===
DO_SPPT = @[DO_OCN_SPPT] ! [Boolean] default = False
! If true perturb the diabatic tendencies in MOM_diabadic_driver
PERT_EPBL = @[PERT_EPBL] ! [Boolean] default = False
! If true perturb the KE dissipation and destruction in MOM_energetic_PBL
! === module MOM_restart ===

! === module MOM_file_parser ===
5 changes: 5 additions & 0 deletions tests/parm/MOM_input_template_100
Original file line number Diff line number Diff line change
Expand Up @@ -851,6 +851,11 @@ GUST_CONST = 0.02 ! [Pa] default = 0.0
FIX_USTAR_GUSTLESS_BUG = False ! [Boolean] default = True
! If true correct a bug in the time-averaging of the gustless wind friction
! velocity
! === module ocean_stochastics ===
DO_SPPT = @[DO_OCN_SPPT] ! [Boolean] default = False
! If true perturb the diabatic tendencies in MOM_diabadic_driver
PERT_EPBL = @[PERT_EPBL] ! [Boolean] default = False
! If true perturb the KE dissipation and destruction in MOM_energetic_PBL

! === module MOM_restart ===

Expand Down
2 changes: 1 addition & 1 deletion tests/parm/control_ca.nml.IN
Original file line number Diff line number Diff line change
Expand Up @@ -187,7 +187,7 @@ deflate_level=1
tlives = 1800
nseed = 100
nfracseed = 0.5
rcell = 0.72
rcell = @[RCELL]
ca_trigger = .True.
ca_entr = .False.
ca_closure = .False.
Expand Down
15 changes: 14 additions & 1 deletion tests/parm/input.mom6.nml.IN
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,17 @@
parameter_filename = 'INPUT/MOM_input',
'INPUT/MOM_override' /



&nam_stochy
new_lscale=.true.,
OCNSPPT=@[OCN_SPPT],
OCNSPPT_LSCALE=500e3,
OCNSPPT_TAU=21600,
ISEED_OCNSPPT=201601010012,
EPBL=@[EPBL],
EPBL_LSCALE=500e3,
EPBL_TAU=21600,
ISEED_EPBL=201601010023,
/
&nam_sfcperts
/
2 changes: 1 addition & 1 deletion tests/parm/input.mom6_ccpp.nml.IN
Original file line number Diff line number Diff line change
Expand Up @@ -194,7 +194,7 @@
tlives = 1800
nseed = 1000000
nfracseed = 0.5
rcell = 0.72
rcell = @[RCELL]
ca_trigger = .true.
ca_entr = .false.
ca_closure = .false.
Expand Down
Loading