Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move diag_table creation to fcst run script #349

Closed
wants to merge 6 commits into from
Closed

Move diag_table creation to fcst run script #349

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
56a9219
Rename file.
christinaholtNOAA Nov 2, 2020
64fa30d
Changes for run-time creation of diag_table.
christinaholtNOAA Nov 2, 2020
2d16a4a
No need to make this file here.
christinaholtNOAA Nov 3, 2020
eec807a
The modulefiles must have changed names.
christinaholtNOAA Nov 3, 2020
9b56257
Merge remote-tracking branch 'dtc/develop' into diag_table_creation
christinaholtNOAA Nov 18, 2020
9ac54a3
Merge remote-tracking branch 'dtc/develop' into diag_table_creation
christinaholtNOAA Feb 10, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 0 additions & 5 deletions scripts/exregional_make_grid.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@
. $USHDIR/link_fix.sh
. $USHDIR/set_FV3nml_sfc_climo_filenames.sh
. $USHDIR/set_FV3nml_stoch_params.sh
. $USHDIR/create_diag_table_files.sh
#
#-----------------------------------------------------------------------
#
Expand Down Expand Up @@ -682,10 +681,6 @@ if [ "${DO_ENSEMBLE}" = TRUE ]; then
Call to function to set stochastic parameters in the FV3 namelist files
for the various ensemble members failed."
fi

create_diag_table_files || print_err_msg_exit "\
Call to function to create a diagnostics table file under each cycle
directory failed."
#
#-----------------------------------------------------------------------
#
Expand Down
14 changes: 14 additions & 0 deletions scripts/exregional_run_fcst.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
#-----------------------------------------------------------------------
#
. $USHDIR/create_model_configure_file.sh
. $USHDIR/create_diag_table_file.sh
#
#-----------------------------------------------------------------------
#
Expand Down Expand Up @@ -459,6 +460,19 @@ cycle's (cdate) run directory (run_dir) failed:
#
#-----------------------------------------------------------------------
#
# Call the function that creates the model configuration file within each
# cycle directory.
#
#-----------------------------------------------------------------------
#
create_diag_table_file \
run_dir="${run_dir}" || print_err_msg_exit "\
Call to function to create a diag table file for the current
cycle's (cdate) run directory (run_dir) failed:
run_dir = \"${run_dir}\""
#
#-----------------------------------------------------------------------
#
# If running ensemble forecasts, create a link to the cycle-specific
# diagnostic tables file in the cycle directory. Note that this link
# should not be made if not running ensemble forecasts because in that
Expand Down
38 changes: 17 additions & 21 deletions ush/create_diag_table_files.sh → ush/create_diag_table_file.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
#
#-----------------------------------------------------------------------
#
function create_diag_table_files() {
function create_diag_table_file() {
#
#-----------------------------------------------------------------------
#
Expand Down Expand Up @@ -45,7 +45,9 @@ function create_diag_table_files() {
#
#-----------------------------------------------------------------------
#
local valid_args=()
local valid_args=(
run_dir \
)
process_args valid_args "$@"
#
#-----------------------------------------------------------------------
Expand All @@ -65,8 +67,6 @@ function create_diag_table_files() {
#-----------------------------------------------------------------------
#
local i \
cdate \
cycle_dir \
diag_table_fp \
settings
#
Expand All @@ -77,39 +77,35 @@ function create_diag_table_files() {
#-----------------------------------------------------------------------
#
print_info_msg "$VERBOSE" "
Creating a diagnostics table file (\"${DIAG_TABLE_FN}\") within each cycle
directory..."
Creating a diagnostics table file (\"${DIAG_TABLE_FN}\") in the specified run
directory (run_dir):
run_dir = \"${run_dir}\""

for (( i=0; i<${NUM_CYCLES}; i++ )); do

cdate="${ALL_CDATES[$i]}"
cycle_dir="${CYCLE_BASEDIR}/$cdate"

diag_table_fp="${cycle_dir}/${DIAG_TABLE_FN}"
print_info_msg "$VERBOSE" "
# Copy template diag_table file from the templates directory to the
# run directory.
diag_table_fp="${run_dir}/${DIAG_TABLE_FN}"
print_info_msg "$VERBOSE" "
Using the template diagnostics table file:

diag_table_tmpl_fp = ${DIAG_TABLE_TMPL_FP}
diag_table_tmpl_fp = ${DIAG_TABLE_TMPL_FP}

to create:

diag_table_fp = \"${diag_table_fp}\""
diag_table_fp = \"${diag_table_fp}\""

settings="
starttime: !datetime ${ALL_CDATES[$i]}
settings="
starttime: !datetime ${CDATE}
cres: ${CRES}
"

$USHDIR/fill_jinja_template.py -q -u "${settings}" -t "${DIAG_TABLE_TMPL_FP}" -o "${diag_table_fp}" || \
print_err_msg_exit "
$USHDIR/fill_jinja_template.py -q -u "${settings}" -t "${DIAG_TABLE_TMPL_FP}" -o "${diag_table_fp}" || \
print_err_msg_exit "
!!!!!!!!!!!!!!!!!

fill_jinja_template.py failed!

!!!!!!!!!!!!!!!!!
"

done
#
#-----------------------------------------------------------------------
#
Expand Down
20 changes: 0 additions & 20 deletions ush/generate_FV3LAM_wflow.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,6 @@ ushdir="${scrfunc_dir}"
. $ushdir/source_util_funcs.sh
. $ushdir/set_FV3nml_sfc_climo_filenames.sh
. $ushdir/set_FV3nml_stoch_params.sh
. $ushdir/create_diag_table_files.sh
#
#-----------------------------------------------------------------------
#
Expand Down Expand Up @@ -315,21 +314,6 @@ $settings"
#
#-----------------------------------------------------------------------
#
# Create the cycle directories.
#
#-----------------------------------------------------------------------
#
print_info_msg "$VERBOSE" "
Creating the cycle directories..."

for (( i=0; i<${NUM_CYCLES}; i++ )); do
cdate="${ALL_CDATES[$i]}"
cycle_dir="${CYCLE_BASEDIR}/$cdate"
mkdir_vrfy -p "${cycle_dir}"
done
#
#-----------------------------------------------------------------------
#
# Create a symlink in the experiment directory that points to the workflow
# (re)launch script.
#
Expand Down Expand Up @@ -720,10 +704,6 @@ Call to function to set stochastic parameters in the FV3 namelist files
for the various ensemble members failed."
fi

create_diag_table_files || print_err_msg_exit "\
Call to function to create a diagnostics table file under each cycle
directory failed."

fi
#
#-----------------------------------------------------------------------
Expand Down
30 changes: 0 additions & 30 deletions ush/setup.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -621,24 +621,6 @@ done
#
#-----------------------------------------------------------------------
#
# Call a function to generate the array ALL_CDATES containing the cycle
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we leave this script as is? I like having an array handy (ALL_CDATES) that contains all the cycles over which the workflow will run. Maybe even print it out for informational purposes. Otherwise everything looks good, so I'll approve.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gsketefian It's fine to leave it in for experiments that are only a couple of cycles, but when using the configuration layer for real time purposes where common practice is to set the last cycle to an arbitrarily long time in the future it becomes more cumbersome than necessary, printing out an array of 1800+ dates for a real time run that defines 5 cycles per day for a year (example from current RRFS_dev* configs). Can you not get this information from the Rocoto database, where it would be more accurate, anyway?

If there are no dependencies on needing to know all the cycles, there should be no need for the var_defns.sh file to know this huge array.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@christinaholtNOAA, @gsketefian, since this array may be useful in the future, let's create an if statement here that only outputs the start and end cycles above a given threshold (for now, 30). I've tested this with Christina's fork and it works as expected.

set_cycle_dates
date_start="${DATE_FIRST_CYCL}"
date_end="${DATE_LAST_CYCL}"
cycle_hrs="${CYCL_HRS_str}"
output_varname_all_cdates="ALL_CDATES"

NUM_CYCLES="${#ALL_CDATES[@]}"

if [ $NUM_CYCLES -gt 30 ] ; then
unset ALL_CDATES
print_info_msg "$VERBOSE" "
Too many cycles in ALL_CDATES, defining in abbreviated form."
ALL_CDATES="${DATE_FIRST_CYCL}${CYCL_HRS[0]}...${DATE_LAST_CYCL}${CYCL_HRS[-1]}"
fi

# dates/hours for which to run forecasts. The elements of this array
# will have the form YYYYMMDDHH. They are the starting dates/times of
# the forecasts that will be run in the experiment. Then set NUM_CYCLES
# to the number of elements in this array.
#
#-----------------------------------------------------------------------
#
set_cycle_dates \
date_start="${DATE_FIRST_CYCL}" \
date_end="${DATE_LAST_CYCL}" \
cycle_hrs="${CYCL_HRS_str}" \
output_varname_all_cdates="ALL_CDATES"

NUM_CYCLES="${#ALL_CDATES[@]}"
#
#-----------------------------------------------------------------------
#
# Set various directories.
#
# HOMErrfs:
Expand Down Expand Up @@ -2662,18 +2644,6 @@ LBC_SPEC_FCST_HRS=(${LBC_SPEC_FCST_HRS[@]})
#
#-----------------------------------------------------------------------
#
# The number of cycles for which to make forecasts and the list of starting
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as above (please leave in the script for now). Thx!

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code would stay unchanged with the added if statement mentioned above.

# dates/hours of these cycles.
#
#-----------------------------------------------------------------------
#
NUM_CYCLES="${NUM_CYCLES}"
ALL_CDATES=( \\
$( printf "\"%s\" \\\\\n" "${ALL_CDATES[@]}" )
)
#
#-----------------------------------------------------------------------
#
# If USE_FVCOM is set to TRUE, then FVCOM data (located in FVCOM_DIR
# in FVCOM_FILE) will be used to update lower boundary conditions during
# make_ics.
Expand Down