From be69f8f08f9490f32895af6364361564e73d24de Mon Sep 17 00:00:00 2001 From: gsketefian <31046882+gsketefian@users.noreply.github.com> Date: Fri, 24 Jul 2020 17:24:44 -0600 Subject: [PATCH] Move creation of model_config file from the experiment generation step to the forecast task. (#256) Move creation of model_config file from the experiment generation step to the forecast task. Description of changes: ---------------------- In PR#245, the creation of model_config files was moved out of the forecast task (in scripts/exregional_run_fcst.sh) and into the experiment generation step (in ush/generate_FV3SAR_wflow.sh), and a model_config file was created only in the cycle directory (i.e. not in each run directory) assuming that the parameters in model_config will not change for forecasts under a given cycle (e.g. for various ensemble members under a given cycle). Although this is true for certain cases (e.g. cold starts), as pointed out by @christinaholtNOAA, this assumption does not hold in certain use cases (e.g. real-time runs) in which there may be multiple forecasts for a given cycle that need different values of the parameters in model_config (e.g. different forecast lengths, different computational resources, etc). For this reason, a model_config file must be created for each specific forecast and placed in the run directory for that forecast. Thus, in this PR, we move the creation of the model_config files back into the ush/exregional_run_fcst.sh script. Tests conducted: --------------- Ran all WE2E tests on hera. All but 3 passed. The 3 that failed are regional_003, regional_004, and regional_010, but those currently fail in the develop branch with the same errors. File-by-file description of modifications: ----------------------------------------- jobs/JREGIONAL_RUN_FCST: * Set the new cdate argument of the exregional_run_fcst() funciton to CDATE (which is set as an environment variable in the rocoto XML). scripts/exregional_run_fcst.sh: * Add new input argument cdate to this function. This is needed in creating the model_config file for this forecast. * Call the new function create_model_config_file() to create a model_config file in the current run directory. * For ensemble forecasts, remove creation of link to a cycle-specific model_config file. This file is now created on a per-forecast basis. * Edit comments. tests/baselines_list.txt: * Order WE2E test names lexicographically. ush/create_model_config_file.sh: * New function to set the parameters in the model_config file for a given cdate and forecast run directory. ush/create_model_config_files.sh: * Remove this file since the work this used to do for all cycles is now done by the new function ush/create_model_config_file.sh for one cycle. [Actually, this file got renamed to ush/create_model_config_file.sh (drop the "s" after file).] ush/generate_FV3SAR_wflow.sh: * Remove creation of model_config files for all cycles. This is now done on a per-forecast basis in scripts/exregional_run_fcst.sh. --- jobs/JREGIONAL_RUN_FCST | 1 + scripts/exregional_run_fcst.sh | 36 ++++-- tests/baselines_list.txt | 2 +- ush/create_model_config_file.sh | 175 +++++++++++++++++++++++++++++ ush/create_model_config_files.sh | 182 ------------------------------- ush/generate_FV3SAR_wflow.sh | 14 +-- 6 files changed, 206 insertions(+), 204 deletions(-) create mode 100644 ush/create_model_config_file.sh delete mode 100644 ush/create_model_config_files.sh diff --git a/jobs/JREGIONAL_RUN_FCST b/jobs/JREGIONAL_RUN_FCST index 5ab82f99d..a4806c765 100755 --- a/jobs/JREGIONAL_RUN_FCST +++ b/jobs/JREGIONAL_RUN_FCST @@ -74,6 +74,7 @@ mkdir_vrfy -p ${run_dir}/RESTART #----------------------------------------------------------------------- # $SCRIPTSDIR/exregional_run_fcst.sh \ + cdate="${CDATE}" \ cycle_dir="${CYCLE_DIR}" \ ensmem_indx="${ENSMEM_INDX}" \ slash_ensmem_subdir="${SLASH_ENSMEM_SUBDIR}" || \ diff --git a/scripts/exregional_run_fcst.sh b/scripts/exregional_run_fcst.sh index a6b51d735..5fa5dc375 100755 --- a/scripts/exregional_run_fcst.sh +++ b/scripts/exregional_run_fcst.sh @@ -12,6 +12,14 @@ # #----------------------------------------------------------------------- # +# Source other necessary files. +# +#----------------------------------------------------------------------- +# +. $USHDIR/create_model_config_file.sh +# +#----------------------------------------------------------------------- +# # Save current shell options (in a global array). Then set new options # for this script/function. # @@ -56,6 +64,7 @@ specified cycle. #----------------------------------------------------------------------- # valid_args=( \ +"cdate" \ "cycle_dir" \ "ensmem_indx" \ "slash_ensmem_subdir" \ @@ -409,15 +418,28 @@ if [ "${USE_CCPP}" = "TRUE" ]; then fi fi +#----------------------------------------------------------------------- +# +# Call the function that creates the model configuration file within each +# cycle directory. +# +#----------------------------------------------------------------------- +# +create_model_config_file \ + cdate="$cdate" \ + run_dir="${run_dir}" || print_err_msg_exit "\ +Call to function to create a model configuration file for the current +cycle's (cdate) run directory (run_dir) failed: + cdate = \"${cdate}\" + run_dir = \"${run_dir}\"" # #----------------------------------------------------------------------- # -# If running enemble forecasts, create links to the cycle-specific -# diagnostic tables file and model configuration file in the cycle -# directory. Note that these links should not be made if not running -# ensemble forecasts because in that case, the cycle directory is the -# run directory (and we would be creating a symlink with the name of a -# file that already exists). +# If running enemble forecasts, create a link to the cycle-specific +# diagnostic tables file in the cycle directory. Note that this link +# should not be made if not running ensemble forecasts because in that +# case, the cycle directory is the run directory (and we would be creating +# a symlink with the name of a file that already exists). # #----------------------------------------------------------------------- # @@ -425,8 +447,6 @@ if [ "${DO_ENSEMBLE}" = "TRUE" ]; then relative_or_null="--relative" diag_table_fp="${cycle_dir}/${DIAG_TABLE_FN}" ln_vrfy -sf ${relative_or_null} ${diag_table_fp} ${run_dir} - model_config_fp="${cycle_dir}/${MODEL_CONFIG_FN}" - ln_vrfy -sf ${relative_or_null} ${model_config_fp} ${run_dir} fi # #----------------------------------------------------------------------- diff --git a/tests/baselines_list.txt b/tests/baselines_list.txt index be021a582..b8b7f9fef 100644 --- a/tests/baselines_list.txt +++ b/tests/baselines_list.txt @@ -1,6 +1,6 @@ GSD_RAP13km -community_ensemble_2mems community_ensemble_008mems +community_ensemble_2mems nco_conus nco_conus_c96 nco_ensemble diff --git a/ush/create_model_config_file.sh b/ush/create_model_config_file.sh new file mode 100644 index 000000000..5192bf886 --- /dev/null +++ b/ush/create_model_config_file.sh @@ -0,0 +1,175 @@ +# +#----------------------------------------------------------------------- +# +# This file defines a function that creates a model configuration file +# for the specified cycle. +# +#----------------------------------------------------------------------- +# +function create_model_config_file() { +# +#----------------------------------------------------------------------- +# +# Save current shell options (in a global array). Then set new options +# for this script/function. +# +#----------------------------------------------------------------------- +# + { save_shell_opts; set -u -x; } > /dev/null 2>&1 +# +#----------------------------------------------------------------------- +# +# Get the full path to the file in which this script/function is located +# (scrfunc_fp), the name of that file (scrfunc_fn), and the directory in +# which the file is located (scrfunc_dir). +# +#----------------------------------------------------------------------- +# + local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + local scrfunc_fn=$( basename "${scrfunc_fp}" ) + local scrfunc_dir=$( dirname "${scrfunc_fp}" ) +# +#----------------------------------------------------------------------- +# +# Get the name of this function. +# +#----------------------------------------------------------------------- +# + local func_name="${FUNCNAME[0]}" +# +#----------------------------------------------------------------------- +# +# Specify the set of valid argument names for this script/function. Then +# process the arguments provided to this script/function (which should +# consist of a set of name-value pairs of the form arg1="value1", etc). +# +#----------------------------------------------------------------------- +# + local valid_args=( +cdate \ +run_dir \ + ) + process_args valid_args "$@" +# +#----------------------------------------------------------------------- +# +# For debugging purposes, print out values of arguments passed to this +# script. Note that these will be printed out only if VERBOSE is set to +# TRUE. +# +#----------------------------------------------------------------------- +# + print_input_args valid_args +# +#----------------------------------------------------------------------- +# +# Declare local variables. +# +#----------------------------------------------------------------------- +# + local i \ + model_config_fp \ + yyyy \ + mm \ + dd \ + hh \ + mm \ + dot_quilting_dot \ + dot_print_esmf_dot +# +#----------------------------------------------------------------------- +# +# Create a model configuration file within each cycle directory. +# +#----------------------------------------------------------------------- +# + print_info_msg "$VERBOSE" " +Creating a model configuration file (\"${MODEL_CONFIG_FN}\") within each +cycle directory..." +# +# Copy template of cycle-dependent model configure files from the templates +# directory to the current cycle directory. +# + model_config_fp="${run_dir}/${MODEL_CONFIG_FN}" + cp_vrfy "${MODEL_CONFIG_TMPL_FP}" "${model_config_fp}" +# +# Extract from cdate the starting year, month, day, and hour of the forecast. +# + yyyy=${cdate:0:4} + mm=${cdate:4:2} + dd=${cdate:6:2} + hh=${cdate:8:2} +# +# Set parameters in the model configure file. +# + dot_quilting_dot="."${QUILTING,,}"." + dot_print_esmf_dot="."${PRINT_ESMF,,}"." + + set_file_param "${model_config_fp}" "PE_MEMBER01" "${PE_MEMBER01}" + set_file_param "${model_config_fp}" "dt_atmos" "${DT_ATMOS}" + set_file_param "${model_config_fp}" "start_year" "$yyyy" + set_file_param "${model_config_fp}" "start_month" "$mm" + set_file_param "${model_config_fp}" "start_day" "$dd" + set_file_param "${model_config_fp}" "start_hour" "$hh" + set_file_param "${model_config_fp}" "nhours_fcst" "${FCST_LEN_HRS}" + set_file_param "${model_config_fp}" "ncores_per_node" "${NCORES_PER_NODE}" + set_file_param "${model_config_fp}" "quilting" "${dot_quilting_dot}" + set_file_param "${model_config_fp}" "print_esmf" "${dot_print_esmf_dot}" +# +#----------------------------------------------------------------------- +# +# If the write component is to be used, then a set of parameters, including +# those that define the write component's output grid, need to be specified +# in the model configuration file (model_config_fp). This is done by +# appending a template file (in which some write-component parameters are +# set to actual values while others are set to placeholders) to model_config_fp +# and then replacing the placeholder values in the (new) model_config_fp +# file with actual values. The full path of this template file is specified +# in the workflow variable WRTCMP_PARAMS_TMPL_FP. +# +#----------------------------------------------------------------------- +# + if [ "$QUILTING" = "TRUE" ]; then + + cat ${WRTCMP_PARAMS_TMPL_FP} >> ${model_config_fp} + + set_file_param "${model_config_fp}" "write_groups" "${WRTCMP_write_groups}" + set_file_param "${model_config_fp}" "write_tasks_per_group" "${WRTCMP_write_tasks_per_group}" + + set_file_param "${model_config_fp}" "output_grid" "\'${WRTCMP_output_grid}\'" + set_file_param "${model_config_fp}" "cen_lon" "${WRTCMP_cen_lon}" + set_file_param "${model_config_fp}" "cen_lat" "${WRTCMP_cen_lat}" + set_file_param "${model_config_fp}" "lon1" "${WRTCMP_lon_lwr_left}" + set_file_param "${model_config_fp}" "lat1" "${WRTCMP_lat_lwr_left}" + + if [ "${WRTCMP_output_grid}" = "rotated_latlon" ]; then + set_file_param "${model_config_fp}" "lon2" "${WRTCMP_lon_upr_rght}" + set_file_param "${model_config_fp}" "lat2" "${WRTCMP_lat_upr_rght}" + set_file_param "${model_config_fp}" "dlon" "${WRTCMP_dlon}" + set_file_param "${model_config_fp}" "dlat" "${WRTCMP_dlat}" + elif [ "${WRTCMP_output_grid}" = "lambert_conformal" ]; then + set_file_param "${model_config_fp}" "stdlat1" "${WRTCMP_stdlat1}" + set_file_param "${model_config_fp}" "stdlat2" "${WRTCMP_stdlat2}" + set_file_param "${model_config_fp}" "nx" "${WRTCMP_nx}" + set_file_param "${model_config_fp}" "ny" "${WRTCMP_ny}" + set_file_param "${model_config_fp}" "dx" "${WRTCMP_dx}" + set_file_param "${model_config_fp}" "dy" "${WRTCMP_dy}" + elif [ "${WRTCMP_output_grid}" = "regional_latlon" ]; then + set_file_param "${model_config_fp}" "lon2" "${WRTCMP_lon_upr_rght}" + set_file_param "${model_config_fp}" "lat2" "${WRTCMP_lat_upr_rght}" + set_file_param "${model_config_fp}" "dlon" "${WRTCMP_dlon}" + set_file_param "${model_config_fp}" "dlat" "${WRTCMP_dlat}" + fi + + fi +# +#----------------------------------------------------------------------- +# +# Restore the shell options saved at the beginning of this script/function. +# +#----------------------------------------------------------------------- +# + { restore_shell_opts; } > /dev/null 2>&1 + +} + diff --git a/ush/create_model_config_files.sh b/ush/create_model_config_files.sh deleted file mode 100644 index d2f9fd357..000000000 --- a/ush/create_model_config_files.sh +++ /dev/null @@ -1,182 +0,0 @@ -# -#----------------------------------------------------------------------- -# -# This file defines a function that creates a model configuration file -# for each cycle to be run. -# -#----------------------------------------------------------------------- -# -function create_model_config_files() { -# -#----------------------------------------------------------------------- -# -# Save current shell options (in a global array). Then set new options -# for this script/function. -# -#----------------------------------------------------------------------- -# - { save_shell_opts; set -u -x; } > /dev/null 2>&1 -# -#----------------------------------------------------------------------- -# -# Get the full path to the file in which this script/function is located -# (scrfunc_fp), the name of that file (scrfunc_fn), and the directory in -# which the file is located (scrfunc_dir). -# -#----------------------------------------------------------------------- -# - local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) - local scrfunc_fn=$( basename "${scrfunc_fp}" ) - local scrfunc_dir=$( dirname "${scrfunc_fp}" ) -# -#----------------------------------------------------------------------- -# -# Get the name of this function. -# -#----------------------------------------------------------------------- -# - local func_name="${FUNCNAME[0]}" -# -#----------------------------------------------------------------------- -# -# Specify the set of valid argument names for this script/function. Then -# process the arguments provided to this script/function (which should -# consist of a set of name-value pairs of the form arg1="value1", etc). -# -#----------------------------------------------------------------------- -# - local valid_args=() - process_args valid_args "$@" -# -#----------------------------------------------------------------------- -# -# For debugging purposes, print out values of arguments passed to this -# script. Note that these will be printed out only if VERBOSE is set to -# TRUE. -# -#----------------------------------------------------------------------- -# - print_input_args valid_args -# -#----------------------------------------------------------------------- -# -# Declare local variables. -# -#----------------------------------------------------------------------- -# - local i \ - cdate \ - cycle_dir \ - model_config_fp \ - yyyy \ - mm \ - dd \ - hh \ - mm \ - dot_quilting_dot \ - dot_print_esmf_dot -# -#----------------------------------------------------------------------- -# -# Create a model configuration file within each cycle directory. -# -#----------------------------------------------------------------------- -# - print_info_msg "$VERBOSE" " -Creating a model configuration file (\"${MODEL_CONFIG_FN}\") within each -cycle directory..." - - for (( i=0; i<${NUM_CYCLES}; i++ )); do - - cdate="${ALL_CDATES[$i]}" - cycle_dir="${CYCLE_BASEDIR}/$cdate" -# -# Copy template of cycle-dependent model configure files from the templates -# directory to the current cycle directory. -# - model_config_fp="${cycle_dir}/${MODEL_CONFIG_FN}" - cp_vrfy "${MODEL_CONFIG_TMPL_FP}" "${model_config_fp}" -# -# Extract from cdate the starting year, month, day, and hour of the forecast. -# - yyyy=${cdate:0:4} - mm=${cdate:4:2} - dd=${cdate:6:2} - hh=${cdate:8:2} -# -# Set parameters in the model configure file. -# - dot_quilting_dot="."${QUILTING,,}"." - dot_print_esmf_dot="."${PRINT_ESMF,,}"." - - set_file_param "${model_config_fp}" "PE_MEMBER01" "${PE_MEMBER01}" - set_file_param "${model_config_fp}" "dt_atmos" "${DT_ATMOS}" - set_file_param "${model_config_fp}" "start_year" "$yyyy" - set_file_param "${model_config_fp}" "start_month" "$mm" - set_file_param "${model_config_fp}" "start_day" "$dd" - set_file_param "${model_config_fp}" "start_hour" "$hh" - set_file_param "${model_config_fp}" "nhours_fcst" "${FCST_LEN_HRS}" - set_file_param "${model_config_fp}" "ncores_per_node" "${NCORES_PER_NODE}" - set_file_param "${model_config_fp}" "quilting" "${dot_quilting_dot}" - set_file_param "${model_config_fp}" "print_esmf" "${dot_print_esmf_dot}" -# -#----------------------------------------------------------------------- -# -# If the write component is to be used, then a set of parameters, in- -# cluding those that define the write component's output grid, need to -# be specified in the model configuration file (model_config_fp). This -# is done by appending a template file (in which some write-component -# parameters are set to actual values while others are set to placehol- -# ders) to model_config_fp and then replacing the placeholder values in -# the (new) model_config_fp file with actual values. The full path of -# this template file is specified in the variable WRTCMP_PA RAMS_TEMP- -# LATE_FP. -# -#----------------------------------------------------------------------- -# - if [ "$QUILTING" = "TRUE" ]; then - - cat ${WRTCMP_PARAMS_TMPL_FP} >> ${model_config_fp} - - set_file_param "${model_config_fp}" "write_groups" "$WRTCMP_write_groups" - set_file_param "${model_config_fp}" "write_tasks_per_group" "$WRTCMP_write_tasks_per_group" - - set_file_param "${model_config_fp}" "output_grid" "\'$WRTCMP_output_grid\'" - set_file_param "${model_config_fp}" "cen_lon" "$WRTCMP_cen_lon" - set_file_param "${model_config_fp}" "cen_lat" "$WRTCMP_cen_lat" - set_file_param "${model_config_fp}" "lon1" "$WRTCMP_lon_lwr_left" - set_file_param "${model_config_fp}" "lat1" "$WRTCMP_lat_lwr_left" - - if [ "${WRTCMP_output_grid}" = "rotated_latlon" ]; then - set_file_param "${model_config_fp}" "lon2" "$WRTCMP_lon_upr_rght" - set_file_param "${model_config_fp}" "lat2" "$WRTCMP_lat_upr_rght" - set_file_param "${model_config_fp}" "dlon" "$WRTCMP_dlon" - set_file_param "${model_config_fp}" "dlat" "$WRTCMP_dlat" - elif [ "${WRTCMP_output_grid}" = "lambert_conformal" ]; then - set_file_param "${model_config_fp}" "stdlat1" "$WRTCMP_stdlat1" - set_file_param "${model_config_fp}" "stdlat2" "$WRTCMP_stdlat2" - set_file_param "${model_config_fp}" "nx" "$WRTCMP_nx" - set_file_param "${model_config_fp}" "ny" "$WRTCMP_ny" - set_file_param "${model_config_fp}" "dx" "$WRTCMP_dx" - set_file_param "${model_config_fp}" "dy" "$WRTCMP_dy" - elif [ "${WRTCMP_output_grid}" = "regional_latlon" ]; then - set_file_param "${model_config_fp}" "lon2" "$WRTCMP_lon_upr_rght" - set_file_param "${model_config_fp}" "lat2" "$WRTCMP_lat_upr_rght" - set_file_param "${model_config_fp}" "dlon" "$WRTCMP_dlon" - set_file_param "${model_config_fp}" "dlat" "$WRTCMP_dlat" - fi - - fi - - done -# -#----------------------------------------------------------------------- -# -# Restore the shell options saved at the beginning of this script/function. -# -#----------------------------------------------------------------------- -# - { restore_shell_opts; } > /dev/null 2>&1 - -} - diff --git a/ush/generate_FV3SAR_wflow.sh b/ush/generate_FV3SAR_wflow.sh index 974033159..18d9d0f81 100755 --- a/ush/generate_FV3SAR_wflow.sh +++ b/ush/generate_FV3SAR_wflow.sh @@ -41,14 +41,13 @@ ushdir="${scrfunc_dir}" # #----------------------------------------------------------------------- # -# Source bash utility functions. +# Source bash utility functions and other necessary files. # #----------------------------------------------------------------------- # . $ushdir/source_util_funcs.sh . $ushdir/set_FV3nml_sfc_climo_filenames.sh . $ushdir/set_FV3nml_stoch_params.sh -. $ushdir/create_model_config_files.sh . $ushdir/create_diag_table_files.sh # #----------------------------------------------------------------------- @@ -255,17 +254,6 @@ done # #----------------------------------------------------------------------- # -# Call the function that creates the model configuration file within each -# cycle directory. -# -#----------------------------------------------------------------------- -# -create_model_config_files || print_err_msg_exit "\ -Call to function to create a model configuration file under each cycle -directory failed." -# -#----------------------------------------------------------------------- -# # For select workflow tasks, copy module files from the various cloned # external repositories to the appropriate subdirectory under the workflow # directory tree. In principle, this is better than having hard-coded