Skip to content

Commit

Permalink
Refactor supported input path handling. (#653)
Browse files Browse the repository at this point in the history
## DESCRIPTION OF CHANGES: 
This PR is one of two in a series that addresses refactoring platform-dependence. The topic of this PR is to refactor the logic for supporting platforms with specific real-time data feeds. Here is a list of features and modifications that were made:

- Apply appropriate style for functions, mainly related to white space for nested code and comments.
- Make external model file offset hours configurable for analysis- and forecast-type files because sometimes we like to start a forecast from a forecast of another model.
- Refactor to reduce duplication of information.
- Set up filenames and paths to be specified with templates in a consistent way for every model instead of with bash logic that is different for every model's naming convention.
- Remove paths that do not exist on platforms I have access to: Jet and Hera.

No changes were made to the logic for input managed by USE_USER_STAGED_EXTRN_FILES or COMINGgfs for NCO mode, although it could make sense to re-assess the NCO mode handling at a later date.

I plan to go through and "review" the code to lead reviewers through this one since it bit of change. It may be helpful to view it using GitHub's ignore whitespace feature. 

## TESTS CONDUCTED: 
Test cases using the WE2E test on Hera; see PR for full list.
   
A test case for the same forecast configuration using known paths on Hera to exercise the new code. 

I checked that arrays were consistent, that the script exits in a sane manner when files are not available on disk or HPSS, and that I haven't broken anything with the way files are handled through the "user specified" mechanism necessary for the test framework.

## ISSUE (optional): 
This work is an incremental change in support of Issue #618 

## CONTRIBUTORS (optional): 
@christopherwharrop-noaa @venitahagerty @robgonzalezpita
  • Loading branch information
christinaholtNOAA authored Jan 13, 2022
1 parent 4300bb5 commit a5eba26
Show file tree
Hide file tree
Showing 4 changed files with 477 additions and 975 deletions.
25 changes: 13 additions & 12 deletions jobs/JREGIONAL_GET_EXTRN_MDL_FILES
Original file line number Diff line number Diff line change
Expand Up @@ -167,28 +167,29 @@ check_var_valid_value "ICS_OR_LBCS" "valid_vals_ICS_OR_LBCS"
#
#-----------------------------------------------------------------------
#
# Set the parameter anl_or_fcst that determines whether we want to get
# analysis or forecast files. This depends on whether we want these files
# to generate initial condition and surface field files or lateral boundary
# condition files. Also, set time_offset_hrs, which is the offset in
# hours between the current cycle's starting time and the starting time
# of the external model providing the LBCs.
# Set parameters for grabbing either the initial conditions from analysis or
# forecast files of external models, or the lateral boundary conditions
# from external models. The script has been called to do the work for
# one or the other.
#
#-----------------------------------------------------------------------
#
if [ "${ICS_OR_LBCS}" = "ICS" ]; then
anl_or_fcst="ANL"
time_offset_hrs="0"
if [ ${EXTRN_MDL_ICS_OFFSET_HRS} -eq 0 ] ; then
anl_or_fcst="ANL"
time_offset_hrs=0
else
anl_or_fcst="FCST"
time_offset_hrs=${EXTRN_MDL_ICS_OFFSET_HRS:-0}
fi
elif [ "${ICS_OR_LBCS}" = "LBCS" ]; then
anl_or_fcst="FCST"
time_offset_hrs="${EXTRN_MDL_LBCS_OFFSET_HRS}"
time_offset_hrs=${EXTRN_MDL_LBCS_OFFSET_HRS:-0}
fi
#
#-----------------------------------------------------------------------
#
# Set the name of and then create the directory in which to stage the
# external model files for the current cycle (if it doesn't already exist).
# Then change location to that directory.
# Create the directory where the exetrnal model files should be stored
#
#-----------------------------------------------------------------------
#
Expand Down
20 changes: 20 additions & 0 deletions ush/config_defaults.sh
Original file line number Diff line number Diff line change
Expand Up @@ -604,6 +604,24 @@ NDAS_OBS_DIR="/path/to/observation-directory/ndas/proc"
# data availble at least every 6 hours. It is up to the user to ensure
# that this is the case.
#
# EXTRN_MDL_ICS_OFFSET_HRS:
# Users may wish to start a forecast from a forecast of a previous cycle
# of an external model. This variable sets the number of hours earlier
# the external model started than when the FV3 forecast configured here
# should start. For example, the forecast should start from a 6 hour
# forecast of the GFS, then EXTRN_MDL_ICS_OFFSET_HRS=6.

# EXTRN_MDL_LBCS_OFFSET_HRS:
# Users may wish to use lateral boundary conditions from a forecast that
# was started earlier than the initial time for the FV3 forecast
# configured here. This variable sets the number of hours earlier
# the external model started than when the FV3 forecast configured here
# should start. For example, the forecast should use lateral boundary
# conditions from the GFS started 6 hours earlier, then
# EXTRN_MDL_LBCS_OFFSET_HRS=6.
# Note: the default value is model-dependent and set in
# set_extrn_mdl_params.sh
#
# FV3GFS_FILE_FMT_ICS:
# If using the FV3GFS model as the source of the ICs (i.e. if EXTRN_MDL_NAME_ICS
# is set to "FV3GFS"), this variable specifies the format of the model
Expand All @@ -619,6 +637,8 @@ NDAS_OBS_DIR="/path/to/observation-directory/ndas/proc"
EXTRN_MDL_NAME_ICS="FV3GFS"
EXTRN_MDL_NAME_LBCS="FV3GFS"
LBC_SPEC_INTVL_HRS="6"
EXTRN_MDL_ICS_OFFSET_HRS="0"
EXTRN_MDL_LBCS_OFFSET_HRS=""
FV3GFS_FILE_FMT_ICS="nemsio"
FV3GFS_FILE_FMT_LBCS="nemsio"
#
Expand Down
Loading

0 comments on commit a5eba26

Please sign in to comment.