Streamline SCM output #219
Conversation
…ange output_frequency to n_itt_out in case configuration files; add override in case configuration namelist for diagnostics averaging cadence; add checks for various periods (output, diagnostics, swrad, lwrad); update plotting scripts to handle different time periodicity
grantfirl
force-pushed
the
update_output
branch
from
f0c7547 to
2d605be
Compare
|
Should be merged after NCAR/ccpp-physics#541 |
There was a problem hiding this comment.
This is quite a PR to review. Two thoughts came to my mind, (1) looking at scm/etc/scripts/gmtb_scm_analysis.py I am puzzled again that in 2021 the simple task of plotting some data is so complicated - this is not your fault, I think that the software doesn't live up to our requirements (unless it is interactive using jupiter and xarray, but even this is tedious); and (2) are you going to update the documentation in a follow-up PR with all the changes made here?
| @@ -102,7 +118,7 @@ module NetCDF_read | |||
|
|
|||
| implicit none | |||
|
|
|||
| real(kind=dp) :: missing_value = -9999.0 | |||
| real(kind=sp) :: missing_value = -9999.0 | |||
There was a problem hiding this comment.
This is to match the variable type in the NetCDF files, correct? Means all data is written as float and not as double. Can you think of any variable that may need double precision accuracy in the output?
There was a problem hiding this comment.
Yes. Not at the moment, but it is easy enough to change if/when dp is required.
Regarding thought 1, it is at least partly my fault. There is a reason the plotting scripts are "not supported". The whole thing is difficult to maintain, error-prone, and has been grown organically for years. Now that I have a better idea of what functionality is needed, it is ripe for a rewrite using proper OOP. It's part of the HSUP work. Regarding thought 2, my plan was to update the docs before release. There are still more changes coming to the input. |
Streamline SCM output
This PR is designed to make the SCM output routine easier to read/modify for users. A lot of the NetCDF I/O commands were removed to separate utility modules so that users see only pertinent information. Also, changes were made to which variables are being written out. For example, radiative fluxes are the ones coming from dcyc (that are valid for every timestep) rather than only radiation timesteps.
This PR also makes it easier to autogenerate output code from a list of CCPP standard names in the future (after transition to capgen.py).
In addition, variables can be output on different time intervals. As of this PR, there are 4 distinct periods: output (some multiple of the timestep), diagnostics (some multiple of the timestep AND output period), SW rad (some multiple of the timestep, but not necessarily the output period), and LW rad (same as SW rad). The SW/LW rad output periods should also be a multiple of the output period, but if they are not, then their outputs will be written out on their least common multiple. This addresses issues reported by users that some quantities are zero-valued except on radiation timesteps. This also enables use of the 'fhzero' namelist setting as in FV3. The same diagnostics that are time-averaged in FV3 on the fhzero interval are also done so in the SCM now. (This is a change from all variables being instantaneous in all previous versions of this code). To enable this, the calls to Diag%phys_zero and Diag%rad_zero every timestep in gmtb_scm.F90 have been removed in favor of the same calls in gfs_phys_time_vary.scm.F90 (insert ccpp-physics PR here).
Note: It is possible to retain the instantaneous nature of all output by setting fhzero equal to the timestep in hours (this value must have 4 decimal places of accuracy to work correctly). Unlike in FV3, fhzero can be a decimal less than one. Alternatively, one can set the n_itt_diag namelist variable (as an integer multiple of the timestep over which to time average diagnostics) in the case configuration namelist to "override" the value of fhzero. Also note, the output_frequency namelist variable (seconds) in the case configuration namelist has been replaced by an integer n_itt_out, which represents an integer multiple of the timestep, for output period.
The python plotting script has been updated to read the new variables from the SCM output, and handle the different output time periods.
The gmtb_scm_output.F90 file has many changes to make it easier to modify. There are now separate subroutines for initializing and appending variables in the netCDF output file for each GFS DDT. Also notice that there are separate output_append_diag subroutines for instantaneous values and time-averaged variables, since some variables in the Diag DDT are instantaneous and some are time-averaged. Time-averaging is accomplished by passing in the inverse of the number of timesteps in the diagnostic interval to the output subroutine as a multiplicative constant to be applied during the write operation. This can also be used, for example, to create tendencies from time-averaged cumulative changes by the multiplicative constant including the inverse of the time-elapsed in seconds.