Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the new capability for MRI-4DVar #1330

Merged
merged 96 commits into from
Dec 11, 2020

Conversation

liujake
Copy link
Contributor

@liujake liujake commented Nov 29, 2020

This large PR adds the new capability for multi-resolution incremental 4DVar.

TYPE: new feature

KEYWORDS: MRI-4DVar, Inverse transform, square root of B, 3-stage

SOURCE: Zhiquan (Jake) Liu, Junmei Ban, Jamie Bresch, Yali Wu, Xin Zhang, Jianyu Liu (NCAR/MMM)

DESCRIPTION OF CHANGES:
Existing 4DVar algorithm is run at the same model resolution for inner loop minimization of different outer loops (default namelist setting with multi_inc=0). This adds MRI-4DVar capability that can run 4DVar at lower resolution for inner loop minimization with a 3-stage procedure (observer, minimizer, regridder) as described by Liu et al. (2020) and can substantially speed up 4DVar. Three major changes include:

  1. 3-stage procedure controlled by a script and offline programs under ~var/mri4dvar;
    Set multi_inc=1 for the "observer" step and multi_inc=2 for the "minimizer" step. The "regridder' step is done using the offline programs.

  2. Inverse transform of square root B introduced under ~var/da/da_vtox_transforms;
    New namelist parameter use_inverse_squarerootb needs to be set .true. when multi_inc=2 and works together with cv_options=5, 6, or 7 and cloud_cv_options=2 or 3.

New namelist parameter use_interpolate_cvt=.false. (default), which is supposed to work together with the obsolete cv_options=3, which is not recommended to use for non-expert users.

  1. Additional file I/O for control variable and radar/radiance OMB files from the observer step. This I/O part will need some further code clean-up and improvement in the future.

LIST OF MODIFIED FILES: 48 files
M Registry/registry.var
M var/build/depend.txt
M var/da/da_main/da_solve.inc
M var/da/da_main/da_wrfvar_top.f90
M var/da/da_minimisation/da_get_innov_vector.inc
M var/da/da_minimisation/da_minimisation.f90
M var/da/da_obs_io/da_obs_io.f90
M var/da/da_obs_io/da_read_iv_for_multi_inc.inc
M var/da/da_obs_io/da_read_obs_radar.inc
M var/da/da_obs_io/da_scan_obs_radar.inc
M var/da/da_obs_io/da_search_obs.inc
M var/da/da_obs_io/da_write_iv_for_multi_inc.inc
M var/da/da_radiance/da_radiance1.f90
A var/da/da_radiance/da_read_iv_rad_for_multi_inc.inc
A var/da/da_radiance/da_write_iv_rad_for_multi_inc.inc
M var/da/da_recursive_filter/da_recursive_filter.f90
A var/da/da_recursive_filter/da_recursive_filter_1d_inv.inc
M var/da/da_recursive_filter/da_transform_through_rf.inc
A var/da/da_recursive_filter/da_transform_through_rf_inv.inc
M var/da/da_setup_structures/da_setup_structures.f90
A var/da/da_setup_structures/da_write_vp.inc
M var/da/da_tools/da_llxy.inc
M var/da/da_vtox_transforms/da_transform_vptox.inc
A var/da/da_vtox_transforms/da_transform_vptox_inv.inc
A var/da/da_vtox_transforms/da_transform_vtovv_inv.inc
A var/da/da_vtox_transforms/da_transform_vtox_inv.inc
M var/da/da_vtox_transforms/da_transform_vvtovp.inc
M var/da/da_vtox_transforms/da_transform_vvtovp_adj.inc
A var/da/da_vtox_transforms/da_transform_vvtovp_inv.inc
M var/da/da_vtox_transforms/da_vertical_transform.inc
M var/da/da_vtox_transforms/da_vtox_transforms.f90
A var/mri4dvar/Makefile
A var/mri4dvar/README.MRI-4DVar
A var/mri4dvar/da_bdy.f90
A var/mri4dvar/da_bilin.f90
A var/mri4dvar/da_thin.f90
A var/mri4dvar/da_vp_bilin.f90
A var/mri4dvar/da_vp_split.f90
A var/mri4dvar/nc_increment.ncl
A var/mri4dvar/nc_vpglobal.ncl
A var/mri4dvar/nc_vphires.ncl
A var/mri4dvar/nc_vpinput.ncl
A var/mri4dvar/nc_vplocal.ncl
A var/mri4dvar/rsl_lite.h
A var/mri4dvar/run_mri3d4dvar.csh_lsf
A var/mri4dvar/run_mri3d4dvar.csh_pbs
A var/mri4dvar/task_for_point.c
A var/mri4dvar/wraper_mri3d4dvar.csh

TESTS CONDUCTED: WRFDA regression test passed. Cycling MRI-4DVar tests were performed over an Arctic domain.

RELEASE NOTE: New capability for Multi-Resolution Incremental 4DVar (MRI-4DVar).
Liu, Z., J. Ban, J.-S, Hong, and Y.-H. Kuo, 2020: Multi-resolution incremental 4D-Var for WRF: Implementation and application at convective scale, Q. J. R. Meteorol. Soc., 146, 3661-3674.

Xiaowen Tang of Nanjing University, China

modified:   Registry/registry.var
modified:   var/build/depend.txt
modified:   var/da/da_define_structures/da_define_structures.f90
new file:   var/da/da_dynamics/da_divergence_constraint.inc
new file:   var/da/da_dynamics/da_divergence_constraint_adj.inc
modified:   var/da/da_dynamics/da_dynamics.f90
modified:   var/da/da_minimisation/da_calculate_gradj.inc
modified:   var/da/da_minimisation/da_calculate_j.inc
modified:   var/da/da_minimisation/da_get_var_diagnostics.inc
modified:   var/da/da_minimisation/da_minimisation.f90
modified:   var/da/da_test/da_check.inc
new file:   var/da/da_test/da_check_dynamics_adjoint.inc
modified:   var/da/da_test/da_test.f90
Xiaowen Tang of Nanjing University, China

modified:   Registry/registry.var
modified:   var/build/depend.txt
modified:   var/da/da_obs/da_fill_obs_structures.inc
modified:   var/da/da_obs_io/da_obs_io.f90
new file:   var/da/da_obs_io/da_read_lsac_util.inc
new file:   var/da/da_obs_io/da_read_obs_lsac.inc
new file:   var/da/da_obs_io/da_scan_obs_lsac.inc
modified:   var/da/da_setup_structures/da_setup_obs_structures.inc
modified:   var/da/da_setup_structures/da_setup_obs_structures_ascii.inc
modified:   var/da/da_setup_structures/da_setup_structures.f90
Shibo Gao of Nanjing University of Information Science & Technology, China.

MPI re-coded by Jamie Bresch of NCAR.

modified:   Registry/registry.var
modified:   var/da/da_minimisation/da_get_innov_vector.inc
modified:   var/da/da_minimisation/da_minimisation.f90
modified:   var/da/da_radar/da_get_innov_vector_radar.inc
modified:   var/da/da_radar/da_radar.f90
new file:   README.CWB_v39a
git cherry-pick -n 070d870 96fb5f3

------
Merge 2016's MRI-4DVAR code changes into WRFDA V3.9.
Tested on NCAR HPC cheyenne and obtained similar result to previous V3.8-based
code ran on yellowstone.
------

modified:   Registry/registry.var
modified:   var/build/depend.txt
modified:   var/da/da_main/da_solve.inc
modified:   var/da/da_main/da_wrfvar_top.f90
modified:   var/da/da_recursive_filter/da_recursive_filter.f90
new file:   var/da/da_recursive_filter/da_recursive_filter_1d_inv.inc
modified:   var/da/da_recursive_filter/da_transform_through_rf.inc
new file:   var/da/da_recursive_filter/da_transform_through_rf_inv.inc
modified:   var/da/da_setup_structures/da_setup_structures.f90
new file:   var/da/da_setup_structures/da_write_vp.inc
modified:   var/da/da_vtox_transforms/da_transform_vptox.inc
new file:   var/da/da_vtox_transforms/da_transform_vptox_inv.inc
new file:   var/da/da_vtox_transforms/da_transform_vtovv_inv.inc
new file:   var/da/da_vtox_transforms/da_transform_vtox_inv.inc
modified:   var/da/da_vtox_transforms/da_transform_vvtovp.inc
modified:   var/da/da_vtox_transforms/da_transform_vvtovp_adj.inc
new file:   var/da/da_vtox_transforms/da_transform_vvtovp_inv.inc
modified:   var/da/da_vtox_transforms/da_vertical_transform.inc
modified:   var/da/da_vtox_transforms/da_vtox_transforms.f90
…ecip_opt=1 bug fix.

modified:   README.CWB_v39a
DIVC works fine with the CWB_v381a code.

CWB_v39a (CWB_v39f2a) includes the new 4DEnsVar capability that
requires "call da_zero_vp_type (vp)" to be removed from
subroutine da_transform_vtox_adj. Therefore, da_zero_vp_type
must now be called before "call da_transform_vtox_adj".

modified:   README.CWB_v39a
modified:   var/da/da_minimisation/da_calculate_gradj.inc
modified:   var/da/da_minimisation/da_calculate_j.inc
The original gen_be_ep2.f90 is renamed to gen_be_ep2_serial.f90 and
its executable is gen_be_ep2_serial.exe.

The new gen_be_ep2.f90 is a lot faster than the original one
even when running in serial mode.
The main improvements are in:
(1) using WRF netCDF ioapi to replace inefficient reading in 2D slices.
(2) eliminating large temporary outputs.

The compilation mode for the new gen_be_ep2.f90 is decided by
the option chosen when 'configure wrfda', either serial or dmpar.

modified:   README.CWB_v39a
modified:   var/build/depend.txt
modified:   var/build/gen_be.make
modified:   var/gen_be/Makefile
modified:   var/gen_be/gen_be_ep2.f90
copied:     var/gen_be/gen_be_ep2.f90 -> var/gen_be/gen_be_ep2_serial.f90
lsac_nvstart:  index of starting vertical grid point
lsac_use_u:    switch for large scale u analysis constraint
lsac_use_v:    switch for large scale v analysis constraint
lsac_use_t:    switch for large scale t analysis constraint
lsac_use_q:    switch for large scale q analysis constraint

modified:   Registry/registry.var
modified:   var/da/da_obs_io/da_obs_io.f90
modified:   var/da/da_obs_io/da_read_lsac_util.inc
…n repository for the coming V3.9.1 release.

(git cherry-pick -n db7841c 49ec556 3e3c4ce ee3fd4a c4eeff5 81ca2ff d21f0db c7405bb)
 1. Bug fix and clean-up for WRFDA pseudo ob capability.
 2. Bug fix for ZTD with 4DVAR when there are ZTD obs in non-first time slots.
 3. Fix incorrect calculation of an unused variable cv_size_domain_jb.
 4. WRFDA registry.var fixes for packaging moist variables and for non-4DVAR.
    This reduces non-4DVAR memory usage by ~35%.
 5. Add packaging in registry.var for WRFDA derived type variables.
    This reduces 3DVAR memory usage by another ~15-20%.

modified:   README.CWB_v39a
modified:   Registry/registry.var
modified:   tools/gen_allocs.c
modified:   tools/gen_scalar_indices.c
modified:   tools/protos.h
modified:   var/build/da_name_space.pl
modified:   var/da/da_gpspw/da_transform_xtoy_gpsztd.inc
modified:   var/da/da_main/da_med_initialdata_input.inc
modified:   var/da/da_main/da_solve.inc
modified:   var/da/da_main/da_update_firstguess.inc
modified:   var/da/da_main/da_wrfvar_init2.inc
modified:   var/da/da_main/da_wrfvar_io.f90
modified:   var/da/da_minimisation/da_calculate_gradj.inc
modified:   var/da/da_minimisation/da_calculate_j.inc
modified:   var/da/da_minimisation/da_sensitivity.inc
modified:   var/da/da_minimisation/da_transform_vtoy.inc
modified:   var/da/da_minimisation/da_transform_vtoy_adj.inc
modified:   var/da/da_radar/da_radar.f90
modified:   var/da/da_radar/da_transform_xtoy_radar.inc
modified:   var/da/da_radar/da_transform_xtoy_radar_adj.inc
modified:   var/da/da_radiance/da_crtm.f90
modified:   var/da/da_radiance/da_transform_xtoy_crtm.inc
modified:   var/da/da_radiance/da_transform_xtoy_crtm_adj.inc
modified:   var/da/da_setup_structures/da_setup_cv.inc
modified:   var/da/da_setup_structures/da_setup_obs_structures.inc
modified:   var/da/da_setup_structures/da_setup_pseudo_obs.inc
modified:   var/da/da_setup_structures/da_setup_structures.f90
modified:   var/da/da_test/da_check_dynamics_adjoint.inc (this is in CWB_v39a branch only)
modified:   var/da/da_test/da_check_vtoy_adjoint.inc
modified:   var/da/da_test/da_check_xtoy_adjoint.inc
modified:   var/da/da_transfer_model/da_transfer_model.f90
modified:   var/da/da_transfer_model/da_transfer_wrftltoxa.inc
modified:   var/da/da_transfer_model/da_transfer_wrftltoxa_adj.inc
modified:   var/da/da_transfer_model/da_transfer_xatowrftl.inc
modified:   var/da/da_transfer_model/da_transfer_xatowrftl_adj.inc
Contributed by Siou-Ying Jiang.
Fix the calculation of Vt component of Vr by adding density
term in the Vt calculation and making qrain unit consistent.

modified:   var/da/da_radar/da_get_innov_vector_radar.inc
modified:   var/da/da_radar/da_radial_velocity.inc
modified:   var/da/da_radar/da_radial_velocity_adj.inc
modified:   var/da/da_radar/da_radial_velocity_lin.inc
modified:   var/da/da_radar/da_transform_xtoy_radar.inc
modified:   var/da/da_radar/da_transform_xtoy_radar_adj.inc
1. Implement obs-model height difference check
2. Write out ZTD innov info properly

modified:   README.CWB_v39a
modified:   Registry/registry.var
modified:   var/da/da_gpspw/da_get_innov_vector_gpsztd.inc
modified:   var/da/da_gpspw/da_gpspw.f90
modified:   var/da/da_physics/da_physics.f90
modified:   var/da/da_physics/da_transform_xtoztd_adj.inc
modified:   var/da/da_physics/da_transform_xtoztd_lin.inc
1. fix the staggering variable problem
2. update namelists
lsac_calcerr is removed, some names are changed and some optinos are added.
The current LSAC namelist options and the defaults are:
  use_lsac            .false.
  lsac_nh_step        1
  lsac_nv_step        1
  lsac_nv_start       1
  lsac_use_u          .true.
  lsac_use_v          .true.
  lsac_use_t          .true.
  lsac_use_q          .true.
  lsac_u_error        2.5 (m/s)
  lsac_v_error        2.5 (m/s)
  lsac_t_error        2.0 (C)
  lsac_q_error        0.002 (kg/kg)
  lsac_print_details  .false.

modified:   Registry/registry.var
modified:   var/da/da_obs_io/da_obs_io.f90
modified:   var/da/da_obs_io/da_read_lsac_util.inc
git cherry-pick -n 5e94060 606ac0e 4cc9707 85e4d11 baa3fe6

1. add offline programs and scripts for MRI-4DVAR
2. add w and cloud control variables for MRI-4DVAR
3. allow properly reading of BE file with cloud/w variables generated by stand-alone GEN_BE
4. bug fix for radar data OMB file I/O for MRI-4DVAR

modified:   var/da/da_main/da_solve.inc
modified:   var/da/da_obs_io/da_search_obs.inc
modified:   var/da/da_recursive_filter/da_transform_through_rf_inv.inc
modified:   var/da/da_setup_structures/da_setup_be_regional.inc
modified:   var/da/da_setup_structures/da_write_vp.inc
modified:   var/da/da_vtox_transforms/da_transform_vptox_inv.inc
modified:   var/da/da_vtox_transforms/da_transform_vtovv_inv.inc
modified:   var/da/da_vtox_transforms/da_transform_vtox_inv.inc
new file:   var/mri4dvar/Makefile
new file:   var/mri4dvar/README.Multi_inc
new file:   var/mri4dvar/da_bdy.f90
new file:   var/mri4dvar/da_bilin.f90
new file:   var/mri4dvar/da_thin.f90
new file:   var/mri4dvar/da_vp_bilin.f90
new file:   var/mri4dvar/da_vp_split.f90
new file:   var/mri4dvar/nc_increment.ncl
new file:   var/mri4dvar/nc_vpglobal.ncl
new file:   var/mri4dvar/nc_vphires.ncl
new file:   var/mri4dvar/nc_vpinput.ncl
new file:   var/mri4dvar/nc_vplocal.ncl
new file:   var/mri4dvar/rsl_lite.h
new file:   var/mri4dvar/run_mri3d4dvar.csh_lsf
new file:   var/mri4dvar/run_mri3d4dvar.csh_pbs
new file:   var/mri4dvar/task_for_point.c
new file:   var/mri4dvar/wraper_mri3d4dvar.csh
…mosaic data.

For CWB's radar data assimilation, the radar data are pre-processed
to be on mass points of the model grid.
When WRFDA reads in the radar mosaic data on model mass points,
it prints out a lot of "Report is outside of domain" messages when
print_detail_radar=.true.

Those "outside of domain" obs appear to be along the domain boundaries.

When calculating grid x/y from lat/lon (by calling da_llxy),
the output x/y are floating numbers and they do not match exactly the grid index.
For example, by giving the lat/lon of (450,1),
the calculated x is 450.000183 and y is 0.999511719.
A few more examples are listed below:
  (450,   2) :  (450.000183,    2.00000000)
  (450,   3) :  (450.000153,    3.00000000)
  (450,   4) :  (450.000153,    3.99951172)
  (450,   5) :  (450.000092,    5.00048828)
  (450,   6) :  (450.000061,    6.00000000)
  (446, 450) :  (446.000580,  450.000000)
  (447, 450) :  (446.999146,  449.999512)
  (448, 450) :  (448.000671,  449.999512)
  (449, 450) :  (448.999268,  450.000000)
  (450, 450) :  (450.000793,  450.000000)

By removing the "=" sign in line 52 and line 53 of var/da/da_tools/da_llxy.inc,
the "outside of domain" prints can be reduced.

Another solution to reducing run-time output is to set print_detail_radar=.false.
for operational runs.

modified:   var/da/da_tools/da_llxy.inc
…ound in obsproc.

modified:   var/obsproc/src/module_qc.F90
…om different sources.

ccv_be_inp_opt=0 (default), original hard-coded values used by cloud_cv_options=3.
ccv_be_inp_opt=1, user-specified values in be_ccv.txt and be_w.txt to be used by cloud_cv_options=3.
ccv_be_inp_opt=2, the same behavior as used in mri4dvar for cloud_cv_options=2.
ccv_be_inp_opt=3, univariate cloud and w BES in independent files for cloud_cv_options=2.

Note that the code compiles but has not yet been tested.

modified:   Registry/registry.var
modified:   var/da/da_setup_structures/da_setup_be_regional.inc
modified:   var/da/da_setup_structures/da_setup_structures.f90
But note that for now use_cv_w also uses the same ccv_be_inp_opt intended for cloud cv.
A new namelist, wcv_be_inp_opt, can be added in the future if desired.

modified:   var/da/da_setup_structures/da_setup_be_regional.inc
Set retrieved qr, qs, qg to zero when echo is non-precip.

modified:   var/da/da_radar/da_get_innov_vector_radar.inc
Also set the default Max_StHeight_Diff_ztd a large value for backward compatibility
when users do not explicitly set Max_StHeight_Diff_ztd.

The same code has been in the master repository as of commit bc60b4c.

modified:   Registry/registry.var
modified:   var/da/da_gpspw/da_get_innov_vector_gpsztd.inc
modified:   var/da/da_gpspw/da_gpspw.f90
…error_options=2.

Add checks for missing Q obs to set proper Q qc flags.

I-Han Chen reported the problem and helped with the test.

modified:   var/da/da_synop/da_get_innov_vector_synop.inc
…tion.

The same code has been in the master repository as of commit bd9a460.

modified:   external/RSL_LITE/module_dm.F
modified:   var/da/da_define_structures/da_allocate_y.inc
modified:   var/da/da_define_structures/da_allocate_y_radar.inc
modified:   var/da/da_define_structures/da_deallocate_observations.inc
modified:   var/da/da_define_structures/da_define_structures.f90
modified:   var/da/da_obs_io/da_obs_io.f90
modified:   var/da/da_obs_io/da_read_obs_radar.inc
modified:   var/da/da_radar/da_ao_stats_radar.inc
modified:   var/da/da_radar/da_get_innov_vector_radar.inc
modified:   var/da/da_radar/da_jo_and_grady_radar.inc
modified:   var/da/da_radar/da_oi_stats_radar.inc
modified:   var/da/da_radar/da_radar.f90
modified:   var/da/da_radar/da_transform_xtoy_radar.inc
modified:   var/da/da_radar/da_transform_xtoy_radar_adj.inc
modified:   var/da/da_setup_structures/da_setup_obs_structures_radar.inc
This fixes the unreasonable 4DVAR QICE and QGRAUP increments when
only surface observations are assimilated with cloud_cv_options=3
and mp_physics=99, a case reported by Ihan.
This should also fix other subtle 4DVAR moist problems.
However, there are probably more fixes needed for rainfall DA.

modified:   Registry/registry.var
modified:   var/da/da_main/da_update_firstguess.inc
modified:   var/da/da_main/da_wrfvar_init2.inc
modified:   var/da/da_transfer_model/da_transfer_model.f90
modified:   var/da/da_transfer_model/da_transfer_wrftltoxa.inc
modified:   var/da/da_transfer_model/da_transfer_wrftltoxa_adj.inc
modified:   var/da/da_transfer_model/da_transfer_xatowrftl.inc
modified:   var/da/da_transfer_model/da_transfer_xatowrftl_adj.inc
To be consistent with what had been committed to the master repository.
1. Change default of mp_physics_ad to 99
2. Remove some unused variables
3. Fix a couple typos (g_->a_)

modified:   Registry/registry.var
modified:   var/da/da_setup_structures/da_setup_be_regional.inc
… sfc wind correction.

As in figure 2 of Stauffer et al. 1991, MWR,
0<z0<0.2 : u40/u10=1.000+0.320*z0**0.2
z0>=0.2 : u40/u10=1.169+0.315*z0

modified:   var/da/da_tools/da_mo_correction.inc
modified:   Registry/registry.var
modified:   var/build/depend.txt
modified:   var/da/da_minimisation/da_get_innov_vector.inc
modified:   var/da/da_minimisation/da_minimisation.f90
modified:   var/da/da_obs_io/da_obs_io.f90
new file:   var/da/da_obs_io/da_read_iv_for_multi_inc_opt2.inc
new file:   var/da/da_obs_io/da_write_iv_for_multi_inc_opt2.inc
…k_max_iv prints.

modified:   var/da/da_obs_io/da_read_obs_radar.inc
modified:   var/da/da_radar/da_get_innov_vector_radar.inc

Missing rv are set incorrectly as no-rain rf in NCAR-processed radar data.
The fix is to set qc to missing_data for -999.99 rv in var/da/da_radar/da_get_innov_vector_radar.inc

NCAR-processed radar data sample
FM-128 RADAR   2017-06-02_03:06:00        22.961       117.830      38.0       1
         5158.0    -999.990   0       0.500      -999.990   0       0.000     -8888.880

Missing rv/rf has error of missing_r in CWB-processed radar data.
The fix is to set a default error value in var/da/da_obs_io/da_read_obs_radar.inc.

CWB-processed radar data sample 1
FM-128 RADAR   2017-07-06_06:00:00        19.570       117.335       0.0       1
        12000.0 -888888.000 -88 -888888.000      -999.990   0       0.000

CWB-processed radar data sample 2
FM-128 RADAR   2017-07-06_06:02:00        24.134       122.223      63.0       1
          946.0       1.300   0       2.000   -888888.000 -88 -888888.000
radar_rhv_err_opt:
 1 (original; errors calculated in da_get_innov_vector_radar)
 2 (new; use error settings from new namelists radar_rhv_rrn_err, radar_rhv_rsn_err, radar_rhv_rgr_err)

modified:   Registry/registry.var
modified:   var/da/da_radar/da_get_innov_vector_radar.inc
modified:   var/da/da_radar/da_radar.f90
  Fixes so that code compiles Ok.
	modified:   Registry/registry.var
	modified:   var/da/da_main/da_solve.inc
	modified:   var/da/da_main/da_wrfvar_top.f90
	modified:   var/da/da_radiance/da_radiance1.f90
  remove obsolete da_read_iv_rad_ascii.inc
	modified:   var/da/da_main/da_solve.inc
	deleted:    var/da/da_radiance/da_read_iv_rad_ascii.inc
	modified:   var/mri4dvar/run_mri3d4dvar.csh_lsf
	modified:   var/mri4dvar/run_mri3d4dvar.csh_pbs
	modified:   var/mri4dvar/wraper_mri3d4dvar.csh
…th versions.

	modified:   Registry/registry.var
	modified:   var/da/da_minimisation/da_get_innov_vector.inc
	modified:   var/da/da_minimisation/da_minimisation.f90
	modified:   var/da/da_obs_io/da_obs_io.f90
	modified:   var/da/da_obs_io/da_read_iv_for_multi_inc.inc
	modified:   var/da/da_obs_io/da_write_iv_for_multi_inc.inc
	deleted:    var/da/da_obs_io/da_write_iv_for_multi_inc_opt2.inc
	modified:   var/build/depend.txt
	modified:   var/da/da_radiance/da_write_filtered_rad.inc
	modified:   var/da/da_radiance/da_write_iv_rad_ascii.inc
	modified:   var/da/da_radiance/da_write_oa_rad_ascii.inc
	modified:   var/da/da_setup_structures/da_setup_structures.f90
	modified:   var/da/da_tools/da_get_time_slots.inc
	modified:   var/da/da_radiance/da_radiance1.f90
	modified:   var/da/da_radiance/da_read_iv_rad_for_multi_inc.inc
	modified:   var/da/da_radiance/da_write_iv_rad_for_multi_inc.inc
	modified:   var/da/da_main/da_solve.inc
@liujake liujake marked this pull request as ready for review December 11, 2020 01:19
@liujake liujake requested a review from a team as a code owner December 11, 2020 01:19
@liujake
Copy link
Contributor Author

liujake commented Dec 11, 2020

@davegill Listed you as a reviewer just in case you'd like to have a quick scan in PR message (and/or code) for this relatively large PR. No impact to the model side. WRFDA regtest passed. Junmei should review it.

Copy link

@junmeiban junmeiban left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested this PR on OSC HPC owens machine.
Two cycling experiments(135km45km and 45km45km) are conducted, and both of them run successfully.

@liujake liujake merged commit 3e35ef7 into wrf-model:develop Dec 11, 2020
@liujake liujake deleted the latest_develop_mri4dvar branch April 11, 2022 15:56
vlakshmanan-scala pushed a commit to scala-computing/WRF that referenced this pull request Apr 4, 2024
This large PR adds the new capability for multi-resolution incremental 4DVar.

TYPE: new feature

KEYWORDS: MRI-4DVar, Inverse transform, square root of B, 3-stage

SOURCE: Zhiquan (Jake) Liu, Junmei Ban, Jamie Bresch, Yali Wu, Xin Zhang, Jianyu Liu (NCAR/MMM)

DESCRIPTION OF CHANGES:
Existing 4DVar algorithm is run at the same model resolution for inner loop minimization of different outer loops (default namelist setting with multi_inc=0). This adds MRI-4DVar capability that can run 4DVar at lower resolution for inner loop minimization with a 3-stage procedure (observer, minimizer, regridder) as described by Liu et al. (2020) and can substantially speed up 4DVar. Three major changes include:

3-stage procedure controlled by a script and offline programs under ~var/mri4dvar;
Set multi_inc=1 for the "observer" step and multi_inc=2 for the "minimizer" step. The "regridder' step is done using the offline programs.

Inverse transform of square root B introduced under ~var/da/da_vtox_transforms;
New namelist parameter use_inverse_squarerootb needs to be set .true. when multi_inc=2 and works together with cv_options=5, 6, or 7 and cloud_cv_options=2 or 3.

New namelist parameter use_interpolate_cvt=.false. (default), which is supposed to work together with the obsolete cv_options=3, which is not recommended to use for non-expert users.

Additional file I/O for control variable and radar/radiance OMB files from the observer step. This I/O part will need some further code clean-up and improvement in the future.
LIST OF MODIFIED FILES: 48 files
M Registry/registry.var
M var/build/depend.txt
M var/da/da_main/da_solve.inc
M var/da/da_main/da_wrfvar_top.f90
M var/da/da_minimisation/da_get_innov_vector.inc
M var/da/da_minimisation/da_minimisation.f90
M var/da/da_obs_io/da_obs_io.f90
M var/da/da_obs_io/da_read_iv_for_multi_inc.inc
M var/da/da_obs_io/da_read_obs_radar.inc
M var/da/da_obs_io/da_scan_obs_radar.inc
M var/da/da_obs_io/da_search_obs.inc
M var/da/da_obs_io/da_write_iv_for_multi_inc.inc
M var/da/da_radiance/da_radiance1.f90
A var/da/da_radiance/da_read_iv_rad_for_multi_inc.inc
A var/da/da_radiance/da_write_iv_rad_for_multi_inc.inc
M var/da/da_recursive_filter/da_recursive_filter.f90
A var/da/da_recursive_filter/da_recursive_filter_1d_inv.inc
M var/da/da_recursive_filter/da_transform_through_rf.inc
A var/da/da_recursive_filter/da_transform_through_rf_inv.inc
M var/da/da_setup_structures/da_setup_structures.f90
A var/da/da_setup_structures/da_write_vp.inc
M var/da/da_tools/da_llxy.inc
M var/da/da_vtox_transforms/da_transform_vptox.inc
A var/da/da_vtox_transforms/da_transform_vptox_inv.inc
A var/da/da_vtox_transforms/da_transform_vtovv_inv.inc
A var/da/da_vtox_transforms/da_transform_vtox_inv.inc
M var/da/da_vtox_transforms/da_transform_vvtovp.inc
M var/da/da_vtox_transforms/da_transform_vvtovp_adj.inc
A var/da/da_vtox_transforms/da_transform_vvtovp_inv.inc
M var/da/da_vtox_transforms/da_vertical_transform.inc
M var/da/da_vtox_transforms/da_vtox_transforms.f90
A var/mri4dvar/Makefile
A var/mri4dvar/README.MRI-4DVar
A var/mri4dvar/da_bdy.f90
A var/mri4dvar/da_bilin.f90
A var/mri4dvar/da_thin.f90
A var/mri4dvar/da_vp_bilin.f90
A var/mri4dvar/da_vp_split.f90
A var/mri4dvar/nc_increment.ncl
A var/mri4dvar/nc_vpglobal.ncl
A var/mri4dvar/nc_vphires.ncl
A var/mri4dvar/nc_vpinput.ncl
A var/mri4dvar/nc_vplocal.ncl
A var/mri4dvar/rsl_lite.h
A var/mri4dvar/run_mri3d4dvar.csh_lsf
A var/mri4dvar/run_mri3d4dvar.csh_pbs
A var/mri4dvar/task_for_point.c
A var/mri4dvar/wraper_mri3d4dvar.csh

TESTS CONDUCTED: WRFDA regression test passed. Cycling MRI-4DVar tests were performed over an Arctic domain.

RELEASE NOTE: New capability for Multi-Resolution Incremental 4DVar (MRI-4DVar).
Liu, Z., J. Ban, J.-S, Hong, and Y.-H. Kuo, 2020: Multi-resolution incremental 4D-Var for WRF: Implementation and application at convective scale, Q. J. R. Meteorol. Soc., 146, 3661-3674.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants