Adding I/O for direct analysis of near-surface wind gust for RRFS-based 3DRTMA

[GangZhao-NOAA]

Description

To improve the analysis of the near-surface wind gust in 3DRTMA, the observations of near-surface wind gust would be analyzed directly in GSI (3DVar and Hybrid 3DEnVar), instead of being a derived product from the near-surface wind analysis.

Since the core subroutines for direct variational assimilation of wind gust (e.g., setupgust.f90, intgust.f90, stpgust.f90, etc.) had already been implemented in GSI for 2DRTMA, so in the work the development in GSI mainly focus on adding I/O of 2-D wind gust firstguess and analysis fields for RRFS-based 3DRTMA, and some minor modifications in observation and background error for wind gust, options to control the analysis of wind gust, etc.

This PR is to address the issue #726 : Adding I/O for direct analysis of near-surface wind gust for RRFS-based 3DRTMA
Fixes #726
Type of change

Please delete options that are not relevant.

• New feature (non-breaking change which adds functionality)

How Has This Been Tested?

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• New and existing tests pass with my changes
  tested with a real case - 2024-02-20_12:00Z,

1. dry-run (using my updated GSI code with wind gust analysis, but actually no wind gust obs is analyzed, so-called dryrun) is compared to control-run (original GSI code running without wind gust obs): the results are identical. This indicates that if without analyzing wind-gust obs, then the updated code generates the analysis identical to the analysis of original/control code. Or say, the added code does not have influence on the other part of code.
2. real case run with updated GSI code to analyze the obs of wind gust:
   The following figure shows the used observations of near-surface wind gust:
   
   the following figure shows the analysis increments:
   

• Any dependent changes have been merged and published
• Regression tests on WCOSS2 (Cactus) and Hera (Rocky-8) : my updated GSI commit #f91f247d) vs control/original GSI code (commit #6d9ebbb7)
  Here is the reports of the regression tests on WCOSS2 (Cactus):

[gang.zhao@clogin02:build] (feature/windgust_in_3dvar_for_3drtma)$ ctest -j 7
Test project /lfs/h2/emc/da/save/gang.zhao/WorkDir/ProdGSI_Dev/gsi_dev/build
    Start 1: global_4denvar
    Start 2: rtma
    Start 3: rrfs_3denvar_glbens
    Start 4: netcdf_fv3_regional
    Start 5: hafs_4denvar_glbens
    Start 6: hafs_3denvar_hybens
    Start 7: global_enkf
1/7 Test #4: netcdf_fv3_regional ..............   Passed  483.15 sec
2/7 Test #3: rrfs_3denvar_glbens ..............   Passed  486.74 sec
3/7 Test #7: global_enkf ......................   Passed  850.98 sec
4/7 Test #2: rtma .............................   Passed  970.28 sec
5/7 Test #6: hafs_3denvar_hybens ..............   Passed  1152.82 sec
6/7 Test #5: hafs_4denvar_glbens ..............   Passed  1213.93 sec
7/7 Test #1: global_4denvar ...................   Passed  1683.16 sec

100% tests passed, 0 tests failed out of 7

Total Test time (real) = 1683.19 sec

Here is the reports of the regression tests on Hera (Rocky8):

(base) [Gang.Zhao@hfe11:build] (feature/windgust_in_3dvar_for_3drtma)$ ctest -j 7
Test project /scratch1/NCEPDEV/da/Gang.Zhao/ProdGSI_dev/gsi_dev/build
    Start 1: global_4denvar
    Start 2: rtma
    Start 3: rrfs_3denvar_glbens
    Start 4: netcdf_fv3_regional
    Start 5: hafs_4denvar_glbens
    Start 6: hafs_3denvar_hybens
    Start 7: global_enkf
1/7 Test #4: netcdf_fv3_regional ..............   Passed  491.53 sec
2/7 Test #3: rrfs_3denvar_glbens ..............***Failed  495.27 sec
3/7 Test #2: rtma .............................   Passed  982.45 sec
4/7 Test #6: hafs_3denvar_hybens ..............   Passed  1168.99 sec
5/7 Test #7: global_enkf ......................   Passed  1239.77 sec
6/7 Test #5: hafs_4denvar_glbens ..............***Failed  1347.87 sec
7/7 Test #1: global_4denvar ...................   Passed  1974.45 sec

71% tests passed, 2 tests failed out of 7

Total Test time (real) = 1974.91 sec

The following tests FAILED:
          3 - rrfs_3denvar_glbens (Failed)
          5 - hafs_4denvar_glbens (Failed)
Errors while running CTest
Output from these tests are in: /scratch1/NCEPDEV/da/Gang.Zhao/ProdGSI_dev/gsi_dev/build/Testing/Temporary/LastTest.log
Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely.
(base) [Gang.Zhao@hfe11:build] (feature/windgust_in_3dvar_for_3drtma)$ ctest -R rrfs_3denvar_glbens
Test project /scratch1/NCEPDEV/da/Gang.Zhao/ProdGSI_dev/gsi_dev/build
    Start 3: rrfs_3denvar_glbens
1/1 Test #3: rrfs_3denvar_glbens ..............   Passed  430.52 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) = 430.55 sec
(base) [Gang.Zhao@hfe11:build] (feature/windgust_in_3dvar_for_3drtma)$ ctest -R hafs_4denvar_glbens
Test project /scratch1/NCEPDEV/da/Gang.Zhao/ProdGSI_dev/gsi_dev/build
    Start 5: hafs_4denvar_glbens
1/1 Test #5: hafs_4denvar_glbens ..............   Passed  1225.37 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) = 1225.39 sec

Note:
_When I was running the regression tests, GSI code was just updated to the latest commit #b53740a7. Considering the frequent update in EMC GSI code recently and saving the time, after this PR has been reviewed and approved by peer-reviewers, I will update the code to latest EMC GSI commit, then re-run the regression tests for final approval.

[AnnetteGibbs-NOAA]

Looks good Gang.

[TingLei-NOAA]

@GangZhao-NOAA Updated version: where is the updated surface wind stored ? if they are in fv3_sfcdata, should this change cause regional regression tests fail if the regression tests compare those files?

[GangZhao-NOAA]

Hi Ting, Yes, 2-D wind gust field (and 2-D Significant Wave Height field) is stored in sfcdata.nc4 file. Because current regional regression tests do not have the configurations for 3drtma, any wind gust and wave height related code will not be invoked in any regression tests. So current regression tests are good. Not sure if a 3drtma regression test task would be added into the regression test list of GSI. Thank you!

…

-Gang

On Tue, Apr 2, 2024 at 9:57 AM TingLei-NOAA ***@***.***> wrote: @GangZhao-NOAA <https://github.com/GangZhao-NOAA> so the updated surface wind is stored in the files as :fv3_sfcdata, right? Seems the current regional tests don't compare them from the control runs and updt runs and, hence, changed results in it doesn't cause the regional tests report the differences. Would you consider to add this comparison into the regional regression tests? — Reply to this email directly, view it on GitHub <#730 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMWMXUYVXHFPQ4CZVW63W2TY3K2NRAVCNFSM6AAAAABFKCI6ISVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZSGEYTOOJTGY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[GangZhao-NOAA]

@TingLei-NOAA
Hi Ting,

Thank you so much for diving in the "cplr_get_fv3_regional_ensperts.f90" and confirming that the ensemble perturbations arrays (used to provide ensemble error covariance) are initialized as zeros (in subroutine get_fv3_regional_ensperts_run)
en_perts(n,1,m)%valuesr4 = zero
Then when running GSI in hybrid envar mode, it is safe for these analysis variables which are NOT available in ensemble background fields yet, such as for significant wave height, 10-m wind gust. It is sure that the ensemble variances for these variables are pure zero in hybrid run.

Thanks again!

-Gang

[TingLei-NOAA]

@GangZhao-NOAA Thanks for doing this development with a clean "GSI " style.
Also thanks for your explanation/clarifications. Now, all look good to me.

[GangZhao-NOAA]

• My GSI code was updated:

1. merging the latest version (#db477e36) of "develop" branch into my GSI code;
2. Based on feedback from peer code reviewers, some minor modifications (more comments to explain the usage of corp_gust/hwllp_gust/oerr_gust) were added in the code;

• Then running with same real case data, the results (cost/grad in fort.220, obs-fitting stats in fort.201~204, fort.218, fort.228) from newly-updated code #e1512de3 and previous commit #f91f247d are identical;

• Running regression tests on Hera and WCOSS2 (updated GSI #e1512de3 vs control code of GSI #db477e36) :
  On hera: all 7 tasks passed.

(base) [Gang.Zhao@hfe02:build] (feature/windgust_in_3dvar_for_3drtma *)$ ctest -j 7
Test project /scratch1/NCEPDEV/da/Gang.Zhao/ProdGSI_dev/gsi_dev/build
    Start 1: global_4denvar
    Start 2: rtma
    Start 3: rrfs_3denvar_glbens
    Start 4: netcdf_fv3_regional
    Start 5: hafs_4denvar_glbens
    Start 6: hafs_3denvar_hybens
    Start 7: global_enkf
1/7 Test #4: netcdf_fv3_regional ..............   Passed  1029.81 sec
2/7 Test #3: rrfs_3denvar_glbens ..............   Passed  1092.93 sec
3/7 Test #2: rtma .............................   Passed  1574.38 sec
4/7 Test #6: hafs_3denvar_hybens ..............   Passed  1587.40 sec
5/7 Test #7: global_enkf ......................   Passed  1777.02 sec
6/7 Test #5: hafs_4denvar_glbens ..............   Passed  1884.95 sec
7/7 Test #1: global_4denvar ...................   Passed  2454.08 sec

100% tests passed, 0 tests failed out of 7

Total Test time (real) = 2454.53 sec

on WCOSS2 (Dogwood): all 7 tasks passed, too.

[gang.zhao@dlogin03:build] (feature/windgust_in_3dvar_for_3drtma)$ ctest -j 7
Test project /lfs/h2/emc/da/save/gang.zhao/WorkDir/ProdGSI_dev/gsi_dev/build
    Start 1: global_4denvar
    Start 2: rtma
    Start 3: rrfs_3denvar_glbens
    Start 4: netcdf_fv3_regional
    Start 5: hafs_4denvar_glbens
    Start 6: hafs_3denvar_hybens
    Start 7: global_enkf
1/7 Test #4: netcdf_fv3_regional ..............***Failed  603.87 sec
2/7 Test #3: rrfs_3denvar_glbens ..............   Passed  608.93 sec
3/7 Test #7: global_enkf ......................   Passed  1032.24 sec
4/7 Test #2: rtma .............................   Passed  1389.34 sec
5/7 Test #6: hafs_3denvar_hybens ..............   Passed  1455.37 sec
6/7 Test #5: hafs_4denvar_glbens ..............   Passed  1455.64 sec
7/7 Test #1: global_4denvar ...................   Passed  1922.99 sec

86% tests passed, 1 tests failed out of 7

Total Test time (real) = 1923.00 sec

The following tests FAILED:
          4 - netcdf_fv3_regional (Failed)
Errors while running CTest
[gang.zhao@dlogin09:build] (feature/windgust_in_3dvar_for_3drtma)$ ctest -R netcdf_fv3_regional
Test project /lfs/h2/emc/da/save/gang.zhao/WorkDir/ProdGSI_dev/gsi_dev/build
    Start 4: netcdf_fv3_regional
1/1 Test #4: netcdf_fv3_regional ..............   Passed  1083.72 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) = 1083.92 sec