Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GitHub Issue NOAA-EMC/GSI#341. Update the use of MW radiances for GFS v16.3 implementation #350

Merged
merged 1 commit into from
May 3, 2022

Conversation

emilyhcliu
Copy link
Contributor

@emilyhcliu emilyhcliu commented Mar 31, 2022

The following changes will be added to GSI Master for v16.3 Implementation:

Source code update:

  • Switching from assimilating antenna temperature (SDR) to brightness temperature (TDR) for AMSU-A, ATMS, and MHS
  • The use precipitation-affected radiances for ATMS and AMSU-A (extened all-sky framework)
  • The use of GMI radiances under all-sky conditions

Fix files update:

  • Add global_anavinfo_allhydro_l127.txt --- add precipitation components
  • Update cloudy_radiance_info.txt --- add GMI obs error assignment and QC parameters
  • Update global_satinfo.txt --- modify channel use flag and error bounds for GMI
  • Update global_scaninfo.txt --- add GMI

Script update:

  • exglobal_atmos_analysis.sh

Working Branch: feature/v16x_allsky

  • The code update in the branch will not change the GSI result from master with the default namelist and default resource files (All regression tests passed)
  • Set the logical ta2tb to true in the namelist to activate the use of brightness temperature for ATMS, AMSU-A and MHS
  • Use global_anavinfo_allhydro.l127.txt to enable the assimilation of precipitation-affected radiances from ATMS and AMSUA
  • Modify the channel use flags for GMI in global_satinfo.txt to turn on the assimilation of GMI (see below for channel selection)

List of changes for Ta2Tb work:

  • antcorr_application.f90
  • crtm_interface.f90
  • read_atms.f90
  • read_bufrtovs.f90
  • obsmod.F90
  • gsimod.F90

Set ta2tb = .true. will trigger the use of Tb for AMSU-A, ATMS, and MHS
Default ta2tb is set to false. This ensures that the code update will not change GSI result from the master

List of changes for assimilating precipitation-affected ATMS and AMSU-A:

  • cloud_efr_mod.f90
  • cplr_gfs_ensmod.f90
  • crtm_interface.f90
  • general_read_gfsatm.f90
  • netcdfgfs_io.f90
  • prt_guess.f90
  • qcmod.f90
  • read_atms.f90
  • read_bufrtovs.f90
  • write_incr.f90
  • setuprad.f90

These code changes will not affect GSI result from master unless the default global_anavinfo.l127.txt is replaced with global_anavinfo_alhydro.l127.txt

The current GSI master (and operational GDAS) is using CRTM2.3.0 for radiance assimilation
The use of precipitation-affected MW data should combined with the use of CRTM 2.4.0 for the following two reasons:

  1. The surface Jacobian under scattering condition from CRTM2.3.0 was not implemented correctly and CRTM 2.4.0 fixed the issue. (Surface Jacobian is used in QC for radiance data)
  2. CRTM2.4.0 has the cloud optical table generated based on GFDL physics (consistent with FV3 model physics)

Note: If we decide not to include precipitation-affected MW radiance in our next implementation, then we should not use CRTM2.4.0 (should stay with CRTM 2.3.0) due to the following two reasons:

  1. CRTM 2.4.0 requires cloud fraction profile in the input for cloudy simulation.
  2. GFDL cloud fraction calculation requires all hydrometeors which is not available in our operational run (only cloud water and cloud water are available)

List of changes for the assimilation of GMI under all-sky conditions:

  • read_gmi.f90
  • crtm_interface.f90
  • setuprad.f90

These code update will not change GSI result from the master since GMI channel use flags are set to -1 (monitoring)
To activate GMI for assimilation, modifiy the use flags as the following:

 gmi_gpm                 1  -1    2.700   25.000   30.000   40.000    0.000      1      1     -1
 gmi_gpm                 2  -1    3.700   40.000   30.000   40.000    0.000      1      1     -1
 gmi_gpm                 3  -1    3.500   40.000   15.000   40.000    0.000      1      1     -1
 gmi_gpm                 4  -1    4.500   55.000   30.000   40.000    0.000      1      1     -1
 gmi_gpm                 5   1    4.000   35.000   15.000   40.000    0.000      1      1     -1
 gmi_gpm                 6   1    3.800   25.000   30.000   40.000    0.000      1      1     -1
 gmi_gpm                 7   1  300.000  500.000    5.000   40.000    0.000      1      1     -1
 gmi_gpm                 8  -1    5.000   50.000   15.000   40.000    0.000      1      1     -1
 gmi_gpm                 9  -1   11.500   50.000   20.000   40.000    0.000      1      1     -1
 gmi_gpm                10   1    5.000   50.000   20.000   40.000    0.000      1      1     -1
 gmi_gpm                11  -1    5.000   50.000   20.000   40.000    0.000      1      1     -1
 gmi_gpm                12   1    2.500   30.000   10.000   40.000    0.000      1      1     -1
 gmi_gpm                13   1    3.000   40.000   10.000   40.000    0.000      1      1     -1

List of changes for exglobal_atmos_analysis.sh:

  1. add link to AMSU-A MetOp-A version 2 coefficient file
  2. add logic (ta2tb) to turn on the use of brightness temperature for ATMS, AMSU-A and MHS for assimilaton
  3. add logic (cao_check) to turn on CAO (cold air outbreak check) for quality control
  4. set thinning box to 145 km for GMI (original is 100 km)

@emilyhcliu
Copy link
Contributor Author

emilyhcliu commented Mar 31, 2022

@MichaelLueken-NOAA I suggest the following people for our internal review of this PR:
@ScottSieron-NOAA
@ADCollard
@HaixiaLiu-NOAA
@RussTreadon-NOAA

Copy link
Contributor

@ADCollard ADCollard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few minor questions and changes.

I flagged a bunch of changes from Scott in TaTb which I think can be removed.

@@ -81,6 +81,10 @@ export imp_physics=${imp_physics:-99}
lupp=${lupp:-".true."}
cnvw_option=${cnvw_option:-".false."}

# Observation usage options
cao_check=${cao_check:-".false."}
ta2tb=${ta2tb:-".false."}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know the intention is to not make any chnages to the results with this pull request, but the CAO check is something that really makes sense and I think it would be better if this defaulted to true.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ADCollard First, thanks for reviewing this PR.
Yes, I will make cao_check to true as default when we do the high-resolution parallel experiment. If the results look good, I will make cao_check=true as default in the GSI master and also in the package we hand over to NCO.

@@ -1015,7 +1057,8 @@ subroutine parallel_read_gfsnc_state_(en_full,m_cvars2d,m_cvars3d,nlon,nlat,nsig
enddo
deallocate(rwork3d1)

if (k3u==0.or.k3v==0.or.k3t==0.or.k3q==0.or.k3cw==0.or.k3oz==0) &
! if (k3u==0.or.k3v==0.or.k3t==0.or.k3q==0.or.k3cw==0.or.k3oz==0) & !orig
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If these changes are permanent, remove comments.

What about k3ql, k3qi, k3qr, k3qs, k3qg?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current one actually works for operational configuration and for the configuration with precipitation turned on without checking k3ql, k3qi, k3qr, k3qs, and k3qg. So, this is not a strict check.

I will see if I can find a way to make it a strict check.

! enddo
! errf(1:ich544)=zero
! varinv(1:ich544)=zero
! do i=1,ich544
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove commented code

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ADCollard These lines are for future testing. So, they are commented out for now.

src/gsi/read_bufrtovs.f90 Outdated Show resolved Hide resolved
src/gsi/read_bufrtovs.f90 Outdated Show resolved Hide resolved
src/gsi/read_gmi.f90 Outdated Show resolved Hide resolved
src/gsi/read_gmi.f90 Show resolved Hide resolved
src/gsi/read_gmi.f90 Show resolved Hide resolved
src/gsi/read_gmi.f90 Show resolved Hide resolved
@@ -1047,7 +1052,7 @@ subroutine setuprad(obsLL,odiagLL,lunin,mype,aivals,stats,nchanl,nreal,nobs,&
endif
endif
! Screening for cold-air outbreak area (only applied to MW for now)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean the cao check is only performed when precip is turned on?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ADCollard Yes, the detection of CAO requires to use all of the five hydrometeors.

@emilyhcliu
Copy link
Contributor Author

@MichaelLueken-NOAA
I have the following four files change in the fix directory:
Fix files update:

  • Add global_anavinfo_allhydro_l127.txt --- add precipitation components
  • Update cloudy_radiance_info.txt --- add GMI obs error assignment and QC parameters
  • Update global_satinfo.txt --- modify channel use flag and error bounds for GMI
  • Update global_scaninfo.txt --- add GMI

They are in feature/v16x_allsky under fix directory.
What should I do to make these four fix files into rev2? Thanks.

@MichaelLueken
Copy link
Contributor

They are in feature/v16x_allsky under fix directory. What should I do to make these four fix files into rev2? Thanks.

@emilyhcliu You don't need to do anything. Once the work in this PR has been merged to the master, I will need to manually merge the changes in feature/v16x_allsky into rev2. I need to do this whether the updated fix hash is in the PR or not, so all I need is the fix branch to merge to rev2 (which you have provided) and that is it.

Copy link
Contributor

@MichaelLueken MichaelLueken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @emilyhcliu, I have completed my initial review of your changes. The first issue is that there are four commit messages. To reduce this to a single commit message, please do the following in your feature/v16x_allsky branch in your fork:

  1. git reset --soft HEAD~1
  2. git reset --soft HEAD~1
  3. git reset --soft HEAD~1
  4. git commit --amend

In the commit message window, please add:

GitHub Issue NOAA-EMC/GSI#341

to the commit message:

GitHub Issue NOAA_EMC/GSI#341. This update includes the following:
(1) Add the capability of assimilating precipitation-affected radiances from AMSU-A and ATMS
(2) Enable the all-sky radiance assimilation for GMI
(3) Add the switch to use brightness temperature (SDR) instead of antenna temperature (TDR) for AMSU-A, ATMS and MHS

Once the commit messages have been corrected, please go through and address the concerns I noted (mostly, removal of commented out lines of code that if they won't be used in the future). Once the changes have been made, please use:

  1. git commit --amend
  2. Save and close the commit message without changing the message.
  3. Push back to the repository using
    git push origin feature/v16x_allsky --force

Please let me know if you have any questions, comments, or concerns.

@@ -1015,7 +1057,8 @@ subroutine parallel_read_gfsnc_state_(en_full,m_cvars2d,m_cvars3d,nlon,nlat,nsig
enddo
deallocate(rwork3d1)

if (k3u==0.or.k3v==0.or.k3t==0.or.k3q==0.or.k3cw==0.or.k3oz==0) &
! if (k3u==0.or.k3v==0.or.k3t==0.or.k3q==0.or.k3cw==0.or.k3oz==0) & !orig
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If commented out line 1060 won't be used in the future, please remove it.

src/gsi/netcdfgfs_io.f90 Outdated Show resolved Hide resolved
modulefiles/modulefile.ProdGSI.hera Outdated Show resolved Hide resolved
src/gsi/antcorr_application.f90 Outdated Show resolved Hide resolved
src/gsi/antcorr_application.f90 Outdated Show resolved Hide resolved
src/gsi/write_incr.f90 Show resolved Hide resolved
src/gsi/write_incr.f90 Show resolved Hide resolved
src/gsi/write_incr.f90 Outdated Show resolved Hide resolved
src/gsi/write_incr.f90 Outdated Show resolved Hide resolved
src/gsi/write_incr.f90 Outdated Show resolved Hide resolved
Copy link
Contributor

@MichaelLueken MichaelLueken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @emilyhcliu, I have completed my debug review of your changes in this PR. There are several unused variables that were flagged while compiling and I have noted them in this review. Please remove or comment out the unused variables (I say comment out because several of the unused variables are being flagged as such due to the code segments that they are used being commented out). Having said that, the debug executable ran through to completion, passing the debug tests. If you have any questions or concerns, please let me know.

real(r_kind),pointer,dimension(:,:) :: g_ps
real(r_kind),pointer,dimension(:,:,:) :: g_vor,g_div,&
g_q,g_oz,g_tv
real(r_kind),pointer,dimension(:,:,:) :: g_ql,g_qi,g_qr,g_qs,g_qg,g_cf
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While compiling in debug mode, g_cf, on line 2644, was noted as being unused. All code segments that use this variable are commented out. If these sections won't be used in the future, please remove g_cf. Otherwise, please comment out g_cf until the other commented out sections are uncommented.

src/gsi/general_read_gfsatm.f90 Outdated Show resolved Hide resolved
src/gsi/netcdfgfs_io.f90 Outdated Show resolved Hide resolved
@@ -91,6 +91,9 @@ subroutine write_fv3_inc_ (grd,sp_a,filename,mype_out,gfs_bundle,ibin)
use obsmod, only: ianldate
use state_vectors, only: allocate_state, deallocate_state

use state_vectors, only: svars3d, ns3d
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On line 94, ns3d was noted as unused. Please remove unused variable.

src/gsi/write_incr.f90 Outdated Show resolved Hide resolved
end if
! test in the future
! if (si_mean >= 20.0_r_kind) then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since si_mean is currently commented out, si_mean on line 2810 is now being flagged as unused. If possible, please move si_mean to the end of the passed variable list and comment it out until this section is brought back. Please note that this will also require some additional work in setuprad.f90.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MichaelLueken-NOAA I made si_mean an optional input.

@emilyhcliu emilyhcliu force-pushed the feature/v16x_allsky branch from 1300c4d to 33b8a04 Compare April 20, 2022 15:56
@emilyhcliu
Copy link
Contributor Author

@MichaelLueken-NOAA I fixed the commit comment as you instructed. I also addressed your comments and changed according. I had committed and push the changes back to the branch. Please let me know if you have any questions/comments.

@MichaelLueken
Copy link
Contributor

@emilyhcliu Your PR has too many commit messages. You will need to reduce the six commits down to one using:

  1. git reset --soft HEAD~1
  2. git commit --amend
  3. Please save and close the commit message window without changing the commit message
  4. git push origin feature/v16x_allsky --force

If you have any questions, please let me know.

@emilyhcliu emilyhcliu force-pushed the feature/v16x_allsky branch from 33b8a04 to 60a3c49 Compare April 20, 2022 17:57
@emilyhcliu
Copy link
Contributor Author

Done!

@MichaelLueken
Copy link
Contributor

Hi @emilyhcliu. Thank you very much for addressing my concerns. There are just a few things I would like to note quickly:

  1. Unused variables:
    • Following an updated clone of your fork, I see that there are still two unused variables:
      • src/gsi/general_read_gfsatm.f90 - g_cf on line 2642. Please comment out unused variable.
      • src/gsi/qcmod.f90 - si_mean on lines 2815 and 3623. Please comment out unused variables (changing passed variables to optional doesn't remove the fact that they aren't used in the code).
  2. Commented out code:
    • If the currently commented out code that hasn't been removed will be used in the future, then it can stay. Please note that it is not desirable to keep commented out code in routines, so if there is no chance that the commented out code will be used again, please remove it. However, it looks like there are commented out lines that are purely for debug purposes:
      • src/gsi/netcdfgfs_io.f90 - please remove debug writes on lines 220 and 222.
      • src/gsi/read_bufrtovs.f90 - please remove debug writes on lines 537, 538, 540, and 542.
  3. Indentation:
    • src/gsi/write_incr.f90 - Please ensure two space indentation for all if statements between lines 254 - 275.
  4. Real precision:
    • src/gsi/read_gmi.f90 - The variable, var_check1, is declared as r_double. On line 420, the value 99999999999 isn't given a precision. Please ensure that the proper precision is given to this value (whether it is _r_kind or _r_double).
  5. Logical Boolean symbols:
    • src/gsi/read_bufrtovs.f90 - Please replace .ne. with /= on line 763.

After changes are made, please commit using:

git commit --amend

and push using:

git push origin feature/v16x_allsky --force

If you have any questions, please let me know.

Copy link
Contributor

@MichaelLueken MichaelLueken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @emilyhcliu. Two more minor points in src/gsi/crtm_interface.f90. There are several commented out lines pertaining to CRTM v2.3.* (specifically, lines 2029 and 2083). Both of these lines appear to be identical. Additionally, they are identical to line 2087, which is not commented. Should these two commend out lines be removed?

cloud_efr(k,ii)=max(5.001_r_kind, cloud_efr(k,ii))
endif
end do
!crtm2.3.x if (.not. regional .and. icfs==0 ) atmosphere(1)%cloud_fraction(k) = cf(kk2)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should line 2083 be removed, since an uncommented out version of the line exists on line 2087?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MichaelLueken-NOAA You were right. I removed them.

@@ -2000,7 +2026,7 @@ subroutine call_crtm(obstype,obstime,data_s,nchanl,nreal,ich, &
if (ii==2 .and. atmosphere(1)%temperature(k)<t0c) &
cloud_cont(k,2)=max(1.001_r_kind*1.0E-6_r_kind, cloud_cont(k,2))
end do
!crtm2.3.x if (.not. regional .and. icfs==0 ) atmosphere(1)%cloud_fraction(k) = cf(kk2)
!crtm2.3.x if (.not. regional .and. icfs==0 ) atmosphere(1)%cloud_fraction(k) = cf(kk2)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should line 2029 be removed, since an uncommented out version of the line exists on line 2087?

@emilyhcliu emilyhcliu force-pushed the feature/v16x_allsky branch from 60a3c49 to 73444cc Compare April 21, 2022 16:07
@emilyhcliu
Copy link
Contributor Author

@MichaelLueken-NOAA I updated the codes as you suggested and pushed the back.
Thanks for reviewing. Please let me know if you have any comments.

Copy link
Contributor

@MichaelLueken MichaelLueken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @emilyhcliu. Thank you very much for making the requested changes. There is one last minor change. On line 759 of src/gsi/read_bufrtovs.f90, you replaced the first .ne. with /=, as requested. However, there is a second .ne. (sacv .ne. 1) that needs to also be replaced with /= (i.e., sacv /= 1). Once this change is made, I should be able to submit this work to the review committee once the current work has been merged to the authoritative repository.

After making the change, please use:

  1. git add src/gsi/read_bufrtovs.f90
  2. git commit --amend
  3. Save and close the commit message window without altering the current commit message
  4. git push origin feature/v16x_allsky --force

If you have any questions, please let me know.

else ! EARS / DB
call ufbrep(lnbufr,data1b8,1,nchanl,iret,'TMBRST')
!if ( amsua .or. amsub .or. mhs .AND. sacv .ne. spc_coeff_versions)then
if ( amsua .or. amsub .or. mhs .AND. spc_coeff_versions /= 1 .AND. sacv .ne. 1)then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for replacing the first .ne. with /=. Please replace the second .ne. with /= on line 759.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MichaelLueken-NOAA Thanks you for your thorough review of my PR. Your request has been addressed.

This update includes the following:
(1) Add the capability of assimilating precipitation-affected radiances from AMSU-A and ATMS
(2) Enable the all-sky radiance assimilation for GMI
(3) Add the switch to use brightness temperature (SDR) instead of antenna temperature (TDR) for AMSU-A, ATMS and MHS
@emilyhcliu emilyhcliu force-pushed the feature/v16x_allsky branch from 73444cc to 3c361da Compare April 25, 2022 17:29
@MichaelLueken MichaelLueken changed the title Update the use of MW radiances for GFS v16.3 implementation GitHub Issue NOAA-EMC/GSI#341. Update the use of MW radiances for GFS v16.3 implementation May 3, 2022
@MichaelLueken
Copy link
Contributor

The due date for the review committee has passed without feedback. I will now give final approval to these changes and merge them to the authoritative repository.

@MichaelLueken MichaelLueken merged commit c13361c into NOAA-EMC:develop May 3, 2022
@MichaelLueken
Copy link
Contributor

The fix files in fix/feature/v16x_allsky were merged to fix/rev2 and pushed to the authoritative repository at 87fe77d.

EdwardSafford-NOAA pushed a commit to EdwardSafford-NOAA/GSI that referenced this pull request May 27, 2022
GitHub Issue NOAA-EMC#341. Update the use of MW radiances for GFS v16.3 implementation
AndrewEichmann-NOAA pushed a commit to AndrewEichmann-NOAA/GSI that referenced this pull request Jun 6, 2022
GitHub Issue NOAA-EMC#341. Update the use of MW radiances for GFS v16.3 implementation
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add changes for all-sky radiance assimilation
4 participants