Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updating AQM to CMAQv5.4 #101

Draft
wants to merge 113 commits into
base: develop
Choose a base branch
from

Conversation

drnimbusrain
Copy link

@drnimbusrain drnimbusrain commented Jul 17, 2024

PR Checklist

Description

This PR provides all necessary updates to AQM develop branch for ARL's work to update CMAQv5.4 submodule (from current CMAQv5.2.1) and all necessary AQM infrastructure to accommodate the CMAQv5.4 changes compared. Since our CMAQv5.4 changes were built upon our fork of the EMC develop branch, then it also contains some initial developments of our in-canopy photolysis infrastructure and code developments. The canopy option is turned off by default, and at this time, we recommend that the canopy option remains turned OFF (in aqm.rc) until further refined developments of the canopy codes in AQM (and later in UFS-SRW-App, UWM, FV3/CCPP, etc. via separate corresponding PRs to their authoritative repos to enable full functionality).

Issue(s) addressed

#100

Dependencies

Updates to UFS-SRW-App at non-default branches (e.g., production/AQM.v7) are needed to accommodate the CMAQv5.4 updates in AQM (see new or updated data section above). The question remains as to where and how to make these PRs. Our changes for CMAQv5.4 were built upon the previous [aqm_dev] branch of UFS-SRW-App, and do not correspond to the production/AQM.v7 branch. Please advise and we can update and make the correct PR to the correct UFS-SRW-App branch to accommodate testing the CMAQv5.4 updates to AQM.

@lwcugb @bbakernoaa @ytangnoaa

Patrick.C.Campbell@noaa.gov and others added 30 commits February 10, 2022 21:27
Fixed Makefile.in typo.
@JianpingHuang-NOAA
Copy link
Collaborator

@bbakernoaa, https://github.com/noaa-oar-arl/AQM/tree/feature/cmaq54 is 3-month old. Where is the updated package?

@bbakernoaa
Copy link
Contributor

this is the latest that we have. We don't have anything newer at this juncture

@JianpingHuang-NOAA
Copy link
Collaborator

I heard that ARL has a fix to the soil type mix-up. Is that not for AQM?

@bbakernoaa
Copy link
Contributor

This is something that needs to be changed not just on the AQM side but also the FV3ATM side as well. We just finished some preliminary testing and such today. We will need to do PRs back to the UFS weather model for both changes (ie AQM and FV3ATM repositories)

@bbakernoaa
Copy link
Contributor

note that when we update to the GFS v17 physics we should also pass the fractional land use over and use this instead of the dominant value. Currently we are basically hard coding the fractional land use to be 1 for each dominate land use type in the grid cell and all other values are 0.

@drnimbusrain
Copy link
Author

drnimbusrain commented Jan 3, 2025 via email

@drnimbusrain
Copy link
Author

@BrianCurtis-NOAA How's it going moving this CMAQ54 PR forward, and is there a status update on a regression test? Do you need help from us to identify inputs, run directory, etc. to help get this setup and moved forward?
@JianpingHuang-NOAA

@BrianCurtis-NOAA
Copy link
Collaborator

@drnimbusrain We will need C96 support for AQM to get a proper regression test (C192 is acceptable if C96 complicates things)

I don't need a workflow to get this merged, but a run directory with a working UFSWM will suffice so I can update the RT system to use it. As long as we have that, we can get this part merged in. Is this something that you might be able to put together?

@bbakernoaa
Copy link
Contributor

@BrianCurtis-NOAA could we not leverage: https://github.com/ufs-community/ufs-weather-model/blob/develop/tests/tests/regional_atmaq

We would just need to update the additional CMAQ related inputs and such for the updates no?

@BrianCurtis-NOAA
Copy link
Collaborator

@BrianCurtis-NOAA could we not leverage: https://github.com/ufs-community/ufs-weather-model/blob/develop/tests/tests/regional_atmaq

We would just need to update the additional CMAQ related inputs and such for the updates no?

Yes, that can be used as a template.

@BrianCurtis-NOAA
Copy link
Collaborator

I'm happy to put the work in taking a run directory and turning it into a proper RT if that will save you from too much work. The work on AQM-Workflow will eventually know how to generate such a run directory, but that will be a while.

@drnimbusrain
Copy link
Author

drnimbusrain commented Jan 18, 2025 via email

@bbakernoaa
Copy link
Contributor

@drnimbusrain I think all we need to do is provide a run directory. The truth is most of the input files as far as emissions and such can be used for either 5.2 or 5.4. We just need to make sure we have an example to show the differences in the ancillary CMAQ files needed. Other than that the input files may need updated on the ATM side to accommodate the updates of the newer ATM hash.

I can take a quick stab at it on Tuesday morning. At least to get something running. Point me to a run directory that you have for whatever resolution. Then @BrianCurtis-NOAA you could then integrate the updated files that RT needs into run RT workflow.

@drnimbusrain
Copy link
Author

drnimbusrain commented Jan 18, 2025 via email

@bbakernoaa
Copy link
Contributor

@BrianCurtis-NOAA @drnimbusrain please take a look here: /scratch1/NCEPDEV/stmp2/Barry.Baker/FV3_RT/rt_994030/regional_atmaq_debug_intel

I was able to get the RT for the regional_atmaq_debug going on hera with the updated feature/cmaq54 branch of the AQM. Note that this was off the latest develop branch of the UFSWM

Files Changed

AE_cb6r3_ae6_aq.nml -> AE_cb6r5_ae7_aq.nml
CSQY_DATA_cb6r3_ae6_aq -> CSQY_DATA_cb6r5_ae7_aq
GC_cb6r3_ae6_aq.nml -> GC_cb6r5_ae7_aq.nml
NR_cb6r3_ae6_aq.nml -> NR_cb6r5_ae7_aq.nml
PHOT_OPTICS.dat
Species_Table_TR_0.nml
field_table
diag_table
aqm.rc 
CMAQ_Control_DESID.nml
CMAQ_Control_DESID_cb6r5_ae7_aq.nml
CMAQ_Control_Misc.nml

@BrianCurtis-NOAA
Copy link
Collaborator

are you using CCPP_SUITES=FV3_GFS_v16 ?

@bbakernoaa
Copy link
Contributor

the RT was using FV3_GFS_v15p2

I'll try to quickly update it to FV3_GFS_v16

@bbakernoaa
Copy link
Contributor

@BrianCurtis-NOAA I updated it to use the FV3_GFS_v16 physics

@JianpingHuang-NOAA
Copy link
Collaborator

@BrianCurtis-NOAA , what is the status of the PR merging ?

@BrianCurtis-NOAA
Copy link
Collaborator

@JianpingHuang-NOAA It will be a little bit. I'll get to looking at Barry's run dir soon.

@JianpingHuang-NOAA
Copy link
Collaborator

@BrianCurtis-NOAA Hi Brian, any update on this?

@JianpingHuang-NOAA
Copy link
Collaborator

@BrianCurtis-NOAA Hi Brian, we need to get this PR merged as soon as possible. Thanks !

@BrianCurtis-NOAA
Copy link
Collaborator

@bbakernoaa I don't see the run directory you have listed above any more. Did you move it or have it somewhere else?

@BrianCurtis-NOAA
Copy link
Collaborator

@bbakernoaa I don't see the run directory you have listed above any more. Did you move it or have it somewhere else?

typo, nevermind

@BrianCurtis-NOAA
Copy link
Collaborator

There have been some issues running the model with a debug build, with the most recent issue:

WARNING: map_precursor: NO particulate mercury in model.       
forrtl: severe (408): fort: (3): Subscript #1 of the array WFC has value 0 which is less than the lower bound of 1

Image              PC                Routine            Line        Source             
fv3.exe            000000000BAA303F  Unknown               Unknown  Unknown
fv3.exe            000000000218451F  asx_data_mod_mp_i         840  ASX_DATA_MOD.F
fv3.exe            000000000210AD01  vdiff_                    236  vdiffproc.F
fv3.exe            0000000001CA856E  cmaq_mod_mp_cmaq_         184  cmaq_mod.F90
fv3.exe            0000000001CA5E8D  cmaq_model_mod_mp         139  cmaq_model_mod.F90
fv3.exe            0000000001C1FC97  aqm_comp_mod_mp_a         219  aqm_comp_mod.F90
fv3.exe            0000000001C1DDE8  aqm_mp_modeladvan         594  aqm_cap.F90

Here is a diff of the changes that have been made so far

diff --git a/src/model/src/centralized_io_module.F b/src/model/src/centralized_io_module.F
index c0605e6..e8d5b3a 100644
--- a/src/model/src/centralized_io_module.F
+++ b/src/model/src/centralized_io_module.F
@@ -1670,7 +1670,7 @@
              if (NLDNSTRIKE) then
                  CALL NEXTIME ( loc_jdate(f_ltng), loc_jtime(f_ltng), file_tstep(f_ltng) )
              end if
-             CALL NEXTIME ( loc_jdate(f_bcon), loc_jtime(f_bcon), file_tstep(f_bcon) )
+!             CALL NEXTIME ( loc_jdate(f_bcon), loc_jtime(f_bcon), file_tstep(f_bcon) )
 #endif
 
              do i = 1, N_FILE_GR
diff --git a/src/shr/aqm_methods.F90 b/src/shr/aqm_methods.F90
index 0098d07..5754a61 100644
--- a/src/shr/aqm_methods.F90
+++ b/src/shr/aqm_methods.F90
@@ -682,18 +682,20 @@ logical function interpx( fname, vname, pname, &
     if (aqm_rc_check(localrc, msg="Failure to retrieve grid coordinates", &
       file=__FILE__, line=__LINE__)) return
 
-    if (vname(1:7) == 'LUFRAC_') then
-      lu_index = 0
-      read(vname(8:9), *, iostat=localrc) lu_index
-      if (aqm_rc_test(localrc /= 0, msg="Failure to identify LU_INDEX", &
-        file=__FILE__, line=__LINE__)) return
-      k = 0
-      do r = row0, row1
-        do c = col0, col1
-          k = k + 1
-          if (int(stateIn % stype(c,r)) == lu_index) buffer(k) = 1.0
+    if (len(vname) > 7 ) then
+      if (vname(1:7) == 'LUFRAC_') then
+        lu_index = 0
+        read(vname(8:9), *, iostat=localrc) lu_index
+        if (aqm_rc_test(localrc /= 0, msg="Failure to identify LU_INDEX", &
+          file=__FILE__, line=__LINE__)) return
+        k = 0
+        do r = row0, row1
+          do c = col0, col1
+            k = k + 1
+            if (int(stateIn % stype(c,r)) == lu_index) buffer(k) = 1.0
+          end do
         end do
-      end do
+      end if
     else
       select case (trim(vname))
         case ('HT')
@@ -1147,18 +1149,20 @@ LOGICAL FUNCTION  XTRACT3 ( FNAME, VNAME,                           &
 !      end do
 !    end if
 
-    if (vname(1:7) == 'LUFRAC_') then
-      lu_index = 0
-      read(vname(8:9), *, iostat=localrc) lu_index
-      if (aqm_rc_test(localrc /= 0, msg="Failure to identify LU_INDEX", &
-        file=__FILE__, line=__LINE__)) return
-      k = 0
-      do r = row0, row1
-        do c = col0, col1
-          k = k + 1
-          if (int(stateIn % stype(c,r)) == lu_index) buffer(k) = 1.0
+    if (len(vname) > 7) then
+      if (vname(1:7) == 'LUFRAC_') then
+        lu_index = 0
+        read(vname(8:9), *, iostat=localrc) lu_index
+        if (aqm_rc_test(localrc /= 0, msg="Failure to identify LU_INDEX", &
+          file=__FILE__, line=__LINE__)) return
+        k = 0
+        do r = row0, row1
+          do c = col0, col1
+            k = k + 1
+            if (int(stateIn % stype(c,r)) == lu_index) buffer(k) = 1.0
+          end do
         end do
-      end do
+      end if
     else
       select case (trim(vname))
         case ('HT')

@ytangnoaa
Copy link

ytangnoaa commented Feb 24, 2025

There have been some issues running the model with a debug build, with the most recent issue:

WARNING: map_precursor: NO particulate mercury in model.       
forrtl: severe (408): fort: (3): Subscript #1 of the array WFC has value 0 which is less than the lower bound of 1

Image              PC                Routine            Line        Source             
fv3.exe            000000000BAA303F  Unknown               Unknown  Unknown
fv3.exe            000000000218451F  asx_data_mod_mp_i         840  ASX_DATA_MOD.F
fv3.exe            000000000210AD01  vdiff_                    236  vdiffproc.F
fv3.exe            0000000001CA856E  cmaq_mod_mp_cmaq_         184  cmaq_mod.F90
fv3.exe            0000000001CA5E8D  cmaq_model_mod_mp         139  cmaq_model_mod.F90
fv3.exe            0000000001C1FC97  aqm_comp_mod_mp_a         219  aqm_comp_mod.F90
fv3.exe            0000000001C1DDE8  aqm_mp_modeladvan         594  aqm_cap.F90

Here is a diff of the changes that have been made so far

diff --git a/src/model/src/centralized_io_module.F b/src/model/src/centralized_io_module.F
index c0605e6..e8d5b3a 100644
--- a/src/model/src/centralized_io_module.F
+++ b/src/model/src/centralized_io_module.F
@@ -1670,7 +1670,7 @@
              if (NLDNSTRIKE) then
                  CALL NEXTIME ( loc_jdate(f_ltng), loc_jtime(f_ltng), file_tstep(f_ltng) )
              end if
-             CALL NEXTIME ( loc_jdate(f_bcon), loc_jtime(f_bcon), file_tstep(f_bcon) )
+!             CALL NEXTIME ( loc_jdate(f_bcon), loc_jtime(f_bcon), file_tstep(f_bcon) )
 #endif
 
              do i = 1, N_FILE_GR
diff --git a/src/shr/aqm_methods.F90 b/src/shr/aqm_methods.F90
index 0098d07..5754a61 100644
--- a/src/shr/aqm_methods.F90
+++ b/src/shr/aqm_methods.F90
@@ -682,18 +682,20 @@ logical function interpx( fname, vname, pname, &
     if (aqm_rc_check(localrc, msg="Failure to retrieve grid coordinates", &
       file=__FILE__, line=__LINE__)) return
 
-    if (vname(1:7) == 'LUFRAC_') then
-      lu_index = 0
-      read(vname(8:9), *, iostat=localrc) lu_index
-      if (aqm_rc_test(localrc /= 0, msg="Failure to identify LU_INDEX", &
-        file=__FILE__, line=__LINE__)) return
-      k = 0
-      do r = row0, row1
-        do c = col0, col1
-          k = k + 1
-          if (int(stateIn % stype(c,r)) == lu_index) buffer(k) = 1.0
+    if (len(vname) > 7 ) then
+      if (vname(1:7) == 'LUFRAC_') then
+        lu_index = 0
+        read(vname(8:9), *, iostat=localrc) lu_index
+        if (aqm_rc_test(localrc /= 0, msg="Failure to identify LU_INDEX", &
+          file=__FILE__, line=__LINE__)) return
+        k = 0
+        do r = row0, row1
+          do c = col0, col1
+            k = k + 1
+            if (int(stateIn % stype(c,r)) == lu_index) buffer(k) = 1.0
+          end do
         end do
-      end do
+      end if
     else
       select case (trim(vname))
         case ('HT')
@@ -1147,18 +1149,20 @@ LOGICAL FUNCTION  XTRACT3 ( FNAME, VNAME,                           &
 !      end do
 !    end if
 
-    if (vname(1:7) == 'LUFRAC_') then
-      lu_index = 0
-      read(vname(8:9), *, iostat=localrc) lu_index
-      if (aqm_rc_test(localrc /= 0, msg="Failure to identify LU_INDEX", &
-        file=__FILE__, line=__LINE__)) return
-      k = 0
-      do r = row0, row1
-        do c = col0, col1
-          k = k + 1
-          if (int(stateIn % stype(c,r)) == lu_index) buffer(k) = 1.0
+    if (len(vname) > 7) then
+      if (vname(1:7) == 'LUFRAC_') then
+        lu_index = 0
+        read(vname(8:9), *, iostat=localrc) lu_index
+        if (aqm_rc_test(localrc /= 0, msg="Failure to identify LU_INDEX", &
+          file=__FILE__, line=__LINE__)) return
+        k = 0
+        do r = row0, row1
+          do c = col0, col1
+            k = k + 1
+            if (int(stateIn % stype(c,r)) == lu_index) buffer(k) = 1.0
+          end do
         end do
-      end do
+      end if
     else
       select case (trim(vname))
         case ('HT')

The array of soil type is supposed to > 0,
https://github.com/noaa-oar-arl/AQM/blob/feature/cmaq54/src/shr/aqm_methods.F90#L799
https://github.com/noaa-oar-arl/AQM/blob/feature/cmaq54/src/shr/aqm_methods.F90#L1264

though something could be wrong. To diagnose the issue, one method is adding some checking in
src/shr/aqm_methods.F90, like

case ("SLTYP")
p2d => stateIn % stype
do r = row0, row1
do c = col0, col1
if ( stateIn % stype (c,r) < 1 ) then
if(aqm_rc_check(-1,msg=' stype wrong ')) return
end if
end do
end do

If the stype does show < 1, additional diagnose can be conducted in

https://github.com/noaa-oar-arl/AQM/blob/feature/cmaq54/src/aqm_comp_mod.F90#L601

case ("soil_type")
call ESMF_FieldGet(field, localDe=localDe, farrayPtr=stateIn % stype, rc=rc)
if (ESMF_LogFoundError(rcToCheck=rc, msg=ESMF_LOGERR_PASSTHRU, &
line=LINE, &
file=FILE)) &
return ! bail
if (any(statein%stype<1)) then
call ESMF_LogSetError(ESMF_RC_INTNRL_BAD, msg="Error stype < 1", &
line=LINE, file=FILE, rcToReturn=rc)
return
endif

@BrianCurtis-NOAA
Copy link
Collaborator

Latest error:

forrtl: severe (408): fort: (8): Attempt to fetch from allocatable variable STREAMFAMILYNAME when it is not allocated

Image              PC                Routine            Line        Source
fv3.exe            000000000BAB10DF  Unknown               Unknown  Unknown
fv3.exe            0000000002B757D9  desid_util_mp_des          95  desid_util.F
fv3.exe            0000000002A1041A  aero_emis_mp_desi         368  AERO_EMIS.F

@ytangnoaa
Copy link

Latest error:

forrtl: severe (408): fort: (8): Attempt to fetch from allocatable variable STREAMFAMILYNAME when it is not allocated

Image              PC                Routine            Line        Source
fv3.exe            000000000BAB10DF  Unknown               Unknown  Unknown
fv3.exe            0000000002B757D9  desid_util_mp_des          95  desid_util.F
fv3.exe            0000000002A1041A  aero_emis_mp_desi         368  AERO_EMIS.F

In the file "CMAQ_Control_DESID.nml", try commenting out the namelist "Desid_SizeDist"

!&Desid_SizeDist
! Desid_Sd_nml =
!
! ! Stream Label | Emiss. Mode | Ref. Mode
! 'ALL' ,'FINE' ,'FINE_REF',
! 'ALL' ,'COARSE' ,'COARSE_REF',
! 'WBDUST' ,'FINE' ,'FINE_WBDUST',
! 'WBDUST' ,'COARSE' ,'COARSE_WBDUST',
! 'SEASPRAY' ,'FINE' ,'FINE_SEASPRAY',
! 'SEASPRAY' ,'COARSE' ,'COARSE_SEASPRAY',
! ! 'AIRCRAFT' ,'FINE' ,'AIR_FINE', !To use these examples, you
! ! 'AIRCRAFT' ,'COARSE' ,'AIR_COARSE', ! must add entries for AIR_FINE
! ! and AIR_COARSE to the data structure
! ! em_aero_ref in AERO_DATA.!
!
!/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants