Merge branch 'whannah/atm/new_physgrid_rebase' (PR #2933)

Infrastructure upgrade to support finite volume physics grid (rebased)

This PR updates the atmosphere grid infrastructure to allow for a
finite volume physics grid (hereafter referred to as physgrid) as
an alternative to the traditional approach of having physics columns
coincide with the GLL nodes of the dynamics grid. Although this
requires many large code changes, (mostly in dyn_grid.F90) the model
is still BFB when run with a typical configuration.

To make sure ongoing development does not break the coupling of the
FV physgrid, I also added a new test "SMS_Ln5.ne4pg2_ne4pg2.FC5AV1C-L"
to the e3sm_atm_developer suite.

[BFB]

* whannah/atm/new_physgrid_rebase: (90 commits)
  Fixed theta bug in stepon
  Used get_temperature to fix Theta test fail
  Fixed theta test bug
  Added max() to fool compiler when fv_nphys = 0
  Add ncols to OMP to fix potential threading issue
  Added new default ATM_NCPL for ne4pg2
  Modified pecount for pg2 test
  Fixed bug related to bad area data in history file
  Added ne4pg2 test to e3sm_atm_developer
  Add support for ne4pg2 w/ topography for testing
  Bug fix
  Added ne30pg2 test to e3sm_atm_extra_coverage
  Removed pg2 test
  Added new test SMS_Ln5.ne30pg2_ne30pg2.FC5AV1C-L
  Added missing hvcoord to use statement
  Added missing  hvcoord to use statement
  Removed initialization of dp3d
  Minor changes, just to be safe
  Minor formatting fix
  Minor bug fix
  ...

Conflicts:
	components/cam/bld/namelist_files/namelist_definition.xml

cime_config/config_component_e3sm.xml

@@ -321,6 +321,7 @@
       <value compset="_CAM.*_CLM.*MPASO" grid="a%ne120np4">96</value>
       <value compset=".+" grid="a%0.23x0.31">96</value>
       <value compset=".+" grid="a%ne4np4">12</value>
+      <value compset=".+" grid="a%ne4np4.pg2">24</value>
       <value compset=".+" grid="a%ne11np4">12</value>
       <value compset=".+" grid="a%ne60np4">96</value>
       <value compset=".+" grid="a%ne120np4">96</value>

main/seq_domain_mct.F90

@@ -672,11 +672,12 @@ subroutine seq_domain_check_grid(dom1, dom2, attr, eps, mpicom, mask)
           diff = abs(data1(n)-data2(n))
           max_diff = max(max_diff,diff)
           if (diff > eps) then
-      !debug            write(logunit,*)'n= ',n,' data1= ',data1(n),' data2= ',data2(n),' diff= ',diff, ' eps= ',eps
+             write(logunit,150) n,data1(n),data2(n),diff,eps
              ndiff = ndiff + 1
           endif
        end if
     end do
+150 format('seq_domain_check_grid - n:',I3,' d1:',F12.6,' d2:',F12.6,' diff:',F18.14,' eps:',F18.14)
 
     call mpi_reduce(max_diff,tot_diff,1,MPI_REAL8,MPI_MAX,0,mpicom,ier)
     if (iamroot) then