[BFB] - Bit-For-Bit

Fix domain files setting for the CONUS RRM configuration. Specification
of the domain files for the CONUS setup were missing from config_grids.xml,
so hgrid was not being set, causing set_horiz_grid to fail during case
setup. Adding specification of which domain files to use fixes this
issue. Fixes #2147.

Add a small test suite to test basic functionality of supported RRM
configurations. These tests use large grids, so more nodes may be
required to provide sufficient memory. This can be specified when
running the test suite using the --force-procs option to create_test.
I.e., create_test e3sm_rrm --force-procs 64.

Add a default PE layout for all RRM grids (any atmosphere grid that
begins with "ne0np4"). These grids are all large, and require a larger
amount of processors to have access to sufficient memory.

Add RRM-specific tests from the e3sm_rrm test suite to the
e3sm_integration test suite to test basic RRM functionality before
merging new code changes.

This fixes the CTest script for machines with nersc_slurm queue types.
An alternative (and potentially preferred) fix would be to figure out how to create a subtype of the slurm queue in the xml and just use that.

Fix CONUS RRM configuration

Fix two issues with the CONUS RRM configuration that prevented configuring a case with CONUS. First, the domain files were not specified for the CONUS grid in config_grids.xml, so hgrid ended up remaining UNSET, causing set_horiz_grid to fail with all components having grid values UNSET. This PR puts the domain file specification back into config_grids.xml for CONUS, and fixes #2147. Second, an older land initial condition (finidat) file was being used for sim year 2000 runs, which was incompatible with recent changes, causing runs to fail. This PR replaces finidat for sim year 2000 with the updated version.  Fixes #1899

This PR also adds a test suite for RRM grids.

[BFB]

Move the RRM test suite to e3sm_extra_coverage and keep only the CONUS
RRM test in e3sm_integration to speed up the integration testing.

Move the allactive usermods_dir to the same
level as the testmods_dir

Remove the CONUS test from the e3sm_extra_coverage suite since it is
already in e3sm_integration.

Add a testmod to turn on most of the options of
the v1 CMIP6 DECK runs including: extra atm output files,
cosp, coupler budgets.

Add CMIP6 wcycl ne30 test cases to production (2 days) and integration (1 day
with debugging).

Remove SMS.ne30_oECv3_ICG.A_WCYCL1850S.  This is replaced by the
SMS_D_Ld1 test with A_WCYCL1850S_CMIP6

So the test work completely fail if git describe fails

Add a ne30_oECv3_ICG.A_WCYCL1850S_CMIP6 test to e3sm_prod and include test mods that turn on production-level output. Also add this case to e3sm_integration but with debugging on.
Remove the SMS.ne30_oECv3_ICG.A_WCYCL1850S, aka "CMIP5", test.

Move the allactive "usermods_dirs" subdirectory up a level to the same as testmods_dirs.

[BFB]

Adding configuration file sections for Summit using PGI, IBM compilers.
Several settings are adopted from Summitdev, a testbed which was a
precursor to Summit.
 * Added specific compiler sections to config_machines.xml
 * Added new machine section in config_machines.xml
 * Updated LSF section in config_batch.xml,removed obsolete lsf_old sec.
 * mpirun.summit script is installed locally in project-wide accessible
   directory. Work in progress to tune it for the environment.

 Testing: Ran FC5AV1C with ne4_ne4 and ne30_ne30.

[BFB] - Bit-For-Bit

Adding configuration file sections for Summit using PGI, IBM compilers.
Several settings are adopted from Summitdev, a testbed which was a
precursor to Summit.

 - Added specific compiler sections to config_machines.xml
- Added new machine section in config_machines.xml
- Updated LSF section in config_batch.xml,removed obsolete lsf_old sec.

Note: mpirun.summit script is installed locally in project-wide accessible
directory. Work in progress to tune it for the environment.
Testing: Ran FC5AV1C with ne4_ne4 and ne30_ne30.

[BFB] - Bit-For-Bit

Fixes NAG identified errors in atm, homme and clm models

Most of changes are related to line length longer than 132 characters,
which NAG doesn't allow. Other changes are related where the whole F90
files were enclosed in #if defines. NAG complains about these files
as it sees no code in those files when those #if defines are not
true. Previously Gautam identified that these kind of files may
force recompiliation of the some codes even if no change is made to
any of the F90 files.

[BFB] - Bit-For-Bit

Add pet testmods for Theta
so that multi-submit test jobs go through production queue.

[BFB]

Minor changes to be compatible with the recent changes in CIME.

[BFB]

Update cime to ESMCI 5.4.0-alpha.25

Feature list:

* Improve handling of "non-core" jobs (st.archive) allow for better queue selection for this job
* Reinstate ability to select queue by num tasks
* Add support for environment and module to depend on job
* Mpirun can now depend on queue
* Implement improvements to st_archive regular expressions
* enabling archiving of dates year >9999
* Add ability to xmlquery derived attributes (e.g. NUM_TASKS)
* Provide infrastructure to allow SCM to run with SE dycore
* Allow custom batch directive prefixes

Bug fixes:

* Report lock file info as debug output, not info.
* Remove aliasing of element blocks in env_batch
* Remove most usage of scan_children and replace with root discovery
* Fix mistake in scripts-regression-tests help dump
* Improve create_test error message when user provides non-existent test suite

[BFB]

This test is typically, by far, the slowest of our integration tests
to complete. Since we are only doing a Ln9 run, it makes no sense to
use a large peylayout. A much better approach is to use the PS test option
to force a smaller layout so we can get in the queue faster.

This PR also defined a small grid layout for sandiatoss3.

This PR allows both phases of the above test to fit into the sandiatoss3
"short" queue.

[BFB]

* esmci_remote_for_split/master: (111 commits)
  remove debug print statement
  fixes a py3 issue
  Allow case_diff to work on cases that are not tests
  Cannot combine soft operations.
  remove refs to ccsm
  remove refs to ccsm
  Add coment
  Permit IOErrors as well as OSErrors when trying to copy crpnc log
  use __init__.py to move entire case object to subdirectory
  add a README
  add __init__ file
  more rearrangement
  fix pylint issues
  update templates
  put case objects in subdirectory
  take it to the limit
  cleanup acme -> e3sm references in rst documentation
  Add an A and X compset test to aux_cime_baselines.
  fix comment
  fix comment
  ...