Merge branch 'jgfouca/branch-for-to-acme-12-06-2017-pr' into master (PR

#1963) Fixes: * DOCN_MODE fix for xmlchange * compare_namelist bug in rare code path * Fix memleak in gptl * Fix setting of build_threaded * Fix unicode problems with expect Feature list: * Add job-id info for CaseStatus entry for job submit * Allow different glc2ocn mappings for liq and ice [BFB] * jgfouca/branch-for-to-acme-12-06-2017-pr: (21 commits) Ignore buildlib python bytecode files This is no longer needed fix issue in scripts_regression_tests Add ability to customize CaseStatus success message for a phase. Fix bug in compare namelists add test for xmlquery DOCN_MODE Fix memory leak of newtimers in merge_thread_data in splitting comp vars make sure comp isnt subset of name Update ChangeLog nthrds > 1 should set build_threaded flag Update ChangeLog Update ChangeLog fix pylint issue fix gptl threaded build, batch change for cheyenne correct path for includes used in build Check for unicode in expect() update ACME to E3SM and changes related to Issue #1519 and PR #2062 Do not use normalization when mapping glc2ocn Add comments about not needing to add fields to x2o_fluxes and x2i_fluxes Point to new glc2ocn mapping files ...
ESMCI · Dec 7, 2017 · 11d4f5b · 11d4f5b
2 parents adc0d5b + b918106
commit 11d4f5b
Show file tree

Hide file tree

Showing 33 changed files with 375 additions and 134 deletions.
diff --git a/.gitignore b/.gitignore
@@ -1,12 +1,13 @@
 # Ignore python bytecode files
 *.pyc
 buildnmlc
+buildlib.*c
 
 # Ignore emacs backup files
 *~
 
-# Ignore doc/build/doctrees
-##doc/build/doctrees 
 # ignore for the merge to master and orphaning of the gh-pages branch
 doc/build
 
+# ignore the JENKINS files created when make html is run
+scripts/Tools/JENKINS*
diff --git a/ChangeLog b/ChangeLog
@@ -1,5 +1,85 @@
 ======================================================================
 
+Originator: Chris Fischer 
+Date: 12-04-2017
+Tag: cime5.4.0-alpha.13
+Answer Changes: None
+Tests: scripts_regression_tests.py, SMS_P8x2.f19_g16.X.yellowstone_intel
+Dependencies:
+
+Brief Summary:
+    - BUILD_THREADED needs to be set if any nthrds > 1.
+
+User interface changes: 
+
+PR summary: git log --oneline --first-parent [previous_tag]..master 
+307251e Merge pull request #2106 from jedwards4b/nthrds_sets_build_threaded
+
+
+Modified files: git diff --name-status [previous_tag]
+M       scripts/lib/CIME/case_setup.py
+
+======================================================================
+
+======================================================================
+
+Originator: Chris Fischer	 
+Date: 12-03-2017
+Tag: cime5.4.0-alpha.12
+Answer Changes: None
+Tests: scripts_regression_tests.py 
+Dependencies:
+
+Brief Summary:
+    - Fix gptl threaded build, batch change for cheyenne.
+    - Update CIME RST documentation.
+    - Correct path for includes used in build.
+    - Check for unicode in expect().
+    - Separate liq and ice in glc2ocn mapping, and fix glc2ocn mapping bug.
+
+User interface changes: 
+    -Splits GLC2OCN_RMAPNAME xml variable into
+     GLC2OCN_LIQ_RMAPNAME and GLC2OCN_ICE_RMAPNAME, and similarly for
+     GLC2OCN_RMAPTYPE
+
+PR summary: git log --oneline --first-parent [previous_tag]..master 
+6dd6f09 Merge pull request #2105 from jedwards4b/gptl_threaded_build_fix
+92f73e3 Merge pull request #2078 from bertinia/master
+5dc7e19 Merge pull request #2101 from jedwards4b/mpiserial_bldfix
+486efd6 Merge pull request #2097 from ESMCI/sarich/fix-buildexe-unicode
+dd525f9 Merge pull request #2066 from billsacks/glc2ocn_separate_liq_ice
+
+Modified files: git diff --name-status [previous_tag]
+M	.gitignore
+M	config/acme/config_grids.xml
+M	config/cesm/config_grids.xml
+M	config/cesm/machines/config_batch.xml
+M	doc/source/conf.py
+M	doc/source/data_models/data-ocean.rst
+M	doc/source/index.rst
+M	doc/source/misc_tools/index.rst
+M	doc/source/users_guide/cloning-a-case.rst
+M	doc/source/users_guide/introduction-and-overview.rst
+M	doc/source/users_guide/optimizing-processor-layout.rst
+M	doc/source/users_guide/porting-cime.rst
+M	doc/source/users_guide/running-a-case.rst
+M	doc/source/users_guide/testing.rst
+M	doc/source/xml_files/acme.rst
+M	doc/source/xml_files/drivers.rst
+M	scripts/Tools/Makefile
+M	scripts/lib/CIME/utils.py
+M	src/build_scripts/buildlib.gptl
+M	src/build_scripts/buildlib.mpi-serial
+M	src/drivers/mct/cime_config/buildnml
+M	src/drivers/mct/cime_config/config_component.xml
+M	src/drivers/mct/cime_config/namelist_definition_drv.xml
+M	src/drivers/mct/main/prep_ocn_mod.F90
+M	src/drivers/mct/shr/seq_flds_mod.F90
+
+======================================================================
+
+======================================================================
+
 Originator: Chris Fischer	 
 Date: 11-22-2017 
 Tag: cime5.4.0-alpha.11

diff --git a/config/acme/config_grids.xml b/config/acme/config_grids.xml
@@ -2138,7 +2138,8 @@
     <!-- MPASO / MPASLI -->
 
     <gridmap glc_grid="mpas.gis20km" ocn_grid="mpas120">
-      <GLC2OCN_RMAPNAME>cpl/gridmaps/mpas.gis20km/map_mpas.gis20km_to_mpas120_nnsmooth_150924.nc</GLC2OCN_RMAPNAME>
+      <GLC2OCN_LIQ_RMAPNAME>cpl/gridmaps/mpas.gis20km/map_mpas.gis20km_to_mpas120_nnsmooth_150924.nc</GLC2OCN_LIQ_RMAPNAME>
+      <GLC2OCN_ICE_RMAPNAME>cpl/gridmaps/mpas.gis20km/map_mpas.gis20km_to_mpas120_nnsmooth_150924.nc</GLC2OCN_ICE_RMAPNAME>
     </gridmap>
 
     <gridmap ocn_grid="oEC60to30" glc_grid="mpas.ais20km">
@@ -2254,15 +2255,17 @@
       <GLC2OCN_FMAPNAME>cpl/gridmaps/mpas.ais20km/map_ais20km_to_oQU240_aave.151209.nc</GLC2OCN_FMAPNAME>
       <OCN2GLC_FMAPNAME>cpl/gridmaps/oQU240/map_oQU240_to_ais20km_aave.151209.nc</OCN2GLC_FMAPNAME>
       <GLC2OCN_SMAPNAME>cpl/gridmaps/mpas.ais20km/map_ais20km_to_oQU240_nearestdtos.151209.nc</GLC2OCN_SMAPNAME>
-      <GLC2OCN_RMAPNAME>cpl/gridmaps/mpas.ais20km/map_ais20km_to_oQU240_nearestdtos.151209.nc</GLC2OCN_RMAPNAME>
+      <GLC2OCN_LIQ_RMAPNAME>cpl/gridmaps/mpas.ais20km/map_ais20km_to_oQU240_nearestdtos.151209.nc</GLC2OCN_LIQ_RMAPNAME>
+      <GLC2OCN_ICE_RMAPNAME>cpl/gridmaps/mpas.ais20km/map_ais20km_to_oQU240_nearestdtos.151209.nc</GLC2OCN_ICE_RMAPNAME>
       <OCN2GLC_SMAPNAME>cpl/gridmaps/oQU240/map_oQU240_to_ais20km_nearestdtos.151209.nc</OCN2GLC_SMAPNAME>
     </gridmap>
 
     <gridmap glc_grid="mpas.ais20km" ocn_grid="oQU120">
       <GLC2OCN_FMAPNAME>cpl/gridmaps/mpas.ais20km/map_ais20km_to_oQU120_aave.160331.nc</GLC2OCN_FMAPNAME>
       <OCN2GLC_FMAPNAME>cpl/gridmaps/oQU120/map_oQU120_to_ais20km_aave.160331.nc</OCN2GLC_FMAPNAME>
       <GLC2OCN_SMAPNAME>cpl/gridmaps/mpas.ais20km/map_ais20km_to_oQU120_nearestdtos.160331.nc</GLC2OCN_SMAPNAME>
-      <GLC2OCN_RMAPNAME>cpl/gridmaps/mpas.ais20km/map_ais20km_to_oQU120_nearestdtos.160331.nc</GLC2OCN_RMAPNAME>
+      <GLC2OCN_LIQ_RMAPNAME>cpl/gridmaps/mpas.ais20km/map_ais20km_to_oQU120_nearestdtos.160331.nc</GLC2OCN_LIQ_RMAPNAME>
+      <GLC2OCN_ICE_RMAPNAME>cpl/gridmaps/mpas.ais20km/map_ais20km_to_oQU120_nearestdtos.160331.nc</GLC2OCN_ICE_RMAPNAME>
       <OCN2GLC_SMAPNAME>cpl/gridmaps/oQU120/map_oQU120_to_ais20km_nearestdtos.160331.nc</OCN2GLC_SMAPNAME>
     </gridmap>
 

diff --git a/config/cesm/config_grids.xml b/config/cesm/config_grids.xml
@@ -2086,33 +2086,48 @@
     <!-- ======================================================== -->
 
     <gridmap ocn_grid="gx1v6" glc_grid="gland4" >
-      <map name="GLC2OCN_RMAPNAME">cpl/gridmaps/gland4km/map_gland4km_to_gx1v6_nnsm_e1000r300_170501.nc</map>
+      <map name="GLC2OCN_LIQ_RMAPNAME">cpl/gridmaps/gland4km/map_gland4km_to_gx1v6_nn_open_ocean_nnsm_e1000r300_marginal_sea_171105.nc</map>
+      <map name="GLC2OCN_ICE_RMAPNAME">cpl/gridmaps/gland4km/map_gland4km_to_gx1v6_nnsm_e1000r300_171105.nc</map>
     </gridmap>
     <gridmap ocn_grid="gx1v7" glc_grid="gland4" >
-      <map name="GLC2OCN_RMAPNAME">cpl/gridmaps/gland4km/map_gland4km_to_gx1v7_nnsm_e1000r300_170501.nc</map>
+      <map name="GLC2OCN_LIQ_RMAPNAME">cpl/gridmaps/gland4km/map_gland4km_to_gx1v7_nn_open_ocean_nnsm_e1000r300_marginal_sea_171105.nc</map>
+      <map name="GLC2OCN_ICE_RMAPNAME">cpl/gridmaps/gland4km/map_gland4km_to_gx1v7_nnsm_e1000r300_171105.nc</map>
     </gridmap>
+    <!-- POP's estuary box model is currently not active for gx3v7, so
+         we need nnsm maps for liquid as well as ice. -->
     <gridmap ocn_grid="gx3v7" glc_grid="gland4" >
-      <map name="GLC2OCN_RMAPNAME">cpl/gridmaps/gland4km/map_gland4km_to_gx3v7_nnsm_e1000r300_170501.nc</map>
+      <map name="GLC2OCN_LIQ_RMAPNAME">cpl/gridmaps/gland4km/map_gland4km_to_gx3v7_nnsm_e1000r500_171105.nc</map>
+      <map name="GLC2OCN_ICE_RMAPNAME">cpl/gridmaps/gland4km/map_gland4km_to_gx3v7_nnsm_e1000r500_171105.nc</map>
     </gridmap>
 
     <gridmap ocn_grid="gx1v6" glc_grid="gland5UM" >
-      <map name="GLC2OCN_RMAPNAME">cpl/gridmaps/gland5km/map_gland5km_to_gx1v6_nnsm_e1000r300_170501.nc</map>
+      <map name="GLC2OCN_LIQ_RMAPNAME">cpl/gridmaps/gland5km/map_gland5km_to_gx1v6_nn_open_ocean_nnsm_e1000r300_marginal_sea_171105.nc</map>
+      <map name="GLC2OCN_ICE_RMAPNAME">cpl/gridmaps/gland5km/map_gland5km_to_gx1v6_nnsm_e1000r300_171105.nc</map>
     </gridmap>
     <gridmap ocn_grid="gx1v7" glc_grid="gland5UM" >
-      <map name="GLC2OCN_RMAPNAME">cpl/gridmaps/gland5km/map_gland5km_to_gx1v7_nnsm_e1000r300_170501.nc</map>
+      <map name="GLC2OCN_LIQ_RMAPNAME">cpl/gridmaps/gland5km/map_gland5km_to_gx1v7_nn_open_ocean_nnsm_e1000r300_marginal_sea_171105.nc</map>
+      <map name="GLC2OCN_ICE_RMAPNAME">cpl/gridmaps/gland5km/map_gland5km_to_gx1v7_nnsm_e1000r300_171105.nc</map>
     </gridmap>
+    <!-- POP's estuary box model is currently not active for gx3v7, so
+         we need nnsm maps for liquid as well as ice. -->
     <gridmap ocn_grid="gx3v7" glc_grid="gland5UM" >
-      <map name="GLC2OCN_RMAPNAME">cpl/gridmaps/gland5km/map_gland5km_to_gx3v7_nnsm_e1000r300_170501.nc</map>
+      <map name="GLC2OCN_LIQ_RMAPNAME">cpl/gridmaps/gland5km/map_gland5km_to_gx3v7_nnsm_e1000r500_171105.nc</map>
+      <map name="GLC2OCN_ICE_RMAPNAME">cpl/gridmaps/gland5km/map_gland5km_to_gx3v7_nnsm_e1000r500_171105.nc</map>
     </gridmap>
 
     <gridmap ocn_grid="gx1v6" glc_grid="gland20" >
-      <map name="GLC2OCN_RMAPNAME">cpl/gridmaps/gland20km/map_gland20km_to_gx1v6_nnsm_e1000r300_170501.nc</map>
+      <map name="GLC2OCN_LIQ_RMAPNAME">cpl/gridmaps/gland20km/map_gland20km_to_gx1v6_nn_open_ocean_nnsm_e1000r300_marginal_sea_171105.nc</map>
+      <map name="GLC2OCN_ICE_RMAPNAME">cpl/gridmaps/gland20km/map_gland20km_to_gx1v6_nnsm_e1000r300_171105.nc</map>
     </gridmap>
     <gridmap ocn_grid="gx1v7" glc_grid="gland20" >
-      <map name="GLC2OCN_RMAPNAME">cpl/gridmaps/gland20km/map_gland20km_to_gx1v7_nnsm_e1000r300_170501.nc</map>
+      <map name="GLC2OCN_LIQ_RMAPNAME">cpl/gridmaps/gland20km/map_gland20km_to_gx1v7_nn_open_ocean_nnsm_e1000r300_marginal_sea_171105.nc</map>
+      <map name="GLC2OCN_ICE_RMAPNAME">cpl/gridmaps/gland20km/map_gland20km_to_gx1v7_nnsm_e1000r300_171105.nc</map>
     </gridmap>
+    <!-- POP's estuary box model is currently not active for gx3v7, so
+         we need nnsm maps for liquid as well as ice. -->
     <gridmap ocn_grid="gx3v7" glc_grid="gland20" >
-      <map name="GLC2OCN_RMAPNAME">cpl/gridmaps/gland20km/map_gland20km_to_gx3v7_nnsm_e1000r300_170501.nc</map>
+      <map name="GLC2OCN_LIQ_RMAPNAME">cpl/gridmaps/gland20km/map_gland20km_to_gx3v7_nnsm_e1000r500_171105.nc</map>
+      <map name="GLC2OCN_ICE_RMAPNAME">cpl/gridmaps/gland20km/map_gland20km_to_gx3v7_nnsm_e1000r500_171105.nc</map>
     </gridmap>
 
   </gridmaps>

diff --git a/config/cesm/machines/config_batch.xml b/config/cesm/machines/config_batch.xml
@@ -294,7 +294,8 @@
       <directive> -l select={{ num_nodes }}:ncpus={{ max_tasks_per_node }}:mpiprocs={{ tasks_per_node }}:ompthreads={{ thread_count }}</directive>
     </directives>
     <queues>
-      <queue default="true" walltimemax="12:00" nodemin="1" nodemax="72">regular</queue>
+      <queue default="true" walltimemax="12:00" nodemin="1" nodemax="32">regular</queue>
+      <queue walltimemax="12:00" nodemin="1" nodemax="32">economy</queue>
     </queues>
   </batch_system>
 

diff --git a/doc/source/conf.py b/doc/source/conf.py
@@ -55,16 +55,16 @@
 # General information about the project.
 project = u'CIME'
 copyright = u'2017, U.S. National Science Foundation and U.S. Department of Energy'
-author = u'Staff of the NSF/CESM and DOE/ACME projects'
+author = u'Staff of the NSF/CESM and DOE/E3SM projects'
 
 # The version info for the project you're documenting, acts as replacement for
 # |version| and |release|, also used in various other places throughout the
 # built documents.
 #
 # The short X.Y version.
-version = u'5.3.0'
+version = u'5.4.0'
 # The full version, including alpha/beta/rc tags.
-release = u'5.3.0'
+release = u'5.4.0'
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.
@@ -91,13 +91,14 @@
 # a list of builtin themes.
 #
 #html_theme = 'alabaster'
-html_theme = 'bizstyle'
+#html_theme = 'bizstyle'
+html_theme = 'classic'
 
 # Theme options are theme-specific and customize the look and feel of a theme
 # further.  For a list of options available for each theme, see the
 # documentation.
 #
-html_theme_options = { }
+html_theme_options = {"stickysidebar": "true"}
 
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
@@ -135,7 +136,7 @@
 #  author, documentclass [howto, manual, or own class]).
 latex_documents = [
     (master_doc, 'on.tex', u'on Documentation',
-     u'Staff of the NSF/CESM and DOE/ACME projects', 'manual'),
+     u'Staff of the NSF/CESM and DOE/E3SM projects', 'manual'),
 ]
 
 
@@ -166,7 +167,7 @@
     (master_doc,
      u'CIME_Users_Guide', 
      u'CIME Users Guide (PDF)', 
-     u'Staff of the NSF/CESM and DOE/ACME projects'),
+     u'Staff of the NSF/CESM and DOE/E3SM projects'),
 ]
 
 
diff --git a/doc/source/data_models/data-ocean.rst b/doc/source/data_models/data-ocean.rst
@@ -13,31 +13,53 @@ DOCN has two distinct modes of operation.
 DOCN can run as a pure data model, reading in ocean SSTs (normally climatological) from input datasets, performing time/spatial  interpolations, and passing these to the coupler. 
 Alternatively, DOCN can compute updated SSTs by running as a slab ocean model where bottom ocean heat flux convergence and boundary layer depths are read in and used with the atmosphere/ocean and ice/ocean fluxes obtained from the driver.
 
-DOCN running in prescribed mode assumes that the only field in the input stream is SST and also that SST is in Celsius and must be converted to Kelvin. 
-All other fields are set to zero except for ocean salinity, which is set to a constant reference salinity value. 
-Normally the ice fraction data (used for prescribed CICE) is found in the same data files that provide SST data to the data ocean model since SST and ice fraction data are derived from the same observational data sets and are consistent with each other. 
-For DOCN prescribed mode, default yearly climatological datasets are provided for various model resolutions.
-
-DOCN running as a slab ocean model is used in conjunction with active ice mode running in full prognostic mode (e.g. CICE for CESM).
-This mode computes a prognostic sea surface temperature and a freeze/melt potential (surface Q-flux) used by the sea ice model. 
-This calculation requires an external SOM forcing data file that includes ocean mixed layer depths and bottom-of-the-slab Q-fluxes. 
-Scientifically appropriate bottom-of-the-slab Q-fluxes are normally ocean resolution dependent and are derived from the ocean model output of a fully coupled CCSM run. 
-Note that this mode no longer runs out of the box, the default testing SOM forcing file is not scientifically appropriate and is provided for testing and development purposes only. 
-Users must create scientifically appropriate data for their particular application or use one of the standard SOM forcing files from the full prognostic control runs. 
-For CESM, some of these are available in the `inputdata repository <https://svn-ccsm-inputdata.cgd.ucar.edu/trunk/inputdata/ocn/docn7/SOM/>`_. 
-The user then modifies the ``$DOCN_SOM_FILENAME`` variable in env_run.xml to point to the appropriate SOM forcing dataset. 
-
-.. note:: A tool is available to derive valid `SOM forcing <http://www.cesm.ucar.edu/models/ccsm1.1/data8/SOM.pdf>`_ and more information on creating the SOM forcing is also available.
+DOCN running in prescribed mode assumes that the only field in the
+input stream is SST and also that SST is in Celsius and must be
+converted to Kelvin.  All other fields are set to zero except for
+ocean salinity, which is set to a constant reference salinity value.
+Normally the ice fraction data (used for prescribed CICE) is found in
+the same data files that provide SST data to the data ocean model
+since SST and ice fraction data are derived from the same
+observational data sets and are consistent with each other.  For DOCN
+prescribed mode, default yearly climatological datasets are provided
+for various model resolutions.
+
+DOCN running as a slab ocean model is used in conjunction with active
+ice mode running in full prognostic mode (e.g. CICE for CESM).  This
+mode computes a prognostic sea surface temperature and a freeze/melt
+potential (surface Q-flux) used by the sea ice model.  This
+calculation requires an external SOM forcing data file that includes
+ocean mixed layer depths and bottom-of-the-slab Q-fluxes.
+Scientifically appropriate bottom-of-the-slab Q-fluxes are normally
+ocean resolution dependent and are derived from the ocean model output
+of a fully coupled CCSM run.  Note that this mode no longer runs out
+of the box, the default testing SOM forcing file is not scientifically
+appropriate and is provided for testing and development purposes only.
+Users must create scientifically appropriate data for their particular
+application or use one of the standard SOM forcing files from the full
+prognostic control runs.  For CESM, some of these are available in the
+`inputdata repository
+<https://svn-ccsm-inputdata.cgd.ucar.edu/trunk/inputdata/ocn/docn7/SOM/>`_.
+The user then modifies the ``$DOCN_SOM_FILENAME`` variable in
+env_run.xml to point to the appropriate SOM forcing dataset.
+
+.. note:: A tool is available to derive valid `SOM forcing
+<http://www.cesm.ucar.edu/models/cesm1.2/data8/doc/SOM.pdf>`_ and more
+information on creating the SOM forcing is also available.
 
 .. _docn-xml-vars:
 
 -------------
 xml variables
 -------------
 
-The following are xml variables that CIME supports for DOCN. 
-These variables are defined in ``$CIMEROOT/src/components/data_comps/docn/cime_config/config_component.xml``.
-These variables will appear in ``env_run.xml`` and are used by the DOCN ``cime_config/buildnml`` script to generate the DOCN namelist file ``docn_in`` and the required associated stream files for the case.
+The following are xml variables that CIME supports for DOCN.  These
+variables are defined in
+``$CIMEROOT/src/components/data_comps/docn/cime_config/config_component.xml``.
+These variables will appear in ``env_run.xml`` and are used by the
+DOCN ``cime_config/buildnml`` script to generate the DOCN namelist
+file ``docn_in`` and the required associated stream files for the
+case.
 
 .. note:: These xml variables are used by the the docn's **cime_config/buildnml** script in conjunction with docn's **cime_config/namelist_definition_docn.xml** file to generate the namelist file ``docn_in``.
 

diff --git a/doc/source/index.rst b/doc/source/index.rst
@@ -43,5 +43,5 @@ Python Module Indices and Search
 
 
 CIME is developed by the 
-`ACME <https://climatemodeling.science.energy.gov/projects/accelerated-climate-modeling-energy>`_ and 
+`E3SM <https://climatemodeling.science.energy.gov/projects/energy-exascale-earth-system-model>`_ and 
 `CESM <http://www.cesm.ucar.edu/>`_ projects.
diff --git a/doc/source/misc_tools/index.rst b/doc/source/misc_tools/index.rst
@@ -17,13 +17,10 @@ tools that are necessary and/or useful when building a climate model.  Guides fo
    :numbered:
 
 
-Statistical Ensemble Test
-
-Mapping Tools
-
-cprnc
-
-load-balancing-tool.rst
+   Statistical Ensemble Test
+   Mapping Tools
+   cprnc
+   load-balancing-tool.rst
 
 Indices and tables
 ==================