Update CIME to ESMCI cime5.8.7 #3092
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Make atm build using CMake! Switch atm component over to using CMake. I tried to mimic the original behavior as closely as possible. I wrote a tool to compare bld logs and this PR appears to be flag-for-flag identical to old system, with a couple minor exceptions. Change list: * Change FFLAGS_NOPT to be an append to the normal set of flags. This reduces duplication with FFLAGS * Remove Depends files for obsolete/unsupported platforms * Make CMake versions of remaining Depends files * New bld_diff tool for comparing bld logs, necessary to validate this work * Add ability to ask for CMake args (get_standard_cmake_args) * CMake macros should NOT be looking at environment, just CMake vars * Homme test needs to use GMAKE_J and do a verbose build * build_model_thread can now use run_sub_or_cmd * case_setup will make both macro types (Make and CMake) * Import a FindPetsc CMake module * components/CMakeLists.txt will be the CMake file to configure builds of our components. It currently is only doing so for cam. * Remove obsolete bld files from cam * Convert cam buildlib to python * Someone appears to have added their genf90'd files to cam a long time ago, remove them. * Convert cosp to Cmake. Cosp files are now just part of libatm, not their own library. This means CMake can handle the circular dependencies. User impact: * Since this is just the initial phase, user impacts are minimal * Build times look roughly the same, CMake is slightly faster * cosp integration with atm is different, it will need to be managed through cosp/Cosp.cmake. * When adding Depends files for machine/compiler-specific flag changes, both a Make and CMake version will be needed until all components are switched over. * It is no-longer enough to have NETCDF settings in the environment, they need to be explicit in the Macros (config_compilers.xml). * Putting up for an early review due to the high-impact of this PR. [BFB]
[BFB] - Bit-For-Bit
Add support for performance archiving on Compy #2943 Added logic for compy to provenance.py, specified the location of the performance archive in the compy entry in config_machines.xml (enabled for all projects) and added the job progress monitoring script syslog.compy. Also removed performance archiving support for edison, now that it has been decommissioned. [BFB] - Bit-For-Bit
Implements a concept of build groups within test_scheduler. [BFB]
…2961) Implement a more-advanced way of sharing builds within test suites Implements a concept of build groups within test_scheduler. [BFB] * origin/jgfouca/cime/create_test_share_upgrade: Swtich build group output to debug Implement a more-advanced way of sharing builds within test suites
addition uf usermods documentation
These are needed for some new drv namelist variables. When they were just in config_component_e3sm.xml, CESM tests failed with a message like this: ERROR: Namelist default for variable glc2ocn_fmapname refers to unknown XML variable GLC2OCN_FMAPNAME.'
Nuopc cmeps removed. Remove the nuopc driver and mediator to a separate repository https://github.com/ESCOMP/CMEPS.git Test suite: scripts_regression_tests.py Test baseline: Test namelist changes: Test status: bit for bit Fixes User interface changes?: Update gh-pages html (Y/N)?: Code review:
modified: cime/config/e3sm/config_grids.xml modified: components/cam/bld/build-namelist modified: components/cam/bld/namelist_files/namelist_defaults_cam.xml modified: components/clm/bld/namelist_files/namelist_defaults_clm4_5.xml
Need to use run_cmd.
These mapping files are only needed in certain configurations. We need to use idmap_ignore so that the driver's buildnml won't abort for configurations where these mapping files aren't provided and aren't needed.
I erroneously made cubed_sphere_map show up in the namelist, but the correct approach was to modify it in the code and hide this from the user. It is now set to 2 in namelist_mod.F90 when fv_nphys>0. I also added formula for setting ncol in horiz_grid. modified: cime/config/e3sm/config_grids.xml modified: components/cam/bld/build-namelist modified: components/cam/bld/config_files/horiz_grid.xml modified: components/cam/bld/namelist_files/namelist_defaults_cam.xml modified: components/homme/src/share/namelist_mod.F90
…9-05-28 Jgfouca/branch for acme split 2019 05 28 Change List: Add first stage of ocean/ice shelf coupling Bring COMPOSE semi-Lagrangian tracer transport code into HOMME and E3SM (required minor changes to build.py) Add checkpoint of timing data immediately after initialization bless_test_results: Be able to handle build-only tests Test suite: scripts_regression_tests Test baseline: Test namelist changes: Test status: BFB Fixes [CIME Github issue #] User interface changes?: Update gh-pages html (Y/N)?: Code review: @jedwards4b
Add test_release compsets as supported
modified: cime/src/drivers/mct/main/seq_domain_mct.F90 modified: components/cam/src/dynamics/se/dp_coupling.F90 modified: components/cam/src/dynamics/se/dyn_comp.F90
Make lnd build using CMake! Includes some minor fixes and cleanups for atm. [BFB] * jgfouca/clm/cmakify: Sandiatoss3 needs newer cmake Run_sub_or_cmd is not thread safe Progress, building on melvin Progress
all stubs is not supported in nuopc Remove test SMS.T42_T42.S when testing with nuopc driver, all stub cases are not supported. Test suite: scripts_regression_tests.py with CIME_DRIVER=nuopc Test baseline: Test namelist changes: Test status: bit for bit Fixes User interface changes?: Update gh-pages html (Y/N)?: Code review:
* esmci_remote_for_split/master: (1854 commits) fix an issue with shr_assert_mod in mkDepends Update for cime5.8.7 revert whitespace change all stubs is not supported in nuopc Remove a file that was accidently added. Fixes for ne120 gx1v7 masks Fixes for ne120 and se CONUS grid masks resolve conflicts fix pylint issue add support for ensembles Update for cime5.8.6 Add ESMF modules for cheyenne workflow for ensemble more help works without option create an ensemble of clones pylint cleanup getting closer Add support to run SE CONUS grid with mt12 mask more workflow ...
…9-07-22 Jgfouca/branch for acme split 2019 07 22 Bug fixes: * ROOTPE needs to be explicitly set to zero for all component models for PTS_MODE (single column model mode) * Need to check for BUILD_COMPLETE after checking locked files * Make create_dirs more robust when multiple threads are running Features: * Infrastructure upgrade to support finite volume physics grid Test suite: scripts_regression_tests Test baseline: Test namelist changes: Test status: [bit for bit, roundoff, climate changing] Fixes [CIME Github issue #] User interface changes?: Update gh-pages html (Y/N)?: Code review: @jedwards4b
Fix some test fails Fix some fails introduced by #3174 Additional fixes happened here: 0bcb41b Test suite: scripts_regression_tests Test baseline: Test namelist changes: Test status: [bit for bit, roundoff, climate changing] Fixes [CIME Github issue #] User interface changes?: Update gh-pages html (Y/N)?: Code review:
The new mapping files were generated by @mvertens with a recent version of ESMF_RegridWeightGen, with '--ignore_unmapped -m conserve'. The new rof -> lnd map should fix the issue with the old map, that mapping a 1 field from rof -> lnd without normalization leads to non-1 values on the destination. Resolves ESMCI/cime#3185
Point to new mapping files for r05 <-> f10 mapping The new mapping files were generated by @mvertens with a recent version of ESMF_RegridWeightGen, with '--ignore_unmapped -m conserve'. The new rof -> lnd map should fix the issue with the old map, that mapping a 1 field from rof -> lnd without normalization leads to non-1 values on the destination. Test suite: Cherry-picked the change into the cime version used in ctsm master, then ran SMS_D_Ld3.f10_f10_musgs.I1850Clm50BgcCrop.cheyenne_intel.clm-default with comparison against master. Test baseline: ctsm1.0.dev052 Test namelist changes: lnd2rof_fmapname, rof2lnd_fmapname Test status: greater than roundoff, but probably not climate changing; only changes cases using the f10 grid (just used for testing) There are differences in the following CLM fields (these are either directly related to the mapped river volume, or depend on it via methane's dependence on total gridcell water volume): RMS CH4PROD 8.0350E-13 NORMALIZED 2.0526E-05 RMS CH4_SURF_AERE_SAT 3.2242E-11 NORMALIZED 7.6731E-04 RMS CH4_SURF_DIFF_SAT 6.5698E-13 NORMALIZED 3.1933E-03 RMS CH4_SURF_EBUL_SAT 1.2144E-29 NORMALIZED 8.1788E-20 RMS CONC_O2_SAT 1.8803E-03 NORMALIZED 2.7509E-02 RMS FCH4 8.8559E-14 NORMALIZED 3.3512E-03 RMS FCH4TOCO2 8.7528E-11 NORMALIZED 2.9232E-03 RMS FCH4_DFSAT 2.2487E-18 NORMALIZED 7.2389E-05 RMS FINUNDATED 2.5729E-07 NORMALIZED 8.6899E-06 RMS NEM 8.7506E-11 NORMALIZED 5.1107E-03 RMS TOTCOLCH4 9.8119E-08 NORMALIZED 2.0761E-07 RMS TWS 2.6250E-02 NORMALIZED 1.5426E-06 RMS VOLR 4.7216E+07 NORMALIZED 7.2529E-02 RMS VOLRMCH 1.9205E+07 NORMALIZED 7.6222E-02 and in the following cpl fields: RMS x2l_Flrr_volr 3.0371E-05 NORMALIZED 6.2596E-02 RMS x2l_Flrr_volrmch 1.3830E-05 NORMALIZED 6.6570E-02 RMS fracr_lfrac 1.5987E-09 NORMALIZED 4.6844E-09 RMS r2x_Forr_rofl 2.2583E-15 NORMALIZED 2.3902E-09 RMS r2x_Forr_rofi 9.3629E-15 NORMALIZED 1.0087E-08 RMS r2x_Flrr_volr 4.6823E-13 NORMALIZED 1.4978E-09 RMS r2x_Flrr_volrmch 2.7486E-13 NORMALIZED 2.0522E-09 RMS x2r_Flrl_rofsur 1.0427E-15 NORMALIZED 2.9982E-09 RMS x2r_Flrl_rofgwl 2.7807E-16 NORMALIZED 1.4963E-09 RMS x2r_Flrl_rofsub 3.1541E-15 NORMALIZED 1.7387E-09 RMS x2r_Flrl_rofi 9.3625E-15 NORMALIZED 1.0212E-08 Fixes #3185 User interface changes?: N Update gh-pages html (Y/N)?: N Code review:
changes to have TG compset with NUOPC cap work correctly This PR only effects the NUOPC driver and are changes to the dlnd nuopc cap working correctly in a TG compset Test suite: NA (just ran a TG compset - the TG NUOPC capability is stil lbeing developed) Test baseline: NA Test namelist changes: No Test status: N/A Fixes: None User interface changes?: No Update gh-pages html (Y/N)?: N Code review:
…branch-for-to-acme-2019-07-29 * commit 'f1a4a59cdc304214271c72ff767462436da510d4': (3197 commits) Point to new mapping files for r05 <-> f10 mapping Fix some test fails Fixes for broken stuff on master fix an issue with shr_assert_mod in mkDepends Update for cime5.8.7 revert whitespace change all stubs is not supported in nuopc Tar old test results Remove a file that was accidently added. Fixes for ne120 gx1v7 masks Fixes for ne120 and se CONUS grid masks resolve conflicts Make create_dirs more robust when multiple threads are running Update CIME to ESMCI cime5.8.6 (#3073) fix pylint issue add support for ensembles Update for cime5.8.6 Add ESMF modules for cheyenne workflow for ensemble more help ... # Conflicts: # cime/ChangeLog # cime/config/cesm/config_files.xml # cime/config/cesm/config_grids.xml # cime/config/cesm/config_grids_common.xml # cime/config/cesm/config_grids_mct.xml # cime/config/cesm/machines/config_batch.xml # cime/config/cesm/machines/config_compilers.xml # cime/config/cesm/machines/config_machines.xml # cime/config/e3sm/machines/config_batch.xml # cime/scripts/Tools/archive_metadata # cime/scripts/Tools/preview_run # cime/scripts/create_clone # cime/scripts/create_newcase # cime/scripts/lib/CIME/XML/env_batch.py # cime/scripts/lib/CIME/XML/env_mach_specific.py # cime/scripts/lib/CIME/build.py # cime/scripts/lib/CIME/case/case.py # cime/scripts/lib/CIME/case/case_clone.py # cime/scripts/lib/CIME/case/case_run.py # cime/scripts/lib/CIME/case/case_st_archive.py # cime/scripts/lib/CIME/case/case_submit.py # cime/scripts/lib/CIME/get_timing.py # cime/scripts/tests/scripts_regression_tests.py # cime/src/build_scripts/buildlib.csm_share # cime/src/components/data_comps/datm/cime_config/namelist_definition_datm.xml # cime/src/components/data_comps/dice/nuopc/dice_comp_mod.F90 # cime/src/components/data_comps/dlnd/nuopc/dlnd_comp_mod.F90 # cime/src/drivers/mct/cime_config/config_component.xml # cime/src/drivers/mct/cime_config/config_component_cesm.xml
jgfouca
changed the title
|
testing passed, squash-merging to next. |
jgfouca
added a commit
that referenced
this pull request
Jul 30, 2019
Update CIME to ESMCI cime5.8.7 Squash merge of jgfouca/branch-for-to-acme-2019-07-29 Features: * New workflow XML files, replaces some of the stuff that was in env_batch * Several updates and fixes for MOM6 in cime xml files, drivers, and dice: [BFB]
jgfouca
added a commit
that referenced
this pull request
Aug 1, 2019
Update CIME to ESMCI cime5.8.7 Squash merge of jgfouca/branch-for-to-acme-2019-07-29 Features: * New workflow XML files, replaces some of the stuff that was in env_batch * Several updates and fixes for MOM6 in cime xml files, drivers, and dice: [BFB]
|
Jim, I added some things to the description but they didn't make it in to the next commit message. |
|
@rljacob , I must have forgotten to refresh this page when I copy/pasted. I'll make sure master is correct. |
rljacob
approved these changes
Aug 2, 2019
tanzeli1982
pushed a commit
that referenced
this pull request
Aug 12, 2019
Update CIME to ESMCI cime5.8.7 Squash merge of jgfouca/branch-for-to-acme-2019-07-29 Features: A new tool generate_cylc_workflow.py is added. This tool will create a cylc 7.8.3 compatible suite.rc file in the case directory. The user sets the parameters in the case for the span of a submitted run and the number of resubmits, sets up and builds the case and then generates the workflow file. The user can either set the RESUBMIT variable using xmlchange or use --cycles argument to generate_cylc_workflow.py to set the number of times the workflow is repeated. New workflow XML files, replaces some of the stuff that was in env_batch Several updates and fixes for MOM6 in cime xml files, drivers, and dice. Adds a --ensemble option to create_clone allowing the user to generate an ensemble of cases cloned from --clone Bug Fixes: ftp protocol won't make 0-size files anymore Other: Driver namelist change: from eps_ogrid = 0.01 to eps_ogrid = 0.1 [BFB] [NML]
rljacob
pushed a commit
that referenced
this pull request
Apr 12, 2021
Update CIME to ESMCI cime5.8.7 Squash merge of jgfouca/branch-for-to-acme-2019-07-29 Features: A new tool generate_cylc_workflow.py is added. This tool will create a cylc 7.8.3 compatible suite.rc file in the case directory. The user sets the parameters in the case for the span of a submitted run and the number of resubmits, sets up and builds the case and then generates the workflow file. The user can either set the RESUBMIT variable using xmlchange or use --cycles argument to generate_cylc_workflow.py to set the number of times the workflow is repeated. New workflow XML files, replaces some of the stuff that was in env_batch Several updates and fixes for MOM6 in cime xml files, drivers, and dice. Adds a --ensemble option to create_clone allowing the user to generate an ensemble of cases cloned from --clone Bug Fixes: ftp protocol won't make 0-size files anymore Other: Driver namelist change: from eps_ogrid = 0.01 to eps_ogrid = 0.1 [BFB] [NML]
rljacob
pushed a commit
that referenced
this pull request
Apr 21, 2021
Update CIME to ESMCI cime5.8.7 Squash merge of jgfouca/branch-for-to-acme-2019-07-29 Features: A new tool generate_cylc_workflow.py is added. This tool will create a cylc 7.8.3 compatible suite.rc file in the case directory. The user sets the parameters in the case for the span of a submitted run and the number of resubmits, sets up and builds the case and then generates the workflow file. The user can either set the RESUBMIT variable using xmlchange or use --cycles argument to generate_cylc_workflow.py to set the number of times the workflow is repeated. New workflow XML files, replaces some of the stuff that was in env_batch Several updates and fixes for MOM6 in cime xml files, drivers, and dice. Adds a --ensemble option to create_clone allowing the user to generate an ensemble of cases cloned from --clone Bug Fixes: ftp protocol won't make 0-size files anymore Other: Driver namelist change: from eps_ogrid = 0.01 to eps_ogrid = 0.1 [BFB] [NML]
rljacob
pushed a commit
that referenced
this pull request
May 6, 2021
Update CIME to ESMCI cime5.8.7 Squash merge of jgfouca/branch-for-to-acme-2019-07-29 Features: A new tool generate_cylc_workflow.py is added. This tool will create a cylc 7.8.3 compatible suite.rc file in the case directory. The user sets the parameters in the case for the span of a submitted run and the number of resubmits, sets up and builds the case and then generates the workflow file. The user can either set the RESUBMIT variable using xmlchange or use --cycles argument to generate_cylc_workflow.py to set the number of times the workflow is repeated. New workflow XML files, replaces some of the stuff that was in env_batch Several updates and fixes for MOM6 in cime xml files, drivers, and dice. Adds a --ensemble option to create_clone allowing the user to generate an ensemble of cases cloned from --clone Bug Fixes: ftp protocol won't make 0-size files anymore Other: Driver namelist change: from eps_ogrid = 0.01 to eps_ogrid = 0.1 [BFB] [NML]
jgfouca
pushed a commit
that referenced
this pull request
Nov 7, 2024
…d-cov-and-sanitizer Add more eamxx workflows for nightlies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Update CIME to ESMCI cime5.8.7
Squash merge of jgfouca/branch-for-to-acme-2019-07-29
Features:
A new tool generate_cylc_workflow.py is added. This tool will create a cylc 7.8.3 compatible suite.rc file in the case directory. The user sets the parameters in the case for the span of a submitted run and the number of resubmits, sets up and builds the case and then generates the workflow file. The user can either set the RESUBMIT variable using xmlchange or use --cycles argument to generate_cylc_workflow.py to set the number of times the workflow is repeated.
New workflow XML files, replaces some of the stuff that was in env_batch
Several updates and fixes for MOM6 in cime xml files, drivers, and dice.
Adds a --ensemble option to create_clone allowing the user to generate an ensemble of cases cloned from --clone
Bug Fixes:
Other:
Driver namelist change: from eps_ogrid = 0.01 to eps_ogrid = 0.1
[BFB]
[NML]