Skip to content

Commit

Permalink
Merge pull request #1970 from ESMCI/jgfouca/branch-for-acme-split-10-…
Browse files Browse the repository at this point in the history 
…17-2017-pr

Bring in E3SM cime changes for past 3 months

Major change list:

Python

Minor improvements to bless_test_results.py
Add mach-specific throughput checking to test machines
Some tools were not loading environment before running which was causing problems on some platforms
Improvements in robustness, error msgs of check_input_data
configure.py new FakeCase class
add FakeCase to fortran_unit_testing/run_tests.py
system_tests_common.py change to tput test
env_batch.py add new CHARGE_ACCOUNT
env_mach_specific.py - Allow config_machines.xml to have any case variable as an attribute,
pes.py - also output threads, root
generic_xml.py - remove minidom
build.py - add USE_MOAB option
case.py - use new CHARGE_ACCOUNT. new SMP_PRESENT
provenancy.py - update E3SM provenance.
test_scheduler.py - tput tolerance. new "R" modifier for single column tests.
utils.py new get_charge_account method
update_acme_tests.py - more tests and test suites.
schema changes

config_compilers_v2.xsd: Add MOAB
config_machines.xsd: Add CHARGE_ACCOUNT for charging to a different name then PROJECT used for directory paths
config_machines_template.xml: Also add CHARGE_ACCOUNT
env_mach_specific.xsd: Change debug to upperBoolean
Tools

new acme_cime_split to help with subtree splitting
new tests for the run_acme script in scripts/test/run_acme_test
xml changes

Change "debug" to "DEBUG" in all config_machines.xml
** data models **

datm/cime_config/namelist_definition_datm.xml new 6hourly forcing data set
** mct driver **

config_component.xml new SMP_PRESENT, USE_MOAB, TEST_TPUT_TOLERANCE, CHARGE_ACCOUNT
cime_comp_mod.F90 new option to shr_mem_getusage to also print.
seq_diag_mct.F90 fix an E3SM diag
timing

fix for BGQ
shr_reprosum_mod.F90 - optionally check for nans.

Rename some man3 pages in PIO1

Many E3SM config changes.

Test suite: scripts_regression_tests
Test baseline:
Test namelist changes:
Test status: bit for bit

Fixes ESMCI/cime#1332
Fixes ESMCI/cime#1626

User interface changes?:

Update gh-pages html N

Code review:
  • Loading branch information
@jgfouca
jgfouca authored Oct 23, 2017
2 parents 376d667 + 120f2a2 commit 5fc342d
Show file tree
Hide file tree
Showing 5 changed files with 133 additions and 19 deletions.
43 changes: 34 additions & 9 deletions cime_config/config_component.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -734,6 +734,15 @@
If TRUE, the component libraries are always built with OpenMP capability.</desc>
</entry>

<entry id="SMP_PRESENT">
<type>logical</type>
<valid_values>TRUE,FALSE</valid_values>
<default_value>FALSE</default_value>
<group>build_def</group>
<file>env_build.xml</file>
<desc>TRUE implies that at least one of the components is built threaded (DO NOT EDIT)</desc>
</entry>

<entry id="USE_PETSC">
<type>logical</type>
<valid_values>TRUE,FALSE</valid_values>
Expand Down Expand Up @@ -761,6 +770,15 @@
machines.</desc>
</entry>

<entry id="USE_MOAB">
<type>logical</type>
<valid_values>TRUE,FALSE</valid_values>
<default_value>FALSE</default_value>
<group>build_def</group>
<file>env_build.xml</file>
<desc>TRUE implies linking to the MOAB library</desc>
</entry>

<entry id="USE_TRILINOS">
<type>logical</type>
<valid_values>TRUE,FALSE</valid_values>
Expand Down Expand Up @@ -895,15 +913,6 @@
<desc>logical to diagnose model timing at the end of the run</desc>
</entry>


<entry id="TIMER_LEVEL">
<type>integer</type>
<default_value>12</default_value>
<group>run_flags</group>
<file>env_run.xml</file>
<desc>timer output depth</desc>
</entry>

<entry id="PROFILE_PAPI_ENABLE">
<type>logical</type>
<valid_values>TRUE,FALSE</valid_values>
Expand Down Expand Up @@ -2375,6 +2384,14 @@
<desc>Expected relative memory usage growth for test</desc>
</entry>

<entry id="TEST_TPUT_TOLERANCE">
<type>real</type>
<default_value>0.25</default_value>
<group>test</group>
<file>env_test.xml</file>
<desc>Expected throughput deviation</desc>
</entry>

<entry id="GENERATE_BASELINE">
<type>logical</type>
<valid_values>TRUE,FALSE</valid_values>
Expand Down Expand Up @@ -2630,6 +2647,14 @@
<desc>project for project-sensitive build and run paths, and job scripts</desc>
</entry>

<entry id="CHARGE_ACCOUNT">
<type>char</type>
<default_value></default_value>
<group>job_submission</group>
<file>env_batch.xml</file>
<desc>project to charge in scripts if different from PROJECT</desc>
</entry>

<entry id="MODEL_VERSION">
<type>char</type>
<default_value>unknown</default_value>
Expand Down
89 changes: 85 additions & 4 deletions cime_config/config_component_acme.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,14 @@
</entry>

<entry id="TIMER_DETAIL">
<type>integer</type>
<default_value>12</default_value>
<group>run_flags</group>
<file>env_run.xml</file>
<desc>timer output depth</desc>
</entry>

<entry id="TIMER_LEVEL">
<type>integer</type>
<default_value>20</default_value>
<group>run_flags</group>
Expand Down Expand Up @@ -170,9 +178,13 @@
<value compset="_DATM" >none</value>
<value compset="_BGC%BPRP">CO2C</value>
<value compset="_BGC%BDRD">CO2C</value>
<value compset="HIST.*_DATM%(QIA|CRU)">CO2A</value>
<value compset="20TR.*_DATM%(QIA|CRU)">CO2A</value>
<value compset="RCP.*_DATM%(QIA|CRU)" >CO2A</value>
<value compset="HIST.*_DATM%(QIA|CRU|GSW)">CO2A</value>
<value compset="20TR.*_DATM%(QIA|CRU|GSW)">CO2A</value>
<value compset="RCP.*_DATM%(QIA|CRU|GSW)" >CO2A</value>
<value compset="_DATM%IAF*_MPASO%OECO" >CO2A</value>
<value compset="_DATM%IAF*_MPASO%OIECO" >CO2A</value>
<value compset="_DATM%IAF*_MPASO%TOIECO_" >CO2A</value>
<value compset="_DATM%IAF_SLND_MPASCICE%BGC_MPASO%TOIECO_" >CO2A</value>
</values>
<group>run_coupling</group>
<file>env_run.xml</file>
Expand Down Expand Up @@ -266,6 +278,8 @@
<value compset=".+" grid="a%ne0np4_conus_x4v1" >96</value>
<value compset=".+" grid="a%ne0np4_svalbard_x8v1" >144</value>
<value compset=".+" grid="a%ne0np4_sooberingoa_x4x8v1" >144</value>
<value compset=".+" grid="a%ne0np4_enax4v1" >96</value>
<value compset=".+" grid="a%ne0np4_twpx4v1">96</value>
<value compset=".+" grid="a%T42">72</value>
<value compset=".+" grid="a%T85">144</value>
<value compset=".+" grid="a%T341">288</value>
Expand Down Expand Up @@ -439,7 +453,6 @@
<default_value>FALSE</default_value>
<values match="last">
<value compset="DATM.+POP\d">TRUE</value>
<value compset="DATM.*_MPASO">TRUE</value>
<value compset="DATM.+DOCN%IAF">TRUE</value>
</values>
<group>run_component_cpl</group>
Expand Down Expand Up @@ -680,6 +693,74 @@
compsets.</desc>
</entry>

<entry id="OCN2GLC_FMAPNAME">
<type>char</type>
<default_value>idmap</default_value>
<group>run_domain</group>
<file>env_run.xml</file>
<desc>ocn2glc flux mapping file</desc>
</entry>

<entry id="OCN2GLC_FMAPTYPE">
<type>char</type>
<valid_values>X,Y</valid_values>
<default_value>Y</default_value>
<group>run_domain</group>
<file>env_run.xml</file>
<desc>ocn2glc flux mapping file decomp type</desc>
</entry>

<entry id="OCN2GLC_SMAPNAME">
<type>char</type>
<default_value>idmap</default_value>
<group>run_domain</group>
<file>env_run.xml</file>
<desc>ocn2glc state mapping file</desc>
</entry>

<entry id="OCN2GLC_SMAPTYPE">
<type>char</type>
<valid_values>X,Y</valid_values>
<default_value>Y</default_value>
<group>run_domain</group>
<file>env_run.xml</file>
<desc>ocn2glc state mapping file decomp type</desc>
</entry>

<entry id="GLC2OCN_FMAPNAME">
<type>char</type>
<default_value>idmap</default_value>
<group>run_domain</group>
<file>env_run.xml</file>
<desc>glc2ocn flux mapping file</desc>
</entry>

<entry id="GLC2OCN_FMAPTYPE">
<type>char</type>
<valid_values>X,Y</valid_values>
<default_value>Y</default_value>
<group>run_domain</group>
<file>env_run.xml</file>
<desc>glc2ocn flux mapping file decomp type</desc>
</entry>

<entry id="GLC2OCN_SMAPNAME">
<type>char</type>
<default_value>idmap</default_value>
<group>run_domain</group>
<file>env_run.xml</file>
<desc>glc2ocn state mapping file</desc>
</entry>

<entry id="GLC2OCN_SMAPTYPE">
<type>char</type>
<valid_values>X,Y</valid_values>
<default_value>Y</default_value>
<group>run_domain</group>
<file>env_run.xml</file>
<desc>glc2ocn state mapping file decomp type</desc>
</entry>

<entry id="CCSM_GCOST">
<type>integer</type>
<valid_values></valid_values>
Expand Down
8 changes: 8 additions & 0 deletions cime_config/config_component_cesm.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,14 @@
<desc>timer output depth</desc>
</entry>

<entry id="TIMER_LEVEL">
<type>integer</type>
<default_value>12</default_value>
<group>run_flags</group>
<file>env_run.xml</file>
<desc>timer output depth</desc>
</entry>

<entry id="DOUT_S_SAVE_INTERIM_RESTART_FILES">
<type>logical</type>
<valid_values>TRUE,FALSE</valid_values>
Expand Down
2 changes: 1 addition & 1 deletion main/cime_comp_mod.F90
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3843,7 +3843,7 @@ subroutine cime_run()
ice(ens1)%iamroot_compid .or. &
glc(ens1)%iamroot_compid .or. &
wav(ens1)%iamroot_compid) then
call shr_mem_getusage(msize,mrss)
call shr_mem_getusage(msize,mrss,.true.)

write(logunit,105) ' memory_write: model date = ',ymd,tod, &
' memory = ',msize,' MB (highwater) ',mrss,' MB (usage)', &
Expand Down
10 changes: 5 additions & 5 deletions main/seq_diag_mct.F90
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1348,13 +1348,13 @@ subroutine seq_diag_ocn_mct( ocn, xao_o, frac_o, infodata, do_o2x, do_x2o, do_xa
di = dom_o%data%rAttr(kArea,n) * frac_o%rAttr(ki,n)
if = f_area; budg_dataL(if,ic,ip) = budg_dataL(if,ic,ip) + do
if (trim(cime_model) == 'acme') then
if = f_hfrz; budg_dataL(if,ic,ip) = budg_dataL(if,ic,ip) + (do+di)*max(0.0_r8,o2x_o%rAttr(index_o2x_Fioo_frazil,n))
if = f_wfrz; budg_dataL(if,ic,ip) = budg_dataL(if,ic,ip) - (do+di)*max(0.0_r8,o2x_o%rAttr(index_o2x_Fioo_frazil,n))
else if (trim(cime_model) == 'cesm') then
if = f_hfrz; budg_dataL(if,ic,ip) = budg_dataL(if,ic,ip) + (do+di)*max(0.0_r8,o2x_o%rAttr(index_o2x_Fioo_q,n))
end if
end do
if (trim(cime_model) == 'acme') then
budg_dataL(f_wfrz,ic,ip) = budg_dataL(f_hfrz,ic,ip) * HFLXtoWFLX * shr_const_rhoice * shr_const_latice
budg_dataL(f_hfrz,ic,ip) = -budg_dataL(f_wfrz,ic,ip) * shr_const_latice
else if (trim(cime_model) == 'cesm') then
budg_dataL(f_wfrz,ic,ip) = budg_dataL(f_hfrz,ic,ip) * HFLXtoWFLX
end if
Expand Down Expand Up @@ -1689,7 +1689,7 @@ subroutine seq_diag_ice_mct( ice, frac_i, infodata, do_i2x, do_x2i)
if = f_wioff; budg_dataL(if,ic,ip) = budg_dataL(if,ic,ip) + di*x2i_i%rAttr(index_x2i_Fixx_rofi,n)

if (trim(cime_model) == 'acme') then
if = f_hfrz ; budg_dataL(if,ic,ip) = budg_dataL(if,ic,ip) - (do+di)*max(0.0_r8,x2i_i%rAttr(index_x2i_Fioo_frazil,n))
if = f_wfrz ; budg_dataL(if,ic,ip) = budg_dataL(if,ic,ip) + (do+di)*max(0.0_r8,x2i_i%rAttr(index_x2i_Fioo_frazil,n))
else if (trim(cime_model) == 'cesm') then
if = f_hfrz ; budg_dataL(if,ic,ip) = budg_dataL(if,ic,ip) - (do+di)*max(0.0_r8,x2i_i%rAttr(index_x2i_Fioo_q,n))
end if
Expand Down Expand Up @@ -1719,7 +1719,7 @@ subroutine seq_diag_ice_mct( ice, frac_i, infodata, do_i2x, do_x2i)
budg_dataL(f_hlatf,ic,ip) = -budg_dataL(f_wsnow,ic,ip)*shr_const_latice
budg_dataL(f_hioff,ic,ip) = -budg_dataL(f_wioff,ic,ip)*shr_const_latice
if (trim(cime_model) == 'acme') then
budg_dataL(f_wfrz ,ic,ip) = budg_dataL(f_hfrz ,ic,ip)*HFLXtoWFLX * shr_const_rhoice * shr_const_latice
budg_dataL(f_hfrz ,ic,ip) = -budg_dataL(f_wfrz ,ic,ip)*shr_const_latice
else if (trim(cime_model) == 'cesm') then
budg_dataL(f_wfrz ,ic,ip) = budg_dataL(f_hfrz ,ic,ip)*HFLXtoWFLX
end if
Expand All @@ -1728,7 +1728,7 @@ subroutine seq_diag_ice_mct( ice, frac_i, infodata, do_i2x, do_x2i)
budg_dataL(f_hlatf,ic,ip) = -budg_dataL(f_wsnow,ic,ip)*shr_const_latice
budg_dataL(f_hioff,ic,ip) = -budg_dataL(f_wioff,ic,ip)*shr_const_latice
if (trim(cime_model) == 'acme') then
budg_dataL(f_wfrz ,ic,ip) = budg_dataL(f_hfrz ,ic,ip)*HFLXtoWFLX * shr_const_rhoice * shr_const_latice
budg_dataL(f_hfrz ,ic,ip) = -budg_dataL(f_wfrz ,ic,ip)*shr_const_latice
else if (trim(cime_model) == 'cesm') then
budg_dataL(f_wfrz ,ic,ip) = budg_dataL(f_hfrz ,ic,ip)*HFLXtoWFLX
end if
Expand Down

0 comments on commit 5fc342d

Please sign in to comment.