Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

create_test problems on anvil #1424

Closed
jonbob opened this issue Apr 19, 2017 · 5 comments · Fixed by #1788
Closed

create_test problems on anvil #1424

jonbob opened this issue Apr 19, 2017 · 5 comments · Fixed by #1788
Assignees
@jgfouca
Labels
bug CIME Fixed in ESMCI Machine Files

Comments

@jonbob
Copy link
Contributor

jonbob commented Apr 19, 2017

On current cime5.3 master (v1.0.0-beta.1-7503-g46f6a17), create_test fails on anvil in the following way:

./create_test ERS.T62_oEC60to30v3.GMPAS-IAF.anvil_intel
ERROR: Mismatch in field machine, partial string 'ERS.T62_oEC60to30v3.GMPAS-IAF.anvil_intel' indicated it should be 'anvil' but you provided 'blues'

I get similar problems using "anvil_gnu", and leaving the compiler off the test definition says:

ERROR: Expected 4th item of 'ERS.T62_oEC60to30v3.GMPAS-IAF.anvil' ('anvil') to be in form machine_compiler
@amametjanov
Copy link
Member

A local workaround is to change login node name's regex so that it doesn't match: e.g.

 <machine MACH="blues">
          <DESC>ANL/LCRC Linux Cluster</DESC>
-         <NODENAME_REGEX>b.*.lcrc.anl.gov</NODENAME_REGEX>
+         <NODENAME_REGEX>d.*.lcrc.anl.gov</NODENAME_REGEX>
          <TESTS>acme_integration</TESTS>
          <COMPILERS>gnu,pgi,intel,nag</COMPILERS>
          <MPILIBS>mvapich,mpich,openmpi,mpi-serial</MPILIBS>

@jayeshkrishna
Copy link
Contributor

jayeshkrishna commented Apr 19, 2017 •
edited
Loading

Or use
./create_test ERS.T62_oEC60to30v3.GMPAS-IAF --machine=anvil --compiler=gnu
for now

@jonbob
Copy link
Contributor Author

jonbob commented Apr 19, 2017 •
edited
Loading

Thanks @jayeshkrishna - I'll let you know if that works.

Update -- yes, it's happy that way

@rljacob
Copy link
Member

rljacob commented Apr 25, 2017

There is still a bug here. We'll track it here: ESMCI/cime#1417

@rljacob rljacob closed this as completed Apr 25, 2017
agsalin pushed a commit that referenced this issue May 1, 2017
@jedwards4b
Merge pull request #1428 from billsacks/move_unit_tests
9c5b1aa 
Move unit tests to scripts/fortran_unit_testing
Moves unit testing infrastructure from tools/unit_testing to
scripts/fortran_unit_testing.

Rationale: tools are supposed to be optional tools, whereas the Fortran
unit tests are a more required part of the testing workflow.

Also cleans up python in fortran_unit_testing directory:

Remove dead code (which has been replaced by use of the standard CIME
utilities)

Clean up pylint problems

Also updates some documentation

Test suite: scripts_regression_tests on yellowstone
Test baseline: n/a
Test namelist changes: n/a
Test status: bit for bit

Fixes #1423
Fixes #1424

User interface changes?: Moves run_tests.py to scripts/fortran_unit_testing

Code review: jedwards
@rljacob
Copy link
Member

rljacob commented Aug 2, 2017

Reopening as its a bug not fixed in ACME.

@rljacob rljacob reopened this Aug 2, 2017
@jgfouca jgfouca mentioned this issue Aug 2, 2017
Closed
@jgfouca jgfouca mentioned this issue Sep 15, 2017
Merged
jgfouca added a commit that referenced this issue Oct 3, 2017
@jgfouca
Merge branch 'jgfouca/update_to_cime_5.3.0.34' into next (PR #1788)
ff78504 
New user interface features:
option --skip-preview-namelist to case.submit to skip calling preview_namelist during case.run
option -M MAIL_USER, --mail-user MAIL_USER to case.submit to set an email for batch system notification
option -m {never,all,begin,end,fail}, --mail-type {never,all,begin,end,fail} to case.submit for when the batch system should send email.
manage_case renamed to more accurate query_config and usage/arguments made easier and output more informative.
Add --pesfile option to create_test
New case.qstatus tool
Allow user to adjust ulimits
Add message at end of successful build
Add new user-mods option to create_clone
New ways to select queue in config_batch.xml
scripts_regression_tests.py now has --compiler and --mpilibs option
new case variables controlling spare nodes (for failing nodes workaround)

User interface bug fixes:
xmlquery options --file and --listall now work together.
xmlquery options clarified and help message improved.
Improve error message for passing incorrect grid alias to create_newcase.
Improve error messages from xmlchange

Removed user features:
Remove long term archiving support. "case.lt_archive" no longer in caseroot.

Other changes to case control system:
Improve ERR test.
New ERS2 test
Add IRT test
Implement NCR with SystemTestsCompareTwo
Rewrite ERP with SystemTestsCompareTwo
Convert the ERR and ERRI test to the compare2 framework
Fix SEQ test on skybridge
new case.test --reset option to manually reset a test to initial conditions.
TOTAL_CORES was inaccurate so remove it.
Only create rest directories if they are going to be populated
get_timing.py should use largest ncpl value, not assume it's the atm value
Improve comment for BASELINE status in TestStatus
Better handling of fails in the build (don't print a python stacktrace)
Add a new directory testmod that creates cpl history files on a daily basis and thereby permits answer changes to be compared more easily.
Add ability for create_test to infer non-default machine from testname
Change all python2 style strings to python3 style strings
Improve handling of hist comparisons in TestStatus.log
version 3.0 schema for entry_id files
Downscale build parallelism on small machines (so we don't overload it)
Bring bless_test_results behavior into an exact match with create_test -g
lcdf4 flag does apply to pnetcdf.
Replace cdf64 logical with PIO_NETCDF_FORMAT character variable
Refactor how spare nodes get computed
Preserve test order in test_scheduler
Allow queue selection by node count instead of task count
Improve error check for non-python sub processes

Data model changes:
Refactored data models
Use %CPLHIST for DATM coupler hist forcing
Add support for data model to read multiple time slices at once
New aquaplanet capability: adds aquaplanet capability to DOCN where the input is from a file rather than analytic; adds new A compsets to the driver config_compsets.xml that have different DOCN functionality: ADSOM - DOCN SOM, ADSOMAQP - DOCN aquaplanet SOM, ADAQP3 - DOCN analytic aquaplanet (mode 3), ADAQFILE - DOCN aquaplanet from file
DATM updates for clm
Change SIAF compset name in dice to IAF to be consistent with other components

Coupler/driver changes:
Map SMB from lnd to glc using bilinear mapping with a conservation correction
Remove more cesm refs in filenames.
seq_comm_mct code cleanup
Implementation of multiple couplers within multi-instance runs.

Other:
User's guide rst files now in cime/doc directory
Get PFUnit working on melvin
Homebrew generic mac port

Fixes #1255
Fixes #1424
Fixes #1426
Fixes #1573
Fixes #1576
Fixes #1585

* jgfouca/update_to_cime_5.3.0.34: (932 commits)
  One more short term archiver fix
  Fix merge conflicts, should mostly work
  cetus xml fix
  Hack to get PET test to work on chama
  Fix HOMME Cmake pointing to wrong cime area
  Update run_acme version
  Fixes for AQUAP
  Removes long term archiving from run_acme
  Move xml compiler and mpilib tags from env and command elements This is required for the new xml schema requirements; compiler and mpilib tags should be in the modules and environment_variables elements rather than their members
  Cherry-pick fixes from older merge branch
  cime_developer now working on melvin
  new git pelayout doc
  Update ChangeLog
  Change atm_grid "48x96" to T31 to match the model_grid_alias that is actually being used.
  Minor fix
  Fix mistake
  Better encapsulation of common calls (create_test, wait_for_tests)
  Add a timeout option to wait_for_tests
  update to sphinx documentation
  Restore task_count
  ...
jgfouca added a commit that referenced this issue Oct 11, 2017
@jgfouca
Merge branch 'jgfouca/update_to_cime_5.3.0.34' into next (PR #1788)
48f54b8 
Update CIME to 5.3.0alpha34

New user interface features:
option --skip-preview-namelist to case.submit to skip calling preview_namelist during case.run
option -M MAIL_USER, --mail-user MAIL_USER to case.submit to set an email for batch system notification
option -m {never,all,begin,end,fail}, --mail-type {never,all,begin,end,fail} to case.submit for when the batch system should send email.
manage_case renamed to more accurate query_config and usage/arguments made easier and output more informative.
Add --pesfile option to create_test
New case.qstatus tool
Allow user to adjust ulimits
Add message at end of successful build
Add new user-mods option to create_clone
New ways to select queue in config_batch.xml
scripts_regression_tests.py now has --compiler and --mpilibs option
new case variables controlling spare nodes (for failing nodes workaround)

User interface bug fixes:
xmlquery options --file and --listall now work together.
xmlquery options clarified and help message improved.
Improve error message for passing incorrect grid alias to create_newcase.
Improve error messages from xmlchange

Removed user features:
Remove long term archiving support. "case.lt_archive" no longer in caseroot.

Other changes to case control system:
Improve ERR test.
New ERS2 test
Add IRT test
Implement NCR with SystemTestsCompareTwo
Rewrite ERP with SystemTestsCompareTwo
Convert the ERR and ERRI test to the compare2 framework
Fix SEQ test on skybridge
new case.test --reset option to manually reset a test to initial conditions.
TOTAL_CORES was inaccurate so remove it.
Only create rest directories if they are going to be populated
get_timing.py should use largest ncpl value, not assume it's the atm value
Improve comment for BASELINE status in TestStatus
Better handling of fails in the build (don't print a python stacktrace)
Add a new directory testmod that creates cpl history files on a daily basis and thereby permits answer changes to be compared more easily.
Add ability for create_test to infer non-default machine from testname
Change all python2 style strings to python3 style strings
Improve handling of hist comparisons in TestStatus.log
version 3.0 schema for entry_id files
Downscale build parallelism on small machines (so we don't overload it)
Bring bless_test_results behavior into an exact match with create_test -g
lcdf4 flag does apply to pnetcdf.
Replace cdf64 logical with PIO_NETCDF_FORMAT character variable
Refactor how spare nodes get computed
Preserve test order in test_scheduler
Allow queue selection by node count instead of task count
Improve error check for non-python sub processes

Data model changes:
Refactored data models
Use %CPLHIST for DATM coupler hist forcing
Add support for data model to read multiple time slices at once
New aquaplanet capability: adds aquaplanet capability to DOCN where the input is from a file rather than analytic; adds new A compsets to the driver config_compsets.xml that have different DOCN functionality: ADSOM - DOCN SOM, ADSOMAQP - DOCN aquaplanet SOM, ADAQP3 - DOCN analytic aquaplanet (mode 3), ADAQFILE - DOCN aquaplanet from file
DATM updates for clm
Change SIAF compset name in dice to IAF to be consistent with other components

Coupler/driver changes:
Map SMB from lnd to glc using bilinear mapping with a conservation correction
Remove more cesm refs in filenames.
seq_comm_mct code cleanup
Implementation of multiple couplers within multi-instance runs.

Other:
User's guide rst files now in cime/doc directory
Get PFUnit working on melvin
Homebrew generic mac port

Fixes #1255
Fixes #1424
Fixes #1426
Fixes #1573
Fixes #1576
Fixes #1585

* jgfouca/update_to_cime_5.3.0.34: (942 commits)
  Apply patch for cpl to avoid hangs
  Adjust ne4 PE layout on Anvil
  Reset configpes_all to what's on master
  Cherry-pick wait_for_tests fix from ESMCI
  Fix xmllint errors on Titan
  Convert attribute DEBUG to uppercase
  Update ne4-wcycl PEs on Anvil
  Fix the rest of the pylint errors for case_st_archive
  Fix jenkins script mistake made during conflict resolution.
  short term archiver get_datenames call for last restarts fixed
  One more short term archiver fix
  Fix merge conflicts, should mostly work
  cetus xml fix
  Hack to get PET test to work on chama
  Fix HOMME Cmake pointing to wrong cime area
  Update run_acme version
  Fixes for AQUAP
  Removes long term archiving from run_acme
  Move xml compiler and mpilib tags from env and command elements This is required for the new xml schema requirements; compiler and mpilib tags should be in the modules and environment_variables elements rather than their members
  Cherry-pick fixes from older merge branch
  ...
jgfouca added a commit that referenced this issue Oct 12, 2017
@jgfouca
Merge branch 'jgfouca/update_to_cime_5.3.0.34' into master (PR #1788)
2a095e6 
Update CIME to 5.3.0alpha34

New user interface features:
option --skip-preview-namelist to case.submit to skip calling preview_namelist during case.run
option -M MAIL_USER, --mail-user MAIL_USER to case.submit to set an email for batch system notification
option -m {never,all,begin,end,fail}, --mail-type {never,all,begin,end,fail} to case.submit for when the batch system should send email.
manage_case renamed to more accurate query_config and usage/arguments made easier and output more informative.
Add --pesfile option to create_test
New case.qstatus tool
Allow user to adjust ulimits
Add message at end of successful build
Add new user-mods option to create_clone
New ways to select queue in config_batch.xml
scripts_regression_tests.py now has --compiler and --mpilibs option
new case variables controlling spare nodes (for failing nodes workaround)

User interface bug fixes:
xmlquery options --file and --listall now work together.
xmlquery options clarified and help message improved.
Improve error message for passing incorrect grid alias to create_newcase.
Improve error messages from xmlchange

Removed user features:
Remove long term archiving support. "case.lt_archive" no longer in caseroot.

Other changes to case control system:
Improve ERR test.
New ERS2 test
Add IRT test
Implement NCR with SystemTestsCompareTwo
Rewrite ERP with SystemTestsCompareTwo
Convert the ERR and ERRI test to the compare2 framework
Fix SEQ test on skybridge
new case.test --reset option to manually reset a test to initial conditions.
TOTAL_CORES was inaccurate so remove it.
Only create rest directories if they are going to be populated
get_timing.py should use largest ncpl value, not assume it's the atm value
Improve comment for BASELINE status in TestStatus
Better handling of fails in the build (don't print a python stacktrace)
Add a new directory testmod that creates cpl history files on a daily basis and thereby permits answer changes to be compared more easily.
Add ability for create_test to infer non-default machine from testname
Change all python2 style strings to python3 style strings
Improve handling of hist comparisons in TestStatus.log
version 3.0 schema for entry_id files
Downscale build parallelism on small machines (so we don't overload it)
Bring bless_test_results behavior into an exact match with create_test -g
lcdf4 flag does apply to pnetcdf.
Replace cdf64 logical with PIO_NETCDF_FORMAT character variable
Refactor how spare nodes get computed
Preserve test order in test_scheduler
Allow queue selection by node count instead of task count
Improve error check for non-python sub processes

Data model changes:
Refactored data models
Use %CPLHIST for DATM coupler hist forcing
Add support for data model to read multiple time slices at once
New aquaplanet capability: adds aquaplanet capability to DOCN where the input is from a file rather than analytic; adds new A compsets to the driver config_compsets.xml that have different DOCN functionality: ADSOM - DOCN SOM, ADSOMAQP - DOCN aquaplanet SOM, ADAQP3 - DOCN analytic aquaplanet (mode 3), ADAQFILE - DOCN aquaplanet from file
DATM updates for clm
Change SIAF compset name in dice to IAF to be consistent with other components

Coupler/driver changes:
Map SMB from lnd to glc using bilinear mapping with a conservation correction
Remove more cesm refs in filenames.
seq_comm_mct code cleanup
Implementation of multiple couplers within multi-instance runs.

Other:
User's guide rst files now in cime/doc directory
Get PFUnit working on melvin
Homebrew generic mac port

Fixes #1255
Fixes #1424
Fixes #1426
Fixes #1573
Fixes #1576
Fixes #1585

* jgfouca/update_to_cime_5.3.0.34: (942 commits)
  Apply patch for cpl to avoid hangs
  Adjust ne4 PE layout on Anvil
  Reset configpes_all to what's on master
  Cherry-pick wait_for_tests fix from ESMCI
  Fix xmllint errors on Titan
  Convert attribute DEBUG to uppercase
  Update ne4-wcycl PEs on Anvil
  Fix the rest of the pylint errors for case_st_archive
  Fix jenkins script mistake made during conflict resolution.
  short term archiver get_datenames call for last restarts fixed
  One more short term archiver fix
  Fix merge conflicts, should mostly work
  cetus xml fix
  Hack to get PET test to work on chama
  Fix HOMME Cmake pointing to wrong cime area
  Update run_acme version
  Fixes for AQUAP
  Removes long term archiving from run_acme
  Move xml compiler and mpilib tags from env and command elements This is required for the new xml schema requirements; compiler and mpilib tags should be in the modules and environment_variables elements rather than their members
  Cherry-pick fixes from older merge branch
  ...
mark-petersen pushed a commit to mark-petersen/E3SM that referenced this issue Jan 19, 2021
@matthewhoffman
Merge branch 'develop' into landice/develop
65b5f28 
Update needed for ACME

Framework features brought in:

* f444d0f Merge PR E3SM-Project#1418 'matthewhoffman/framework/output_record_reference_time' into develop
* 263e14f Merge PR E3SM-Project#1428 'mark-petersen/framework/couple_fixes' into develop
* bcce31d Merge PR E3SM-Project#1424 'amametjanov:az/tools/cp-prebuilt-tools' into develop
* 98cfeea Merge PR# 1349 'akturner/framework/forcing_cleanup' into develop
* 9359319 Merge PR E3SM-Project#1347 'akturner/framework/forcing_restart_timestamp' into develop
* e9ce203 Merge PR E3SM-Project#1348 'akturner/framework/forcing_at_init' into develop
* 4974284 Merge PR E3SM-Project#1368 'akturner/framework/improved_messages_in_driver' into develop
* 86d50c5 Merge PR E3SM-Project#1417 'akturner/framework/forcing_multiple_blocks' into develop
* 9116da3 Merge branch 'framework/validation-of-streams-using-interval_in-interval_out' into develop
* e466b46 Merge branch 'framework/interval_in-interval_out-support-for-streams' into develop
* 30dc955 Merge branch 'az/framework/mpas_dmpar-race-fix' into develop
* b632938 Merge branch 'framework/i8_interval_division' into develop
* 6dac06c Merge branch 'framework/log_write_IBM_error' into develop
* 960a648 Merge branch 'framework/cleanup-logging-stream-manager' into develop
* 504c282 Merge branch 'framework/make-streams-with-direction-none-inactive' into develop
* 5903748 Merge branch 'framework/correctly_remove_blk_fields' into develop
* 3565965 Merge branch 'framework/iostreams-real4dfield-bug' into develop
* 8b60591 Merge branch 'framework/missing-deallocate-nEdgesOnCellField-bootstrapping' into develop
* 70b953b Merge branch 'master' into develop
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jgfouca jgfouca

Labels
bug CIME Fixed in ESMCI Machine Files
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Update CIME to 5.3.0alpha34 E3SM-Project/E3SM
Update CIME to 5.3.0alpha29 E3SM-Project/E3SM
5 participants
@rljacob @amametjanov @jgfouca @jayeshkrishna @jonbob