bring in HOMME as subtree attempt 2 #362
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
git-svn-id: file:///Users/mataylo/repos/homme/branches/updateCprnc-4152@4160 b670e762-8b1b-0410-9e79-e6a6fc92c689 Former-commit-id: 0bcdceae08a051bc81377fb66020639c78d19cd6
git-svn-id: file:///Users/mataylo/repos/homme/branches/updateCprnc-4152@4163 b670e762-8b1b-0410-9e79-e6a6fc92c689 Former-commit-id: bc0c527779e6b91c59b2566b1fa2e21155973926
git-svn-id: file:///Users/mataylo/repos/homme/trunk@4164 b670e762-8b1b-0410-9e79-e6a6fc92c689 Former-commit-id: e0f25ef0f4a21b7c73572e72f9cb01fbba8b7381
git-svn-id: file:///Users/mataylo/repos/homme/branches/nextCAM@4166 b670e762-8b1b-0410-9e79-e6a6fc92c689 Former-commit-id: e7bc465202815f7fac1348f3431b9a7341023586
git-svn-id: file:///Users/mataylo/repos/homme/trunk@4167 b670e762-8b1b-0410-9e79-e6a6fc92c689 Former-commit-id: ad4624f4e3a34b4a500ac8c6db2266c3886477eb
… performance impact on multicore systems ... git-svn-id: file:///Users/mataylo/repos/homme/branches/nextCAM@4172 b670e762-8b1b-0410-9e79-e6a6fc92c689 Former-commit-id: b59164c425928fd49163289ba9629d7a17d5caf4
git-svn-id: file:///Users/mataylo/repos/homme/branches/nextCAM@4173 b670e762-8b1b-0410-9e79-e6a6fc92c689 Former-commit-id: dc3cdccaf95973a8b349c92996bd8130d67eabeb
git-svn-id: file:///Users/mataylo/repos/homme/branches/nextCAM@4174 b670e762-8b1b-0410-9e79-e6a6fc92c689 Former-commit-id: 24b72f0693544fe9c9d2f8b9a937491fc52544fb
git-svn-id: file:///Users/mataylo/repos/homme/branches/nextCAM@4175 b670e762-8b1b-0410-9e79-e6a6fc92c689 Former-commit-id: 103decc7a2f22dc3280badeee871d5ed67a68dae
git-svn-id: file:///Users/mataylo/repos/homme/branches/nextCAM@4179 b670e762-8b1b-0410-9e79-e6a6fc92c689 Former-commit-id: c677396e1f60890d7cf8171cd26182509e58fd13
…melist call yet) git-svn-id: file:///Users/mataylo/repos/homme/branches/nextCAM@4180 b670e762-8b1b-0410-9e79-e6a6fc92c689 Former-commit-id: 48dac37f253df83633a7ea5dbae0f411b46370c2
git-svn-id: file:///Users/mataylo/repos/homme/branches/nextCAM@4181 b670e762-8b1b-0410-9e79-e6a6fc92c689 Former-commit-id: 87ee9714817d186a8f6fe5f14b72aa9129abc945
git-svn-id: file:///Users/mataylo/repos/homme/branches/nextCAM@4182 b670e762-8b1b-0410-9e79-e6a6fc92c689 Former-commit-id: 41f2252363d460bdec43aa6ee0ae7f10645513cd
git-svn-id: file:///Users/mataylo/repos/homme/branches/nextCAM@4183 b670e762-8b1b-0410-9e79-e6a6fc92c689 Former-commit-id: d0c6830a99b7bc1bfa9aa5a34a5773d6b38775bc
git-svn-id: file:///Users/mataylo/repos/homme/branches/nextCAM@4184 b670e762-8b1b-0410-9e79-e6a6fc92c689 Former-commit-id: 5193ede23fe3eedde95d3ed276fbab349c5f06d8
git-svn-id: file:///Users/mataylo/repos/homme/branches/nextCAM@4185 b670e762-8b1b-0410-9e79-e6a6fc92c689 Former-commit-id: 622e58657e4c7e2e507741d50b8d2e2e4492dac3
git-svn-id: file:///Users/mataylo/repos/homme/branches/nextCAM@4186 b670e762-8b1b-0410-9e79-e6a6fc92c689 Former-commit-id: 4ffd60aa20c5724e3b5fcc2f11ebba39dbb347c4
…as these arrays were never used. git-svn-id: file:///Users/mataylo/repos/homme/trunk@4197 b670e762-8b1b-0410-9e79-e6a6fc92c689 Former-commit-id: c26926b86812ac1a3f8cf5802292a015ed7df1ea
git-svn-id: file:///Users/mataylo/repos/homme/branches/nextCAM@4199 b670e762-8b1b-0410-9e79-e6a6fc92c689 Former-commit-id: 231e4c56361478a7f5574c49a85f0d6704132b0b
…can be re-used for each additional tracer git-svn-id: file:///Users/mataylo/repos/homme/branches/nextCAM@4200 b670e762-8b1b-0410-9e79-e6a6fc92c689 Former-commit-id: 48efc1fb06e94b1ec2b2139745bd9a5476224427
git-svn-id: file:///Users/mataylo/repos/homme/trunk@4202 b670e762-8b1b-0410-9e79-e6a6fc92c689 Former-commit-id: e4c257bb9161f903783f905982d8f95f37c8b149
git-svn-id: file:///Users/mataylo/repos/homme/trunk@4203 b670e762-8b1b-0410-9e79-e6a6fc92c689 Former-commit-id: 9761dfd392fffcf525ac50fce7422cbf2c599eff
…m (divergence implied by fvm) git-svn-id: file:///Users/mataylo/repos/homme/branches/nextCAM@4205 b670e762-8b1b-0410-9e79-e6a6fc92c689 Former-commit-id: 158c18c7506e566732fa2a226a7008d6c77841b8
git-svn-id: file:///Users/mataylo/repos/homme/branches/nextCAM@4206 b670e762-8b1b-0410-9e79-e6a6fc92c689 Former-commit-id: 6133ca3b6015ffcf09690913d2920115d19617b1
git-svn-id: file:///Users/mataylo/repos/homme/branches/nextCAM@4207 b670e762-8b1b-0410-9e79-e6a6fc92c689 Former-commit-id: 262b132b2cf4f07e67686b632d7d24ea659a1d1f
git-svn-id: file:///Users/mataylo/repos/homme/branches/nextCAM@4208 b670e762-8b1b-0410-9e79-e6a6fc92c689 Former-commit-id: 652f919cb11d26826e73d486c46c8b0d3c09a272
git-svn-id: file:///Users/mataylo/repos/homme/branches/nextCAM@4209 b670e762-8b1b-0410-9e79-e6a6fc92c689 Former-commit-id: fe40e813308d18c16e5a1fef075ce1865525c23c
…adata to the *_latlon.nc template file git-svn-id: file:///Users/mataylo/repos/homme/trunk@4210 b670e762-8b1b-0410-9e79-e6a6fc92c689 Former-commit-id: e66db434a0adb973f8961a7ab814b0a012488be2
git-svn-id: file:///Users/mataylo/repos/homme/trunk@4211 b670e762-8b1b-0410-9e79-e6a6fc92c689 Former-commit-id: 231005974ece4966486132bddc37744becf4e37f
git-svn-id: file:///Users/mataylo/repos/homme/trunk@4221 b670e762-8b1b-0410-9e79-e6a6fc92c689 Former-commit-id: 4d4ac3759332cb50249168a59f39132429ce86ff
This was the last remaining difference between the CPU and GPU code in terms of implementation. Now, internal values are copied to the receive buffer, and edgeVunpack uses the receive buffer. This helps in two ways. First, there is a lower overall amount of data to transfer. Second, copying internal values has no dependencies. This gave a slight speed-up for GPU code.
Also got rid of the default Kieee flag for PGI (which turns off vectorization unnecessarily)
There were some areas where I forgot to put things in the "present" clause, and PGI was generating copies for those variables.
It turns out that those loops over acc device updates were killing me with overhead. So I'm now packing them, sending one update over PCI-e, and unpacking on the GPU.
|
This PR can't be both [non-BFB] and [CC]. If in doubt, say its [CC]. |
HOMME expects namelist file either on standard input or in "input.nl". On Mira this file is supplied by "-i input.nl" to qsub. Source contains cpp-macros to read the namelist on BlueGene/L. This updates the _BGL with a more generic __bg__ macro.
* XL's OMP runtime always returns false for omp_get_nested(). Protecting the call to it with cpp macro defined for BG arch. Also, this call returns a logical value, not integer. * Setting flag -qmaxmem=-1 to permit compiler optimizations to take as much memory as needed.
Nested OMP works on Mira with one level of nesting. Current source
contains two levels of nesting: 1 in prim_advection_mod.F90 and
1 in edge_mod.F90. This commit protects the second level of nesting
with a BG macro.
Also, GPTL timers cause run-time errors inside nested threading
regions. This is caused by either
1. non-unique nested thread id's, or
2. nested threads get rescheduled to other loop iterations prior to
completing the current iteration.
In either case, 'timer was already off' precedes a malloc segfault.
Commenting out the nested timers for now.
Mira doesn't like my non-OpenACC OpenACC code, so I'm if-defing it out.
Merge branch 'jgfouca/homme/bring_in_as_subtree_attempt_2' of github.com:ACME-Climate/ACME into jgfouca/homme/bring_in_as_subtree_attempt_2
* Fusing four small loops into one in edgeVunpack to avoid loop
and OMP pardo overheads.
* Reordering v accesses to prefer column-oriented array storage,
which stores south and north borders contiguously.
[BFB]
jgfouca
added a commit
that referenced
this pull request
Nov 16, 2015
…(PR #362) This is the third merge for this branch. This branch brings in the full HOMME dycore into ACME. The full HOMME model is included under models/homme, including the ability to run standalone HOMME. HOMME Is brought in as a subtree, with history going back to HOMME1_3_1. The dycore should be roundoff with the 1.5 year old version of HOMME currently in ACME (which is about 1.5 years out of date). But because of these roundoff changes, all comparison tests should fail. [Non-BFB] * jgfouca/homme/bring_in_as_subtree_attempt_2: (22 commits) Updating edgeVpack/edgeVunpack in HOMME Fixing XLF compile issues on Mira Remove a C comment from Fortran source file Build standalone HOMME on Mira Protect nested OMP on Mira IBM compiler updates Compiler flags for OpenMP on Mira Replace _BGL with __bg__ Updated to cuda7.0 on Titan Updating paths for CIME directories Improved PCI-e efficiency Getting rid of excess PCI-e copies Improved scripts for thatchroof profiling Adding scripts for thatchroof machine edgeVunpack using receive buffer now OACC now compatable with new HOMME Update to compile paths Compilation files for HOMME titan openacc PGI 15.10 bug fixes,undoing edge_mod workarounds OpenACC now machine precision same w/ CPU! ...
|
We need to freeze updates to this branch to get it merged into next (today) and master in the next 1-2 days. |
|
for the record: it is non-BFB and non-CC |
jgfouca
added a commit
that referenced
this pull request
Nov 16, 2015
…r (PR #362) This branch brings in the full HOMME dycore into ACME. The full HOMME model is included under models/homme, including the ability to run standalone HOMME. HOMME Is brought in as a subtree, with history going back to HOMME1_3_1. The dycore should be roundoff with the 1.5 year old version of HOMME currently in ACME (which is about 1.5 years out of date). But because of these roundoff changes, all comparison tests should fail. [Non-BFB] * jgfouca/homme/bring_in_as_subtree_attempt_2: (22 commits) Updating edgeVpack/edgeVunpack in HOMME Fixing XLF compile issues on Mira Remove a C comment from Fortran source file Build standalone HOMME on Mira Protect nested OMP on Mira IBM compiler updates Compiler flags for OpenMP on Mira Replace _BGL with __bg__ Updated to cuda7.0 on Titan Updating paths for CIME directories Improved PCI-e efficiency Getting rid of excess PCI-e copies Improved scripts for thatchroof profiling Adding scripts for thatchroof machine edgeVunpack using receive buffer now OACC now compatable with new HOMME Update to compile paths Compilation files for HOMME titan openacc PGI 15.10 bug fixes,undoing edge_mod workarounds OpenACC now machine precision same w/ CPU! ...
rljacob
added a commit
that referenced
this pull request
Aug 11, 2016
12d2135 Merge pull request #388 from ESMCI/jgfouca/need_to_report_build_exceptions 8f677cd Add test to ensure build fails report info to teststatus.log 7095ef0 Need to report build exception contents 7c9cc94 Merge pull request #387 from ESMCI/jgfouca/fix_case_build_return_code bf941ed case.build needs to check success in order to return a sane error code df432e8 update ChangeLog 154d5f8 Merge pull request #378 from ESMCI/rljacob/update-config-files 6b8fc76 Merge pull request #382 from ESMCI/sarich/fix-taskmaker-counter 5df46a2 Merge pull request #381 from jedwards4b/test_fixes e58d624 component_compare_test was not properly reporting failures 0e0e577 Update acme config_files for mpas 84122dd Merge pull request #376 from ESMCI/jgfouca/changes_from_acme 38e2f8a More stuff from ACME c86398e Merge pull request #375 from ESMCI/jgfouca/portable_run_cmd_utest daaf621 Merge pull request #373 from ESMCI/jgfouca/enhance_bisect e61ba96 Change MPAS compset for test c77a64e Add homme python test 3031f00 Better support for 'none' module system 3ceef7b Make run_cmd_no_fail unit test more portable bbd20fb Merge branch 'jgfouca/fix_module_list' (PR #374) 6eb8143 Reactivate creation of software_environment.txt 4dd30c6 Ensure module setup is sourced before list c8bd20e fix bug in translation from perl 20a3412 cime_bisect: Add better support for modifying create_test run 77871de update changelog e74906b comment out code until it works for tcsh users 4a13413 fix issue with module list 6ad4b2d update changelog cef688d update changelog 835b511 Merge pull request #367 from jedwards4b/user_mod_0len_fix b280b55 Merge pull request #362 from ESMCI/jgfouca/remove_perl_taskmaker 12a30ee Merge pull request #356 from ESMCI/jgfouca/minor_timing_chg 3fb0b80 Merge pull request #355 from ESMCI/jgfouca/wait_for_test_refactor d083933 user_nl_ file was being removed if a user_nl file in any mods directory was missing 39de940 Fix comment 5d38420 Revert "Merge pull request #343 from ESMCI/wilke/scripts/xmlchange" 8dc2354 Merge pull request #363 from ESMCI/rljacob/machines/fix-acme 0773aac Increase default walltime for blues c4dce0f Remove last uses of taskmaker.pl 76eb1bc Remove -A directive from edison d67b267 Merge pull request #361 from billsacks/cism_nag bf02e3e Merge pull request #357 from ekluzek/fixpionml c9b8910 Seperate out modelio namelist definition since it uses the same names, but defines them differently 805ad7f Add -mismatch_all when compiling cism with nag fff9a9b Set CHECK_TIMING to true in addition to SAVE_TIMING if --save-timing given to create_test f25a518 wait_for_tests will now always specifically wait for the RUN phase e19e72c Update drv buildnamelist test to work with cime5 db1538e Merge branch 'douglasjacobsen/add_lanl_machines' (PR #353) 893c6c6 Add support for LANL's mustang and wolf to cime dea8a3a Merge pull request #350 from ESMCI/rerun_test_functionality 9b4488b Changes based on github feedback c103e08 Add SAVE_TIMING_DIR for edison 98f95bb Merge pull request #341 from ESMCI/santos/fix-env-leakage 1b775e4 Fixes post-upstream-merge ea97b56 Merge branch 'master' into rerun_test_functionality 1ca1b83 Merge pull request #348 from ESMCI/jayeshkrishna/machinefiles/get_acme_cime_dev_working_on_mira 252aea7 Merge pull request #343 from ESMCI/wilke/scripts/xmlchange 2db894f Add missing files 622b7d0 Complete ba69385 progress 84000ee Fixing the runjob command for ACME on Mira 1a55232 Adding config for ERS_Ld3.ne30_g16_rx1.A test d1df346 Error handling; check for correct length of key-value pair array after split fa1cb49 listofsettings allways an array, test for length of array a6a3d33 Changed number of expected positional arguments to 0 or 1 , warnings and debug statements 74d2100 checking for missing values in settings string from command line cd350da Remove `GenericXML` check for env variables. 6ca6b59 progress e7b334e Merge pull request #340 from ESMCI/douglasjacobsen/fix_test_template 43807a5 Add white space after batch directives in script templates e061505 Merge pull request #337 from ESMCI/jgfouca/autosave_env_info edc1671 Autosave environment information in case_setup. 33ce89b Merge pull request #336 from ESMCI/jgfouca/fix_create_test_not_catching_missing_project c3f8f84 create_test was not failing the create_newcase phase when project info was missing fef81df Merge pull request #335 from ESMCI/jgfouca/add_queue_option_to_create_test 731f8a0 Add ability to select queue to create_test and create_newcase 6f9613f Merge pull request #333 from ESMCI/jgfouca/even_more_sky_env_fixes 11877a2 Fix mismatch between MPI_PATH and the mpi module being loaded d37e177 Merge pull request #322 from ESMCI/jgfouca/restore_good_python_version_error ad18c34 Merge pull request #331 from ESMCI/jgfouca/reduce_output_from_check_input_data 0268cb7 Only report present files in debug mode 47b4216 Fix spelling mistake dddd7f0 Merge pull request #327 from ESMCI/jgfouca/fix_more_sky_env_issues 07aeb52 Get cime_developer building again on skybridge. 0aceb94 Merge branch 'wilke/template/directives' (PR #324) d8f331c moved batchdirectives to top of the template dbbd3a1 Users should get a nice error when their python is too old 5eea798 Merge pull request #320 from ESMCI/jgfouca/fix_skybridge_env_issues ca3e004 Fix skybridge environment problems, port to new SEMS modules 1a82a57 Merge pull request #318 from ESMCI/jgfouca/remove_sentinel_concept e266dd9 Remove sentinel concept from jenkins_generic_job aae8e30 comments for cime5.0.5 e20f807 Merge pull request #317 from billsacks/restore_lii 1967bb3 Restore LII test c404dfa Merge remote-tracking branch 'origin/master' 0406b7d updates for cime5.0.4 7930cd7 Merge pull request #316 from ESMCI/jgfouca/update_code_checker 009f7a1 code_checker: Leverage .gitignore by using git ls-files instead of find to get list of ifles to check c5555c7 Merge pull request #315 from Katetc/master c4817d7 fix issue 314 7a493a0 remove multiple run lines from test file c269ab9 Merge pull request #313 from ESMCI/jgfouca/correctly_report_problem_in_test 734e4b4 Rolling the Intel compiler back to v15.0.2 9a46a99 On batch systems, be sure to report that the problem is with wait_for_tests, not create_test 0e95d19 add some more info to README file e9586c1 Changes required to support the new Hobart cluster configuration cdb7805 update documentation for --xml options 98a0380 Merge pull request #307 from ESMCI/rljacob/tests/add-readme 46ff3c6 work on test rerunability c5f2ae9 Merge pull request #308 from ESMCI/jgfouca/fix_check_input_bug bddea72 document ERR test 8639ba1 create ability to run tests in same case more than once b18580b Fix minor erroneous output bug in check_input_data 6f3e448 Add README back to Testcases fd001a7 Add README back to SystemTests 929f05a Merge pull request #305 from ESMCI/jgfouca/advanced_profiling_tool 20c2e22 Make prof tools a bit more user friendly 1f9b49b Merge pull request #304 from jedwards4b/dynamic_system_test_dirs 0ba7f50 handle so that we dont have a list of test names to maintain 92a266e handle so that we dont have a list of test names to maintain 99f3d35 New tool 'advanced-py-prof' 4e6f7f5 initialize contents 3703d94 machine specific fixes for edison/cori/slurm systems f6dc40a fixes git issue 303 7bacf48 repeat change for acme 503d5ad load system test directories dynamically based on paths in config_files.xml b28ddff Merge pull request #302 from ESMCI/jgfouca/profiling_tool_etc 0e1553f Add a new tool for very simple python profiling 1eb2a56 Merge pull request #301 from jedwards4b/shell_commands_delete 2f78ec7 remove any existing shell_commands files from case before writing new ones 5f85c05 update changelog for tag 4ef656d pylint fixes 8624ece fix needed for scripts_regression_tests following PR298 6cc9110 These were supposed to be in PR296 d19afa9 Merge remote-tracking branch 'jedwards/testing_fixes' (PR #296) b302243 Merge pull request #298 from ESMCI/jgfouca/restore_verbose 0a95b04 component_compare_test should fail if one of the components to be compared is not found" 9aa8683 Reintroduce verbose option into the refactored logging system a796384 fix for lii test and response to review d8c9b2e Merge pull request #286 from jedwards4b/buildnml_output_fix 1894e28 improve documetation of debug option, remove incorrect documentation of verbose option 21ef9ae remove whitespace surrounding test names f6fdbb7 fix erp test update ChangeLog b8b4723 Fix LII test daa0c63 rename bisect unit test from acme to cime 4f1a079 move pecount code from create_test to create_newcase 24eb48f move clm include directory to prevent build confusion 1ee4449 add support for ascii testfile, allow multiple compilers in tests d6c28b4 fix memleak test giving error if baseline not found 8ff5f19 Merge pull request #287 from ESMCI/nag_mismatch a57e410 Remove `-mismatch_all` from NAG options in CESM. 5040500 output from buildnml scripts now prints git-subtree-dir: cime git-subtree-split: 12d2135
jgfouca
added a commit
that referenced
this pull request
Feb 27, 2018
…r (PR #362) This branch brings in the full HOMME dycore into ACME. The full HOMME model is included under models/homme, including the ability to run standalone HOMME. HOMME Is brought in as a subtree, with history going back to HOMME1_3_1. The dycore should be roundoff with the 1.5 year old version of HOMME currently in ACME (which is about 1.5 years out of date). But because of these roundoff changes, all comparison tests should fail. [Non-BFB] * jgfouca/homme/bring_in_as_subtree_attempt_2: (22 commits) Updating edgeVpack/edgeVunpack in HOMME Fixing XLF compile issues on Mira Remove a C comment from Fortran source file Build standalone HOMME on Mira Protect nested OMP on Mira IBM compiler updates Compiler flags for OpenMP on Mira Replace _BGL with __bg__ Updated to cuda7.0 on Titan Updating paths for CIME directories Improved PCI-e efficiency Getting rid of excess PCI-e copies Improved scripts for thatchroof profiling Adding scripts for thatchroof machine edgeVunpack using receive buffer now OACC now compatable with new HOMME Update to compile paths Compilation files for HOMME titan openacc PGI 15.10 bug fixes,undoing edge_mod workarounds OpenACC now machine precision same w/ CPU! ...
yunpengshan2014
pushed a commit
that referenced
this pull request
Apr 2, 2024
* update version * add yml for osx * modify yml files * clean up
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This branch brings in the full HOMME dycore into ACME. The full HOMME model is included under models/homme, including the ability to run standalone HOMME. HOMME Is brought in as a subtree, with history going back to HOMME1_3_1. The dycore should be roundoff with the 1.5 year old version of HOMME currently in ACME (which is about 1.5 years out of date). But because of these roundoff changes, all comparison tests should fail.
[Non-BFB]