Update MPAS-O for halo exchange reuse #2041
Conversation
|
Thanks to @Larofeticus for continued work on MPAS performance, and @ndkeen for collaboration and testing within E3SM. |
|
The added MPAS framework subroutines are already in E3SM, from this MPAS PR: In MPAS-Ocean stand alone, this passed all tests in nightly regression suite. BFB with previous commit using both gnu/opt and intel/debug and between 1 and 2 threads. Tested in E3SM with: |
…2041) Update MPAS-O for halo exchange reuse This PR speeds up the ocean model by storing and reusing halo exchange data structures, rather than recompute them on every halo exchange. This is only applied to the barotropic subcycle in this PR, which has the largest number of halo exchange calls. Tests have shown an ocean speed-up of 8-12% on cori-knl with threading, but it should benefit all architectures and thread configurations. Tested with: * PEM_Ln9.T62_oQU240.GMPAS-IAF.cori-knl_intel * SMS_D.T62_oQU120_ais20.MPAS_LISIO_TEST.theta_intel * SMS_D.T62_oQU120_ais20.MPAS_LISIO_TEST.cori-knl_intel [BFB]
|
merged to next |
|
Off-line testing by @pwolfram showed a severe memory leak with this version. I ran a test of GMPAS-IAF.T62_oECv3 on anvil and it failed with memory issues after a month and a half -- so this need to be reverted until fixed. |
|
Jon: I was also noticing that we had a few failures on next related to memory leak. I was hoping to ignore them, but apparently not! FYI: edison and cori-haswell both see the following tests fails due to memory leak: and then cori-knl sees the same failure with |
|
Reverted MPAS-O commit due to insufficient memory error. See MPAS-Dev/MPAS#1490 |
|
@ndkeen - thanks for the note. It does add some confidence that the automated testing does catch egregious memory leaks like this |
New gx3v7_coastal grid file Updated the namelists for `runoff_to_ocn` to use a new grid file for the coastal mask of the gx3v7 grid (old grid file had different area than full gx3v7 grid file); after generating new grid files, I also updated the cesm defaults to use the new maps instead of the old / broken ones. Test suite: Ran a single C compset using the `rx1 -> gx3v7` maps, verified that answers changed slightly (have not tested the `r05 -> gx3v7` map) Test baseline: Same case as above, but run off `master` instead of my branch Test namelist changes: `ROF2OCN_LIQ_RMAPNAME`, `ROF2OCN_ICE_RMAPNAME`, `GLC2OCN_LIQ_RMAPNAME` and `GLC2OCN_ICE_RMAPNAME` change when running CESM with `gx3v7` ocean grid Test status: roundoff Fixes #2041 User interface changes?: nope Update gh-pages html (Y/N)?: nope Code review: none
Improve qv_sat to allow more caller info
This PR speeds up the ocean model by storing and reusing halo exchange data structures, rather than recompute them on every halo exchange. This is only applied to the barotropic subcycle in this PR, which has the largest number of halo exchange calls. Tests have shown an ocean speed-up of 8-12% on cori-knl with threading, but it should benefit all architectures and thread configurations.
[BFB]