WIP: Bump Parthenon version to current develop and new B Cleanup w/GMG
#109
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 7 commits
July 23, 2024 15:33
* PtoU for face B field was zeroing dB on poles, not B, which was useless * Parthenon now has a def SMALL which conflicted with our macro. Their version is *very* small though, so left ours in place.
The existing magnetic field cleanup invoked when resizing a simulation uses a fork of Parthenon's old BiCGSTAB solver. This is fine, but doesn't support SMR/AMR grids. Parthenon's new multigrid solvers *do* support SMR grids, So I've added a magnetic field cleanup which uses them. As a bonus, they converge much faster and to seemingly arbitrary precision. The new cleanup does not support cell-centered B fields, so the old one has been left in for now. However, it doesn't work in this branch due to the old solver breaking, so it may as soon be eliminated as repaired, I'm not sure.
Stuff from testing GMG at scale: read enough zones to interpolate all face-centered fields, better handle boundaries/ghosts. GMG is slower than expected, probably due to single-level boundary conditions (which amount to a different boundary per MG level), but eventually converges to the divB < 1e-7 for 120M cells in 1/2hr on a CPU, which we'll call a win.
Move to the actual, official parthenon `develop`. We still carry three patches, two of which are submitted upstream. Not sure the third is even necessary anymore, but best to carry it until we can test everywhere.
Merged
added 22 commits
August 26, 2024 11:09
(And reconfigure the fix a bit to copy/allocate less)
…ers so anyone using them can restore them
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a testing branch for moving KHARMA back atop Parthenon's
developbranch. It seems to break a few minor things, and one major thing: the old BiCGSTAB solver KHARMA leverages for cleaning the magnetic field no longer works with new tasking.However, Parthenon now has much more powerful geometric multigrid (GMG) solvers, and I've written a new magnetic field cleanup based on them which is faster and seemingly more reliable, but which more importantly supports SMR/AMR grids as we need for resizing INCITE simulations. (Note it doesn't support cell-centered fields, and likely not Dirichlet boundaries either)
Currently this breaks old stuff and is maybe slow, and there's more modernization (type-based variables & packs) that I want to throw in before merging.
If you happen to need to resize a simulation onto an SMR grid, though, this will do it!