Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

+(*)Correct rotational symmetry with USE_LEITHY #1

Conversation

Hallberg-NOAA
Copy link

This commit revises the smoothing code used when USE_LEITHY = True to give answers that respect rotational symmetry and it also corrects some horizontal indexing bugs and problems with the staggering in some halo update and smooth_x9 calls and reduces some loop ranges to their minimal required values. The specific changes include:

  1. Corrected a horizontal indexing bug when interpolating Kh_h and Ah_h to corner (q) points when USE_LEITHY = True. These had previously been inappropriately copied from the thickness point to the southwest of the corner point. This required symmetric-memory-mode calculations of the thickness point viscosities whenever USE_LEITHY is true, but to avoid adding complicated logic, the symmetric-memory loop bounds are used for the calculation of Kh.

  2. Revised smooth_x9 to give rotationally symmetric answers. The previous code was retained behind the new nonsym_sums logical argument to smooth_x9 to generate the previous non-rotationally symmetric answers if they are actively being used. However, if they are not actively being used, all the unused code hidden behind the broken_symmetry=.true. branch in smooth_x9 should be eliminated.

  3. Added the new runtime argument LEITHY_NONSYMMETRIC_SUMS in cases when USE_LEITHY = True to determine whether to use the new rotationally symmetric smoothing code.

  4. Eliminated 4 unneeded halo update calls, and added error handling for the case where Leith options are used with insufficiently wide halos.

  5. Added a new 2-d variable for the squared viscosity for smoothing that can be used for halo updates and to avoid having a variable with confusingly inconsistent dimensions at various points in the code.

  6. Corrected the position arguments on 2 smooth_x9 calls and 4 pass_var calls that are used when USE_LEITHY=.true. and SMOOTH_AH=.true. As previously written, these smooth_x9 and pass_var calls would work when in non-symmetric memory mode but would give incorrect answers when in symmetric memory mode.

  7. Adjusted 13 do-loop extents in the USE_LEITHY=.true. code to reflect the loop ranges over which arrays are actually used, and another 78 loops now always work as though they are working with symmetric-memory.

These revisions can change answers when USE_LEITHY is true, but answers are bitwise identical in all other cases. There are new or corrected entries in some MOM_parameter_doc files.

@Hallberg-NOAA
Copy link
Author

I have updated this PR so that it does not do any extra calculations in non-symmetric memory mode, and to eliminate the code that gave the previous (clearly incorrect) answers when USE_LEITHY is True. If, however, we do want to retain those older answers, this could easily be revised to retain them (at least for some memory configuration settings). This revised version has been tested and demonstrated to reproduce across PE layout for both symmetric and non-symmetric memory mode.

  This commit revises the smoothing code used when USE_LEITHY = True to give
answers that respect rotational symmetry and it also corrects some horizontal
indexing bugs and problems with the staggering in some halo update and smooth_x9
calls and reduces some loop ranges to their minimal required values.  The
specific changes include:

  1. Corrected a horizontal indexing bug when interpolating Kh_h and Ah_h to
     corner (q) points when USE_LEITHY = True.  These had previously been
     inappropriately copied from the thickness point to the southwest of the
     corner point.  This required symmetric-memory-mode calculations of the
     thickness point viscosities whenever USE_LEITHY is true, but to avoid adding
     complicated logic, the symmetric-memory loop bounds are used for the
     calculation of Kh.

  2. Revised smooth_x9 to give rotationally symmetric answers and split it into
     the two routines smooth_x9_h and smooth_x9_uv to reduce the memory used by
     this routine and reduce the use of optional arguments.

  3. Eliminated 4 unneeded halo update calls, and added error handling for the
     case where Leith options are used with insufficiently wide halos.

  4. Added new integers to indicate the loop ranges over which the viscosities
     and related variables should be calculated, depending on which options are
     active, and then adjusted 91 do-loop extents horizontal_viscosity code to
     reflect the loop ranges over which arrays are actually used.

  5. Added a new 2-d variable for the squared viscosity for smoothing that can
     be used for halo updates and to avoid having a variable with confusingly
     inconsistent dimensions at various points in the code.

  6. Corrected the position arguments on 2 smooth_x9 calls and 4 pass_var calls
     that are used when USE_LEITHY=.true. and SMOOTH_AH=.true.  As previously
     written, these smooth_x9 and pass_var calls would work when in non-symmetric
     memory mode but would give incorrect answers when in symmetric memory mode.

  These revisions change answers when USE_LEITHY is true, but answers are
bitwise identical in all other cases.
@Hallberg-NOAA Hallberg-NOAA force-pushed the LeithE_rotational_symmetry branch from 51ecd65 to f38c8d5 Compare December 1, 2023 20:02
@iangrooms iangrooms merged commit 887e802 into iangrooms:fix_biharmonic_leith Dec 22, 2023
7 of 10 checks passed
@iangrooms
Copy link
Owner

I finally got a chance to look at and merge this PR. Thanks for putting it together!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants