Enable customizable idealized, doubly-periodic tests

[LarissaReames-NOAA]

Description

This PR adds a doubly-periodic idealized test case to test_cases.F90 that is customizable by the user at run time via input.nml and that can be run within the UFS framework and is fully compatible with CCPP physics.

An important modification was made to make this case compatible with CCPP. To enable the user to choose the canned "SupeCell" sounding from GFDL's original supercell test, we copied over the routines from GFDL MP that were previously preventing this code from compiling with CCPP/"GFS physics".

We have added documentation to help the user, including some code to generate an idealized surface file that the user can customized to their liking. The PR at the UWM level will also have a corresponding regression test. It won't be baseline tested, but it will provide an easy route for user to generate a run directory.

Many thanks go to @MicroTed for all of his help and contributions to this effort as well. He's agreed to provide a test run directory on that can be cloned by those who'd like to test the code inside a pre-generated run directory. You can find that here: https://github.com/MicroTed/ufs_ideal_run.git

Fixes ufs-community/ufs-weather-model/issues/2558

How Has This Been Tested?

These code changes have been compiled and tested on both Hera and Jet. Tests have been run to check that all the various types of sounding options can be used successfully. The full suite of regression tests will be run at the weather model level.

Checklist:

Please check all whether they apply or not

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• Any dependent changes have been merged and published in downstream modules

[lharris4]

Hello @LarissaReames-NOAA . Can you explain what this new test case is intended to test in FV3, and how to validate the solution? Also, have these test cases and this implementation been scientifically vetted? In particular, care must be taken in the initialization so it is in-balance and consistent with the FV3 vertical discretization.

Thanks
Lucas

[LarissaReames-NOAA]

Hello @LarissaReames-NOAA . Can you explain what this new test case is intended to test in FV3, and how to validate the solution? Also, have these test cases and this implementation been scientifically vetted? In particular, care must be taken in the initialization so it is in-balance and consistent with the FV3 vertical discretization.

Thanks

Lucas

Hi Lucas,

This is meant to be a framework for users to generate their own idealized cases, not to provide a case that produces a single, verifiable solution. It's intended (and has been used successfully at NSSL) to test how the model core and physics respond to various changes in schemes and settings. This sort of functionality was listed as a community need at the first UFS workshop to enable non-real-data cases, hence we're submitting this PR so that it can be in the trunk instead of having to pass around external forks and branches.

We made every effort to copy the general format and sequence that was used in other test cases in the test_cases file, so unless those aren't consistent with the vertical discretization, I have no reason to believe that what we provide here wouldn't also be. However, I'm happy to implement any changes that those more familiar with this code base than I say are necessary.

[lharris4]

We will need some scientific description of these idealized storm cases and of what results are expected, and the specific scientific or development need for these new cases, before this PR can be further considered. You mentioned successful use at NSSL; could you elaborate?

…

On Thu, Jan 16, 2025 at 5:05 PM Larissa Reames ***@***.***> wrote: Hello @LarissaReames-NOAA <https://github.com/LarissaReames-NOAA> . Can you explain what this new test case is intended to test in FV3, and how to validate the solution? Also, have these test cases and this implementation been scientifically vetted? In particular, care must be taken in the initialization so it is in-balance and consistent with the FV3 vertical discretization. Thanks Lucas Hi Lucas, This is meant to be a framework for users to generate their own idealized cases, not to provide a case that produces a single, verifiable solution. It's intended (and has been used successfully at NSSL) to test how the model core and physics respond to various changes in schemes and settings. This sort of functionality was listed as a community need at the first UFS workshop to enable non-real-data cases, hence we're submitting this PR so that it can be in the trunk instead of having to pass around external forks and branches. We made every effort to copy the general format and sequence that was used in other test cases in the test_cases file, so unless those aren't consistent with the vertical discretization, I have no reason to believe that what we provide here wouldn't also be. However, I'm happy to implement any changes that those more familiar with this code base than I say are necessary. — Reply to this email directly, view it on GitHub <#370 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMUQRVBXG2SVVGAMTJLGHK32LAUKZAVCNFSM6AAAAABVFZTMTWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKOJXGAYDKMBRGA> . You are receiving this because you commented.Message ID: ***@***.***>

[MicroTed]

@lharris4 Although the PR description may seem to give the impression of providing a case study, that is really only a provided example of an ability to run any suite in an general idealized environment with full physics. Therefore, no particular scientific results are expected. I brought up this capability at the 2023 UFS physics workshop, and there was general agreement that such capability would be useful (e.g., @ligiabernardet). I think of it as a 3D counterpart to the SCM. The included python script allows a user to set up any surface inputs as desired (e.g., land surface type and pattern), so it could be used for any kind of setup at any scale. The included sounding and initialization for an isolated storm are just one example of how it can be used. It will allow physics developers (e.g., CCPP) to run their own 3D tests in a controlled environment. At NSSL, this capability has been used for studying characteristics of deep convection and for CCPP microphysics testing.
Hope that helps!

[LarissaReames-NOAA]

I'll point out again that this PR does not provide for specific idealized storm cases as no input data is hardcoded to be set a particular way. It provides a framework for the user to create their own cases via the namelist and/or a user-provided sounding file, hence there can be no one specific expected outcome, nor is there a single "correct answer" against which to compare a case like this. However, there will be a regression test in the ufs-weather-model repository that will be baseline tested just to ensure consistency.

As far as scientific/developmental need, I'll provide a couple of points:

1. When testing new physics, particularly microphysics, it's helpful to be able to run a variety of environments and vertical shears to make sure that the code is producing reasonable solutions, whether it be a supercell test that needs to have the correct 3D reflectivity structures, and if the scheme is capable, the correct dual-pol signatures. For squall lines simulations, is the partitioning of convective versus stratiform cloud reasonable? What about the rate of upscale growth? These experiments can be informed both by observations and by published results from other models.

2. The ability to run idealized convective simulations can facilitate the use of very high grid resolutions (e.g., dx ~ 50-100 m). Even if the final intent is to perform real-world simulations, being able to test physics parameterizations at these resolutions is a logical (and computationally cheap) first step toward research at these scales. Simpler simulations often enable discovery of fundamental processes which may be masked in real-data runs which have considerable mesoscale heterogeneity.

I hope this additional explanation is helpful.

[bensonr]

The test_cases.F90 file is reserved for idealized dynamical core-only tests with well-defined initial conditions and peer-reviewed and/or accepted analytical solutions to allow the focus to remain on the development and behaviors of the dynamical core. Unless I am mistaken, the framework you are proposing will provide initial conditions via soundings and other means to foster the testing and development of physical parameterizations.

As the additions are specific to the UFS model, we propose you introduce this new functionality in files contained within the driver/UFS directory. We strongly encourage you to not introduce saturation tables, but instead allow hooks into existing tables and other related logic contained within the CCPP parameterizations.

[LarissaReames-NOAA]

@bensonr Thanks for that clarification on what is and isn't supposed to be in test_cases.F90. We were unaware of that restriction.

Based on your suggestion, I've moved all code that sets up the initial state of this new case to driver/UFS/fv_ideal.F90. Since you said that the cases in init_double_periodic are only to be used in dynamical core tests, I chose to use a compiler switch of ifdef GFS_PHYS to choose which initialization subroutine to call when grid_type=4. I've also used this compiler switch in one other place to prevent the overwriting of coriolis parameter choices made the new ideal case setup. Does this choice of compiler switch work for your purposes, or did you want to maintain the ability to call init_double_periodic even when GFS_PHYS is used? I wanted to avoid having to add an additional flag to the dycore namelist, hence the use of the compiler flag, but I'm happy to change that if this does not work.

Thanks again for the clarity here, it's much appreciated as we work to provide this capability to the community.

[ligiabernardet]

The capability provided by @LarissaReames-NOAA will be very helpful to UFS users and developers since, as she said, it demonstrates how the model core and physics respond to various changes in schemes and settings. This feature has been requested by numerous UFS users. The results are the outcome of the combined behavior of the dynamical core and the physics suite. Having the tests in driver/UFS/fv_ideal.F90 seems adequate.

[bensonr]

@ligiabernardet - as people work to implement new functionality, we'd prefer they start a discussion or enter an issue tagged as an enhancement. This opens a dialogue and provides a mechanism for feedback at the beginning of development and not once a fully-implemented solution is in place. Doing so will save needless back and forth at the time a PR is entered and speed the process of review and merge.

[LarissaReames-NOAA]

@bensonr I apologize that we didn't start an issue or discussion first. I implemented this solution many years ago (like 2019ish) just for in-house use when I worked for NSSL. Then, when others saw our results, they started asking for the capability for themselves. It was sort of organic and all happened outside of the GitHub environment and before most of these processes were even broadly known/followed organization-wide. If there are still issues with my proposed implementation, or you'd like to have more discussions on it, I'm happy to create an issue now even if it is a bit out of order.

[bensonr]

@LarissaReames-NOAA - no need to apologize. While it is our of order, please do create an issue so we can track that against the PR. I will work with our preferred list of reviewers to ensure things are acceptable.

[bensonr]

@LarissaReames-NOAA - movement on this PR will be delayed as we are in the midst of our quinquennial review. Please give us two weeks to get PR reviews and whatnot done.