Run Hurricane Matthew test case in WRF container on Derecho

[jaredalee]

Describe the Task

Using the WRF container for a test on Derecho, simulate the Hurricane Matthew test case from the WRF Online Tutorial (https://www2.mmm.ucar.edu/wrf/OnLineTutorial/CASES/SingleDomain/index.php). Document the steps/commands needed to complete the execution. Attempt to use two nodes when running WRF for this case to demonstrate full multi-node capability of the WRF container. Make special note whether nodes=2:ppn=128 works fine on Derecho, or if fewer processors per node than 128 are needed due to the relatively small domain/grid size of this test case (if domain patches are too small due to requesting too many processors/tasks, then WRF will stop immediately).

Time Estimate

1–2 days

Sub-Issues

Consider breaking the task down into sub-issues.

• Add a checkbox for each sub-issue here.

Relevant Deadlines

Target completion 1 Mar 2024.

Funding Source

7790013

Define the Metadata

Assignee

• Select engineer(s) or no engineer required
• Select scientist(s) or no scientist required

Labels

• Select component(s)
• Select priority
• Select requestor(s)

Projects and Milestone

• Select Repository and/or Organization level Project(s) or add alert: NEED PROJECT ASSIGNMENT label
• Select Milestone as the next official version or Future Versions

Task Checklist

• Complete the issue definition above, including the Time Estimate and Funding Source.
• Fork this repository or create a branch of develop.
  Branch name: feature_<Issue Number>_<Description>
• Complete the development and test your changes.
• Add/update log messages for easier debugging.
• Add/update unit tests.
• Add/update documentation.
• Push local changes to GitHub.
• Submit a pull request to merge into develop.
  Pull request: feature <Issue Number> <Description>
• Define the pull request metadata, as permissions allow.
  Select: Reviewer(s) and Development issues
  Select: Repository level development cycle Project for the next official release
  Select: Milestone as the next official version
• Iterate until the reviewer(s) accept and merge your changes.
• Delete your fork or branch.
• Close this issue.

[jaredalee]

@hahnd @rcplane In order for the METplus container (#48) to read in data properly, use this namelist.input and vars_io.txt file from the attached .tar.gz file to write out pressure-level diagnostics files (wrfout_plev*). It will also create height-level diagnostics files (wrfout_zlev*), as a demonstration of capability. As a side benefit, the vars_io.txt file also suppresses numerous extraneous variables from the wrfout files to make it less cluttered, further demonstrating some capabilities.

matthew_config.tar.gz

[georgemccabe]

@jaredalee, are the only inputs needed to run WRF included in the tar file you provided (namelist.input and vars_io.txt)? If so, I would suggest that we add those files to the NCAR/i-wrf repository under use_cases/Hurricane_Matthew/WRF since they are small. That way the only input data we would need to provide separately for this use case would be the MADIS observation files.

[jaredalee]

@georgemccabe There are other inputs needed to run WPS (geogrid, ungrib, metgrid), real, and WRF, such as GFS model data for ICs/LBCs. The WRF Online Tutorial site has a link that can be used to download that data. So we could either ask users to download the data from there themselves, or we could also host all the required data ourselves, which could be insurance against MMM revamping the tutorial and updating it with a newer case. There's also a WPS namelist that's required, but that's just a small text file like the WRF namelist. We could either store the namelists in our repo somewhere for download, or include instructions on how to modify the default namelists that will get installed with WPS & WRF. It all depends on how easy it will be for users to move, replace, edit, or symlink to files within or outside the container image, and that's something I don't currently have a good feel for.

[jaredalee]

I'm realizing we never closed the loop on this to get a working version running on Derecho. @hahnd, @rcplane, and @georgemccabe, can any of you provide an update on where this stands, to test and document (part of #38) running the WPS/WRF container on Derecho for this case? I'd love to be able to resolve this issue, especially as we already have the Hurricane Matthew case tested and documented for Jetstream2, Red Cloud, and Windows (https://i-wrf.readthedocs.io/en/latest/Users_Guide/usecases.html#hurricane-matthew). Then once we resolve this issue, we can finally resolve #41.

[georgemccabe]

@hahnd, @rcplane, have either of you run WRF using Apptainer? If we know that it will run successfully using Apptainer, I think the instructions for running on Jetstream2 or RedCloud could be easily modified to use Apptainer instead of Docker.

[hahnd]

I have not run it with Apptainer yet. I just created a new image yesterday with WPS v4.6.0, which is the latest (and only) release to include compiler options for Intel oneAPI. The image is pushed to docker hub: ncar/iwrf:20241203.

[georgemccabe]

When this work is completed, update the ncar/iwrf:latest tag to point to the same version as ncar/iwrf:20241203

[hahnd]

@rcplane - have you gotten any further with the export to apptainer?