Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature #2699 consensus_config #2736

Merged
merged 8 commits into from
Nov 17, 2023
Merged

Feature #2699 consensus_config #2736

merged 8 commits into from
Nov 17, 2023

Conversation

JohnHalleyGotway
Copy link
Collaborator

@JohnHalleyGotway JohnHalleyGotway commented Nov 14, 2023 •
edited
Loading

Expected Differences

  • Do these changes introduce new tools, command line arguments, or configuration file options? [Yes]

    If yes, please describe:

    Adds new diag_required and diag_min_req config options to the consensus dictionary.

  • Do these changes modify the structure of existing or add new output data types (e.g. statistic line types or NetCDF variables)? [No]

    If yes, please describe:

Pull Request Testing

  • Describe testing already performed for these changes:

    Ran this testing GHA workflow to confirm that the code all compiles and runs as expected. It does produce a different output file, as expected, based on changes to in unit_tc_pairs.xml.

  • Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:

This feature branch is compiled and available for your use on seneca in:
/d1/projects/MET/MET_pull_requests/met-12.0.0/beta2/MET-feature_2699_consensus_config

  • Do these changes include sufficient documentation updates, ensuring that no errors or warnings exist in the build of the documentation? [Yes]

  • Do these changes include sufficient testing updates? [Yes]
    Updated unit_tc_paris.xml test to use real data to create a consensus using GFSO, HMON, and HWRF.

  • Will this PR result in changes to the test suite? [Yes]

    If yes, describe the new output and/or changes to the existing output:

    Modifies the contents of a single TC-Pairs output file al092022_20220926_DIAGNOSTICS_CONSENSUS.tcst due to modified inputs.

  • Please complete this pull request review by [Wed 11/15/23].

Pull Request Checklist

See the METplus Workflow for details.

  • Review the source issue metadata (required labels, projects, and milestone).
  • Complete the PR definition above.
  • Ensure the PR title matches the feature or bugfix branch name.
  • Define the PR metadata, as permissions allow.
    Select: Reviewer(s) and Development issue
    Select: Milestone as the version that will include these changes
    Select: Coordinated METplus-X.Y Support project for bugfix releases or MET-X.Y.Z Development project for official releases
  • After submitting the PR, select the ⚙️ icon in the Development section of the right hand sidebar. Search for the issue that this PR will close and select it, if it is not already selected.
  • After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.
  • Close the linked issue and delete your feature or bugfix branch from GitHub.

@JohnHalleyGotway
Per #2699, add hooks for diag_required and min_diag_req consensus con…
d2bea10 
…fig entries. This just adds them. Still need to update TC-Pairs to actually use the logic.
@JohnHalleyGotway
Per #2699, update consensus deriviation logic in track_info.h/.cc to …
695e51b 
…check for required and minimum number of diagnostic members.
@JohnHalleyGotway
Per #2699, update consensus deriviation logic in tc_pairs.cc to check…
1f82b47 
… for required and minimum number of diagnostic members.
@JohnHalleyGotway
Merge remote-tracking branch 'origin/develop' into feature_2699_conse…
1efa9af 
…nsus_config
@JohnHalleyGotway
Per #2699, delete duplicated documentation line.
2bc96ce
@JohnHalleyGotway
Per #2699, swap in a more resonable unit test for tc_pairs consensus …
cf6e80b 
…of diagnostics, work on log messages, and also make sure that the TCDIAG output columns are all populated.
@JohnHalleyGotway JohnHalleyGotway added this to the MET 12.0.0 milestone Nov 14, 2023
@JohnHalleyGotway JohnHalleyGotway linked an issue Nov 14, 2023 that may be closed by this pull request
Refine TC-Pairs consensus diagnostics configuration options #2699
Closed
21 tasks
@jvigh
Copy link
Contributor

jvigh commented Nov 17, 2023

@JohnHalleyGotway Are these errors in the diff testing important:
https://github.com/dtcenter/MET/actions/runs/6894967897/job/18759196089?pr=2736

jvigh
jvigh reviewed Nov 17, 2023
View reviewed changes
docs/Users_Guide/tc-pairs.rst Outdated
@@ -7,7 +7,7 @@ TC-Pairs Tool
Introduction
============

The TC-Pairs tool provides verification for tropical cyclone forecasts in ATCF file format. It matches an ATCF format tropical cyclone (TC) forecast with a second ATCF format reference TC dataset (most commonly the Best Track analysis). The TC-Pairs tool processes both track and intensity adeck data and probabilistic edeck data. The adeck matched pairs contain position errors, as well as wind, sea level pressure, and distance to land values for each TC dataset. The edeck matched pairs contain probabilistic forecast values and the verifying observation values. The pair generation can be subset based on user-defined filtering criteria. Practical aspects of the TC-Pairs tool are described in :numref:`TC-Pairs_Practical-information`.
The TC-Pairs tool provides verification for tropical cyclone forecasts in ATCF file format. It matches an ATCF format tropical cyclone (TC) forecast with a second ATCF format reference TC dataset (most commonly the Best Track analysis). The TC-Pairs tool processes both track and intensity adeck data and probabilistic edeck data. The adeck matched pairs contain position errors, as well as wind, sea level pressure, and distance to land values for each TC dataset. The edeck matched pairs contain probabilistic forecast values and the verifying observation values. The pair generation can be subset based on user-defined filtering criteria. Practical aspects of the TC-Pairs tool are described in :numref:`TC-Pairs_Practical-information`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is meant by "wind" in this sentence? If it is actually storm intensity, I suggest changing "wind" to "intensity". But if it's something different (like wind radii), then it'd be good to clarify that as well.

Similarly, is "sea level pressure" actually "minimum sea level pressure" (the lowest sea level pressure near the storm center)? If so, I'd suggest changing this as well.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the clarification @jvigh.

Changing the wording...
FROM: The adeck matched pairs contain position errors, as well as wind, sea level pressure, and distance to land values for each TC dataset.
TO: The adeck matched pairs contain position errors, as well as values for each TC dataset, including wind speed, wind radii, minimum sea level pressure, and distance to land.

jvigh
jvigh approved these changes Nov 17, 2023
View reviewed changes
Copy link
Contributor

@jvigh jvigh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have reviewed the commit history, code changes, GHA, and the full documentation chapter. Everything looks good. I've left one small note of clarification.

I note that I did not have time to run the tools on Seneca and test edge cases.

Nice work on this.

@JohnHalleyGotway
Copy link
Collaborator Author

@JohnHalleyGotway Are these errors in the diff testing important: dtcenter/MET/actions/runs/6894967897/job/18759196089?pr=2736

@jvigh yes, those ERRORS are important, but they won't be addressed via this PR. They illustrate a short-coming in the existing PERL and R-based differencing logic that MET currently uses.

@natalieb-noaa will be working on an overhaul to MET's unit testing framework and differencing logic... switching over to using the METplus Python-based one, and likely enhancing it along the way. So this is definitely an example of a difference in the output that should be handled in a more elegant way!

@JohnHalleyGotway JohnHalleyGotway merged commit bbe91b4 into develop Nov 17, 2023
6 checks passed
@JohnHalleyGotway JohnHalleyGotway deleted the feature_2699_consensus_config branch November 17, 2023 16:50
This was referenced Nov 17, 2023
Closed
Merged
JohnHalleyGotway pushed a commit that referenced this pull request Nov 17, 2023
@github-actions
Update develop-ref after #2728, #2723, #2736 (#2747)
f39afeb 
Co-authored-by: jprestop <jpresto@ucar.edu>
Co-authored-by: Seth Linden <linden@seneca.rap.ucar.edu>
Co-authored-by: John Halley Gotway <johnhg@ucar.edu>
Co-authored-by: Daniel Adriaansen <dadriaan@ucar.edu>
Co-authored-by: John and Cindy <halleygotway@Halleys-Mac-mini.local>
Co-authored-by: rgbullock <bullock@ucar.edu>
Co-authored-by: Randy Bullock <bullock@seneca.rap.ucar.edu>
Co-authored-by: Dave Albo <dave@seneca.rap.ucar.edu>
Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu>
Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com>
Co-authored-by: hsoh-u <hsoh@ucar.edu>
Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu>
Co-authored-by: Seth Linden <linden@ucar.edu>
Co-authored-by: lisagoodrich <33230218+lisagoodrich@users.noreply.github.com>
Co-authored-by: davidalbo <dave@ucar.edu>
Co-authored-by: Lisa Goodrich <lisag@ucar.edu>
Co-authored-by: metplus-bot <97135045+metplus-bot@users.noreply.github.com>
Co-authored-by: j-opatz <59586397+j-opatz@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Jonathan Vigh <jvigh@ucar.edu>
Co-authored-by: Tracy Hertneky <39317287+hertneky@users.noreply.github.com>
Co-authored-by: David Albo <dave@ucar.edu>
Co-authored-by: Dan Adriaansen <dadriaan@ucar.edu>
Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>
Co-authored-by: root <root@83062d57c5dd>
fix 2518 dtypes appf docs (#2519)
fix 2531 compilation errors (#2533)
fix #2531 compilation_errors_configure (#2535)
fix #2514 develop clang (#2563)
fix #2575 develop python_convert (#2576)
Fix Python environment issue (#2407)
fix definitions of G172 and G220 based on comments in NOAA-EMC/NCEPLIBS-w3emc#157. (#2406)
fix #2380 develop override (#2382)
fix #2408 develop empty config (#2410)
fix #2390 develop compile zlib (#2404)
fix #2412 develop climo (#2422)
fix #2437 develop convert (#2439)
fix for develop, for #2437, forgot one reference to the search_parent for a dictionary lookup.
fix #2452 develop airnow (#2454)
fix #2449 develop pdf (#2464)
fix #2402 develop sonarqube (#2468)
fix #2426 develop buoy (#2475)
fix 2596 main v11.1 rpath compilation (#2614)
fix #2514 main_v11.1 clang (#2628)
fix #2644 develop percentile (#2647)
fix #2730 develop SI_BCU (#2732)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jvigh jvigh jvigh approved these changes

Assignees
No one assigned
Labels
None yet
Projects
No open projects
MET-12.0.0 Development
Status: 🏁 Done
Milestone
MET-12.0.0
Development

Successfully merging this pull request may close these issues.

Refine TC-Pairs consensus diagnostics configuration options
2 participants
@JohnHalleyGotway @jvigh