Implement aggregation of the new ECNT statistics: SPREAD_MD, MAE, MAE_OERR, BIAS_RATIO, N_GE_OBS, ME_GE_OBS, N_LT_OBS, and ME_LT_OBS

[JohnHalleyGotway]

Describe the Enhancement

MET development version 11.0.0-beta4 made several changes to the "stat" output. This issue is to enhance METcalcpy to aggregate these newly added statistics across multiple cases.

• Issue Add the Mean Absolute Difference (SPREAD_MD) to the ECNT line type MET#2332 adds 1 new column (SPREAD_MD) to the ECNT line type generated by Ensemble-Stat and the HiRA method in Point-Stat. See Table 13.2. The SPREAD_MD is the ensemble mean absolute difference and is an alternative measure of ensemble spread. This column should be aggregated over multiple cases using a weighted average approach, where the weight is defined by the number of matched pairs in the TOTAL column.

• Issue Add MAE to the ECNT line type from Ensemble-Stat and for HiRA MET#2325 adds 2 new columns (MAE and MAE_OERR) to the ECNT line type generated by Ensemble-Stat and the HiRA method in Point-Stat. See Table 13.2. The MAE is the mean absolute error. These columns should be aggregated together over multiple cases using a weighted average approach, where the weight is defined by the number of matched pairs in the TOTAL column.

These MAE's should be added to the list of statistics where 0 is the perfect score.

• Issue Add new bias ratio statistic to the ECNT line type from Ensemble-Stat and for HiRA MET#2058 adds 5 new columns (BIAS_RATIO, N_GE_OBS, ME_GE_OBS, N_LT_OBS, ME_LT_OBS) to the ECNT line type generated by Ensemble-Stat and the HiRA method in Point-Stat. See Table 13.2. All 5 columns are added so that the BIAS_RATIO can be aggregated over multiple runs, as described below.
  • N_GE_OBS is an integer and is aggregated over multiple runs by summing the values.
  • ME_GE_OBS is a float and is aggregated as a weighted average where the weights are defined by the N_GE_OBS column.
  • N_LT_OBS is an integer and is aggregated over multiple runs by summing the values.
  • ME_LT_OBS is a float and is aggregated as a weighted average where the weights are defined by the N_LT_OBS column.
  • BIAS_RATIO is a float >0. It aggregated by first aggregating ME_GE_OBS and ME_LT_OBS and then computing BIAS_RATIO = ME_GE_OBS / absolute value of ME_LT_OBS.

The ME_GE_OBS and ME_LT_OBS columns are mean errors and should added to list of statistics where 0 is the perfect score.
The BIAS_RATIO is the ratio of those 2 ME values, and I suppose a perfect score would be 0/0 which is undefined. This probably should go in the list of statistics where 1 is the perfect score... where the amount of high bias equal the amount of low bias. In this way, its similar to the frequency bias (FBIAS) statistic.

Sample data for this issue can be found in the nightly build output in seneca:/d1/projects/MET/MET_regression/develop/NB20221103/MET-develop/test_output (or a more recent date).
And here's a tarfile of that output:
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/NB20221103_develop_test_output.tar.gz

Time Estimate

Estimate the amount of work required here.
Issues should represent approximately 1 to 3 days of work.

Sub-Issues

Consider breaking the enhancement down into sub-issues.

• Add a checkbox for each sub-issue here.

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

Split 2799991 and 2773542

Define the Metadata

Assignee

• Select engineer(s) or no engineer required
• Select scientist(s) or no scientist required

Labels

• Select component(s)
• Select priority
• Select requestor(s)

Projects and Milestone

• Select Repository and/or Organization level Project(s) or add alert: NEED PROJECT ASSIGNMENT label
• Select Milestone as the next official version or Future Versions

Define Related Issue(s)

Consider the impact to the other METplus components.

• METplus, MET, METdataio, METviewer, METexpress, METcalcpy, METplotpy

Enhancement Checklist

See the METplus Workflow for details.

• Complete the issue definition above, including the Time Estimate and Funding Source.
• Fork this repository or create a branch of develop.
  Branch name: feature_<Issue Number>_<Description>
• Complete the development and test your changes.
• Add/update log messages for easier debugging.
• Add/update unit tests.
• Add/update documentation.
• Push local changes to GitHub.
• Submit a pull request to merge into develop.
  Pull request: feature <Issue Number> <Description>
• Define the pull request metadata, as permissions allow.
  Select: Reviewer(s) and Linked issues
  Select: Repository level development cycle Project for the next official release
  Select: Milestone as the next official version
• Iterate until the reviewer(s) accept and merge your changes.
• Delete your fork or branch.
• Close this issue.