Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bugfix: Investigate intermittent results from unit_plot_tc.xml unit test #2582

Closed
8 of 22 tasks
JohnHalleyGotway opened this issue Jun 21, 2023 · 1 comment
Closed
8 of 22 tasks
Assignees
Labels
component: testing Software testing issue MET: Plotting Tools priority: medium Medium Priority requestor: METplus Team METplus Development Team type: bug Fix something that is not working
Milestone

Comments

@JohnHalleyGotway
Copy link
Collaborator

Describe the Problem

The call to the plot_tcmpr.R Rscript in unit_plot_tc.xml produces inconsistent results when run via the MET nightly build. For NB20230620 and NB20230621, differences were flagged in two images:

test_output/plot_tc/ABS_AMAX_WIND-BMAX_WIND_boxplot.png
test_output/plot_tc/TK_ERR_boxplot.png

NB20230620 produced the expected result in MET-develop whereas NB20230621 produced the expected result in MET-develop-ref. So the behavior is intermittent.

Expected Behavior

The expected results are on the left while the unexpected results are on the right:
Screen Shot 2023-06-21 at 10 48 47 AM

Screen Shot 2023-06-21 at 10 50 56 AM

To Reproduce

To start, recommend updating unit_plot_tc.xml by adding the -save_data option for plot_tcmpr.R. That will save off the data that was used for plotting to a file which may provide more clues if/when this unexpected behavior occurs in the nightly build.

This should be added to both the develop and develop-ref branches.

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

  • Select engineer(s) or no engineer required
  • Select scientist(s) or no scientist required

Labels

  • Select component(s)
  • Select priority
  • Select requestor(s)

Projects and Milestone

  • Select Organization level Project for support of the current coordinated release
  • Select Repository level Project for development toward the next official release or add alert: NEED CYCLE ASSIGNMENT label
  • Select Milestone as the next bugfix version

Define Related Issue(s)

Consider the impact to the other METplus components.

Bugfix Checklist

See the METplus Workflow for details.

  • Complete the issue definition above, including the Time Estimate and Funding Source.
  • Fork this repository or create a branch of main_<Version>.
    Branch name: bugfix_<Issue Number>_main_<Version>_<Description>
  • Fix the bug and test your changes.
  • Add/update log messages for easier debugging.
  • Add/update unit tests.
  • Add/update documentation.
  • Push local changes to GitHub.
  • Submit a pull request to merge into main_<Version>.
    Pull request: bugfix <Issue Number> main_<Version> <Description>
  • Define the pull request metadata, as permissions allow.
    Select: Reviewer(s) and Development issue
    Select: Organization level software support Project for the current coordinated release
    Select: Milestone as the next bugfix version
  • Iterate until the reviewer(s) accept and merge your changes.
  • Delete your fork or branch.
  • Complete the steps above to fix the bug on the develop branch.
    Branch name: bugfix_<Issue Number>_develop_<Description>
    Pull request: bugfix <Issue Number> develop <Description>
    Select: Reviewer(s) and Development issue
    Select: Repository level development cycle Project for the next official release
    Select: Milestone as the next official version
  • Close this issue.
@JohnHalleyGotway JohnHalleyGotway added type: bug Fix something that is not working component: testing Software testing issue priority: medium Medium Priority alert: NEED ACCOUNT KEY Need to assign an account key to this issue requestor: METplus Team METplus Development Team MET: Plotting Tools labels Jun 21, 2023
@JohnHalleyGotway JohnHalleyGotway added this to the MET 11.1.0 milestone Jun 21, 2023
@JohnHalleyGotway JohnHalleyGotway self-assigned this Jun 21, 2023
@JohnHalleyGotway JohnHalleyGotway moved this from 📋 Backlog to 🏗 In progress in MET-11.1.0 Development Jun 21, 2023
@JohnHalleyGotway JohnHalleyGotway changed the title Bugfix: Fix intermittent results from unit_plot_tc.xml unit test Bugfix: Investigate intermittent results from unit_plot_tc.xml unit test Jun 21, 2023
@JohnHalleyGotway
Copy link
Collaborator Author

The nightly build run on seneca for NB20230622 did NOT flag any differences after I added the -save_data option for both the develop and develop-ref branches. Will close this issue for now since this transient behavior did not reoccur today. But if/when it does reoccur, recommend reopening this issue to debug further.

@github-project-automation github-project-automation bot moved this from 🏗 In progress to ✅ Done in MET-11.1.0 Development Jun 22, 2023
JohnHalleyGotway added a commit that referenced this issue Jul 31, 2023
* Update develop-ref after #2555 and #2561 (#2568)

Co-authored-by: John Halley Gotway <johnhg@ucar.edu>
Co-authored-by: Seth Linden <linden@seneca.rap.ucar.edu>
Co-authored-by: jprestop <jpresto@ucar.edu>
Co-authored-by: Daniel Adriaansen <dadriaan@ucar.edu>
Co-authored-by: John and Cindy <halleygotway@Halleys-Mac-mini.local>
Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu>
Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com>
Co-authored-by: hsoh-u <hsoh@ucar.edu>
Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu>
Co-authored-by: Seth Linden <linden@ucar.edu>
Co-authored-by: lisagoodrich <33230218+lisagoodrich@users.noreply.github.com>
Co-authored-by: davidalbo <dave@ucar.edu>
Co-authored-by: Lisa Goodrich <lisag@ucar.edu>
Co-authored-by: metplus-bot <97135045+metplus-bot@users.noreply.github.com>
Co-authored-by: j-opatz <59586397+j-opatz@users.noreply.github.com>
Fix Python environment issue (#2407)
fix definitions of G172 and G220 based on comments in NOAA-EMC/NCEPLIBS-w3emc#157. (#2406)
fix #2380 develop override (#2382)
fix #2408 develop empty config (#2410)
fix #2390 develop compile zlib (#2404)
fix #2412 develop climo (#2422)
fix #2437 develop convert (#2439)
fix for develop, for #2437, forgot one reference to the search_parent for a dictionary lookup.
fix #2452 develop airnow (#2454)
fix #2449 develop pdf (#2464)
fix #2402 develop sonarqube (#2468)
fix #2426 develop buoy (#2475)
fix 2518 dtypes appf docs (#2519)
fix 2531 compilation errors (#2533)
fix #2531 compilation_errors_configure (#2535)

* Per #2582, update unit_plot_tc.xml to use the -save_data option.

* modified the config for mode regarding multivar_intensity_flag

* #2633 Set the default seeps_p1_thresh to >=0.1&&<=0.85

* #2633 Added default setting of seeps_p1_thresh

* Per #2633, small tweaks to the config file comments and documentation

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: metplus-bot <97135045+metplus-bot@users.noreply.github.com>
Co-authored-by: John Halley Gotway <johnhg@ucar.edu>
Co-authored-by: David Albo <dave@ucar.edu>
Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu>
@JohnHalleyGotway JohnHalleyGotway removed the alert: NEED ACCOUNT KEY Need to assign an account key to this issue label Oct 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: testing Software testing issue MET: Plotting Tools priority: medium Medium Priority requestor: METplus Team METplus Development Team type: bug Fix something that is not working
Projects
No open projects
Status: Done
Development

No branches or pull requests

1 participant