Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MOBT-211: mosg__model_run attribute handling in weather symbols #1670

Merged
merged 14 commits into from
Feb 18, 2022

Conversation

bayliffe
Copy link
Contributor

Addresses https://github.com/metoppv/mo-blue-team/issues/211

This PR adds the capability to pull through model and cycle information in the mosg__model_run attribute to the weather symbol outputs. The changes affect both wxcode and wxcode-modal. Steps taken were:

  • factor out existing code from weighted_blend.py
    • extend the functionality to handle duplicate entries
    • add additional checks to ensure the function can only be applied in suitable cases
    • add unit tests
  • add record_run_attr options to the wxcode CLI and pull that through the plugin
  • add record_run_attr options to the wxcode-modal CLI and pull that through the plugin

The acceptance test data relating to these CLIs has been updated where necessary, ensuring it is compliant with our metadata standards. See issue for acceptance test data branch.

Testing:

  • Ran tests and they passed OK
  • Added new tests for the new feature(s)

@codecov
Copy link

codecov bot commented Feb 10, 2022

Codecov Report

Merging #1670 (05cd6ce) into master (897aa49) will increase coverage by 0.01%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1670      +/-   ##
==========================================
+ Coverage   98.18%   98.20%   +0.01%     
==========================================
  Files         110      110              
  Lines       10110    10173      +63     
==========================================
+ Hits         9927     9990      +63     
  Misses        183      183              
Impacted Files Coverage Δ
improver/blending/utilities.py 100.00% <100.00%> (ø)
improver/blending/weighted_blend.py 100.00% <100.00%> (ø)
improver/utilities/cube_checker.py 100.00% <100.00%> (ø)
improver/wxcode/modal_code.py 100.00% <100.00%> (ø)
improver/wxcode/weather_symbols.py 98.47% <100.00%> (+0.02%) ⬆️
improver/cube_combiner.py 100.00% <0.00%> (ø)
improver/utilities/cube_manipulation.py 99.07% <0.00%> (+0.05%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 897aa49...05cd6ce. Read the comment docs.

Copy link
Contributor

@MoseleyS MoseleyS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few suggestions to think about.

improver/blending/utilities.py Outdated Show resolved Hide resolved
improver/wxcode/modal_code.py Outdated Show resolved Hide resolved
improver/wxcode/weather_symbols.py Outdated Show resolved Hide resolved
This will enable the reuse of this functionality, possibly with
modifications, for diagnostics that are created after model blending.
It is not possible to create a model_run_attr from previously model
blended inputs if they do not contain a model_run attribute. The cycles
at which these source models were run has been lost. As such the
set_record_run_attr function will raise an exception.
Added tests to cover the invocation of the set_record_run_attr function 
from the weather symbols and modal symbol plugins.
Copy link
Contributor

@MoseleyS MoseleyS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some more little things for you to consider

Comment on lines 218 to 223
try:
cube.coord("blend_time")
except CoordinateNotFoundError:
return False
else:
return True
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
try:
cube.coord("blend_time")
except CoordinateNotFoundError:
return False
else:
return True
return True if "blend_time" in [c.name() for c in cube.coords()] else False

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love a one liner, but I'm not sure the proposed approach is clearer, so I am going to leave this one.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could still do it in four lines instead of six:

    if "blend_time" in [c.name() for c in cube.coords()]:
        return True
    else:
        return False

Actually, it could also be return if "blend_time" in [c.name() for c in cube.coords()]

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll go with return "blend_time" in [c.name() for c in cube.coords()] as it is clear and concise.

improver/blending/utilities.py Outdated Show resolved Hide resolved
improver/blending/utilities.py Outdated Show resolved Hide resolved
improver/wxcode/modal_code.py Outdated Show resolved Hide resolved
MoseleyS
MoseleyS previously approved these changes Feb 16, 2022
Copy link
Contributor

@gavinevans gavinevans left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @bayliffe 👍

I've added some comments with help from @mspelman07.

improver/wxcode/modal_code.py Outdated Show resolved Hide resolved
improver/blending/utilities.py Outdated Show resolved Hide resolved
improver/blending/utilities.py Outdated Show resolved Hide resolved
improver/blending/utilities.py Show resolved Hide resolved
improver_tests/wxcode/wxcode/test_WeatherSymbols.py Outdated Show resolved Hide resolved
improver_tests/blending/test_utilities.py Show resolved Hide resolved
improver/blending/utilities.py Show resolved Hide resolved
Copy link
Contributor

@gavinevans gavinevans left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the changes @bayliffe 👍

I think that this is fine apart from a couple of typos.

improver/blending/utilities.py Outdated Show resolved Hide resolved
Co-authored-by: gavinevans <gavin.evans@metoffice.gov.uk>


def set_record_run_attr(
cubelist: CubeList, record_run_attr: str, model_id_attr: Optional[str]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default value for model_id_attr in ModalWeatherCode is None, which this method now says it won't accept. We either need to not call this method in this instance, or change the typing back to include None.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you referring to the change from Union[str, None] to Optional[str]? The use of Optional[str] where None is allowed is preferred to Union[str, None] by mypy: https://mypy.readthedocs.io/en/stable/kinds_of_types.html#optional-types-and-the-none-type.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I was. I didn't know that. Thanks Gavin!

@MoseleyS MoseleyS merged commit 4ab4120 into metoppv:master Feb 18, 2022
@bayliffe bayliffe deleted the mobt211 branch February 22, 2022 13:41
MoseleyS added a commit to MoseleyS/improver that referenced this pull request Mar 2, 2022
* master:
  MOBT127 tiny tweak to filter_realizations (metoppv#1682)
  Mobt 160 ecc masked data (metoppv#1662)
  Convert test_ManipulateReliabilityTable to pytest (metoppv#1678)
  Alter spot extract cli (metoppv#1666)
  Enable site cube input to ConstructReliabilityCalibrationTables (metoppv#1667)
  Corrects example of a CLI in the Read the Doc documentation (metoppv#1673)
  MOBT-211: mosg__model_run attribute handling in weather symbols (metoppv#1670)

# Conflicts:
#	improver_tests/wxcode/wxcode/test_WeatherSymbols.py
fionaRust added a commit that referenced this pull request Mar 9, 2022
* master:
  Remove flawed interpretation of the blend_time coordinate. (#1690)
  DOC: Removal of unnecessary exclusions in sphinx apidoc build (#1689)
  MOBT127 tiny tweak to filter_realizations (#1682)
  Mobt 160 ecc masked data (#1662)
  Convert test_ManipulateReliabilityTable to pytest (#1678)
  Alter spot extract cli (#1666)
  Enable site cube input to ConstructReliabilityCalibrationTables (#1667)
  Corrects example of a CLI in the Read the Doc documentation (#1673)
  MOBT-211: mosg__model_run attribute handling in weather symbols (#1670)
MoseleyS pushed a commit to MoseleyS/improver that referenced this pull request Aug 22, 2024
…ppv#1670)

* Factor out set_record_run_attr function.

This will enable the reuse of this functionality, possibly with
modifications, for diagnostics that are created after model blending.

* Update record_run_attr function to avoid duplicate entries.

* Add exception for model blended input without model_run_attr.

It is not possible to create a model_run_attr from previously model
blended inputs if they do not contain a model_run attribute. The cycles
at which these source models were run has been lost. As such the
set_record_run_attr function will raise an exception.

* Add model_run setting for wxcode.

* Add model_run_setting for wxmodal.

* Update checksums to match new test data.

* Additional tests.

Added tests to cover the invocation of the set_record_run_attr function 
from the weather symbols and modal symbol plugins.

* Typo fix.

* Add exception for missing model_id_attr in instances where it is required.

* Add unit tests for new is_model_blended function.

* Review changes.

* Blend test tweak.

* Review changes.

* Update improver/blending/utilities.py

Co-authored-by: gavinevans <gavin.evans@metoffice.gov.uk>

Co-authored-by: gavinevans <gavin.evans@metoffice.gov.uk>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants