Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CORP decomposition wrapper #811

Open
nicholasloveday opened this issue Jan 23, 2025 · 1 comment
Open

CORP decomposition wrapper #811

nicholasloveday opened this issue Jan 23, 2025 · 1 comment

Comments

@nicholasloveday
Copy link
Collaborator

Currently we can use the scores package to calculate the CORP decomposition https://www.pnas.org/doi/10.1073/pnas.2016191118#sec-4

It would be nice to have a wrapper to do this for us to remove the manual steps.

One design choice that needs to be made is, do we:

  • a) create one wrapper function where you pass in the scoring function, or
  • b) create a wrapper function for every relevant scoring function?
@nikeethr
Copy link
Collaborator

Just checking, I assume you mean wrapper (as in a function transformer or higher order function) and not decorator? Decorators may cause forced nesting, whereas a wrapper on its own is okay.

If you're going the function wrapper route and CORP is fairly generic, I'd just have the one wrapper and cater to edge cases as needed. If you feel CORP is very metric dependent, then maybe have the one wrapper offload/route (e.g. using a dict -> score mapping or similar) to a more metric specific wrappers for each score. For an initial implementation this may be fine, but if it is potentially impacting a lot of the codebase then please consider some additional thoughts I had below:


Alternative design:

Its harder to track side-effects and structure that may be necessary when using wrappers (which store state internally) - classes however can dictate how much visibility an object should have based on its structure.

I'm not sure of the exact details but for example if CORP requires defining MSB, DSC and UNC for a particular score, that can be abstracted in the class definition, initialization and/or using builder patterns

e.g.

# more likely an internal structure to setup what needs to be done
(msb, dsc, unc) = CorpDecomposer(<init_options>)
                      .with_inputs(obs, fcst)
                      .with_score(mse)
                      .with_calibration_method(...)
                      .with_extra_options(...)
                      .decompose()
### AND/OR ### 

def mse_corp(...):
    # above stuff goes in here, and user can use this less verbose API instead.
    # in fact this could still be a convenient wrapper, but the internal logic is implemented in a decomposed/extensible way.

This allows you to:

  • Apply methods (e.g. calibration) in a more extensible way
  • morph the actual decomposer itself e.g. MoreCorpyDecomposer(CorpDecomposer) for tricky edge cases if you need to.
  • be adaptable to non-trivial score apis
  • do things like corp1 = CorpDecomposer(<defaults>); corp2 = corp1.with_calibration_method(blah2) and compare.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants