Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add plot for distribution of convergence diagnostics #105

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

aloctavodia
Copy link
Contributor

@aloctavodia aloctavodia commented Nov 15, 2024

I don't have a good name for this plot so its code name is plot_nose. The motivation is to visually inspect the values of ESS and R-hat for many variables.

In PyMC-BART we have a plot_convergence function that generates a similar plot, which we used to check the "BART variables" which are always multidimensional with a size at least equal to the observations. The interest is not on any individual variables. This type of plot may be of interest to others outside of PyMC-BART.

azp.plot_nose(radon, var_names=["za_county"])

I feel tempted to call "rhat_rank" just "rhat". What do you think?

nose_default

specific diagnostics can be passed as strings.

azp.plot_nose(radon, var_names=["za_county"], diagnostics=["ess_tail", "ess_median"])

nose_method

some ess methods accept a prob argument, we can pass it also as part of the string, like a "tuple"

azp.plot_nose(radon,
              var_names=["za_county"],
              diagnostics=["ess_tail(0.1, 0.9)", "ess_local(0.1, 0.9)", "ess_quantile(0.9)"],
              ref_line=False)

nose_probs

We could also add reference lines for ESS and r-hat following standard recommendations (added). For PyMC-BART, we have an "inflated" R-hat reference value. We use a heuristics to increase the 1.01 value based on the number of variables. Not saying we should use that heuristic, just saying we may want to have something similar.


📚 Documentation preview 📚: https://arviz-plots--105.org.readthedocs.build/en/105/

@codecov-commenter
Copy link

codecov-commenter commented Nov 15, 2024

Codecov Report

Attention: Patch coverage is 12.96296% with 47 lines in your changes missing coverage. Please review.

Project coverage is 84.67%. Comparing base (838fa5e) to head (a23d3fc).

Files with missing lines Patch % Lines
src/arviz_plots/plots/convergencedistplot.py 11.32% 47 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #105      +/-   ##
==========================================
- Coverage   86.03%   84.67%   -1.37%     
==========================================
  Files          24       25       +1     
  Lines        2843     2897      +54     
==========================================
+ Hits         2446     2453       +7     
- Misses        397      444      +47     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@AlexAndorra
Copy link

I love that @aloctavodia , I think it'd be super helpful for most models, because a huge majority is highly dimensional and the classic plots don't allow you to diagnose the parameters quickly

@AlexAndorra
Copy link

Anything I can help with to merge this one @aloctavodia ?

Copy link

@AlexAndorra AlexAndorra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks all good @aloctavodia , except for a typo I think. After that we can merge

azp.style.use("arviz-clean")

data = load_arviz_data("radon")
pc = azp.plot_ess(

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be azp.plot_convergence_dist?

Copy link
Contributor Author

@aloctavodia aloctavodia Dec 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! It seems I just copied and pasted another example.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants