-
Notifications
You must be signed in to change notification settings - Fork 55
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add SBC-ECDF interpretation guide #236
Comments
Thanks for initiating the discussion, Lasse! I agree that we should link to this great blog post in multiple places. Yet, I think that users might appreciate a gist of how to interpret SBC(-ECDF) plots even directly in the bayesflow tutorial resources. Do you think that there is some minimal viable SBC interpretation guide that would be appropriate within a bayesflow tutorial notebook, which would then link to the blog post for more details? |
I agree that it would be cool to provide more guidance in the BayesFlow context. We could take a tutorial notebook that consistently generates a well-calibrated posterior with simple shape (e.g., Gaussian), apply perturbations similar to the blog post (e.g., see lines 160-168), show the resulting rank histograms + ECDF plots with the built-in BayesFlow diagnostics, and refer to the blog post for further reading. The issue I see is that we would do more or less the same as the blog post while running the risk of bloating the tutorial materials, so we can explore whether it is feasible with minimal code. |
I agree that pointing to the notebook may be sufficient. @elseml would you perhaps add this link at the right places in some of our basic example notebooks? |
Add reference to SBC interpretation guide in starter examples (#236)
Initial thoughts by @marvinschmitt:
Alternatively, I suggest that we simply point to this excellent blog post (in tutorials and even in the documentation): https://hyunjimoon.github.io/SBC/articles/rank_visualizations.html
In my opinion, it covers all important cases and clearly illustrates the advantages of ECDF plots vs. rank histograms as well as ECDF diff plots vs. standard ECDF plots.
The text was updated successfully, but these errors were encountered: