Skip to content
/ PSIS Public

Pareto smoothed importance sampling (PSIS) and PSIS leave-one-out cross-validation for Python and Matlab/Octave

License

Notifications You must be signed in to change notification settings

avehtari/PSIS

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

33 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Pareto smoothed importance sampling (PSIS) and PSIS leave-one-out cross-validation reference code

Introduction

These files implement Pareto smoothed importance sampling (PSIS) and PSIS leave-one-out cross-validation for Matlab/Octave and Python (Python port made by Tuomas Sivula).

These code are not maintained and are here for historical reference. Instead of these, use well maintained implementations available for R, Python, and Julia as listed below.

R

  • PSIS and PSIS-LOO are implemented in the loo R package, which is also available from CRAN.
  • PSIS and all Pareto $\hat{k}$ diagnostics are implemented in the posterior R package, which is also available from CRAN.

Python

  • PSIS, PSIS-LOO, and Pareto $\hat{k}$ diagnostics are implemented in the ArviZ.py package.

  • In this repo

    • 'psis.py' - Includes the following functions in a Python (Numpy) module
    • psislw - Pareto smoothing of the log importance weights
    • psisloo - Pareto smoothed importance sampling leave-one-out log predictive densities
    • gpdfitnew - Estimate the paramaters for the Generalized Pareto Distribution
    • gpinv - Inverse Generalised Pareto distribution function.
    • sumlogs - Sum of vector where numbers are represented by their logarithms

Julia

  • PSIS, PSIS-LOO, and Pareto $\hat{k}$ diagnostics are implemented in the ArviZ.jl package.

Matlab/Octave

  • In this repo
    • 'psislw.m' - Pareto smoothing of the log importance weights
    • 'psisloo.m' - Pareto smoothed importance sampling leave-one-out log predictive densities
    • 'gpdfitnew.m' - Estimate the paramaters for the Generalized Pareto Distribution
    • 'sumlogs.m' - Sum of vector where numbers are represented by their logarithms

References

  • Aki Vehtari, Andrew Gelman and Jonah Gabry (2017). Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Statistics and Computing, 27(5):1413–1432. doi:10.1007/s11222-016-9696-4. arXiv preprint arXiv:1507.04544
  • Aki Vehtari, Daniel Simpson, Andrew Gelman, Yuling Yao, and Jonah Gabry (2024). Pareto smoothed importance sampling. Journal of Machine Learning Research, accepted for publication. arXiv preprint arXiv:1507.02646
  • Jin Zhang & Michael A. Stephens (2009) A New and Efficient Estimation Method for the Generalized Pareto Distribution, Technometrics, 51:3, 316-325, DOI: 10.1198/tech.2009.08017

About

Pareto smoothed importance sampling (PSIS) and PSIS leave-one-out cross-validation for Python and Matlab/Octave

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published