evidence_pop

Parallelised non-parametric inference of evidence $Z$ using posterior samples. This code implements the method described in Rinaldi et al. (2024). In a nutshell, the inference scheme follows these steps:

1. Approximate the posterior distribution $P(x)$ with a DPGMM;
2. Evaluate $p(Z(x_i))$ for different samples $x_i$ and approximate each of these with a DPGMM;
3. Reconstruct $p(Z)$ using (H)DPGMM.

The (H)/DPGMM reconstructions are made using FIGARO (Rinaldi & Del Pozzo 2024).

Getting started

You can install evidence_pop from this repository:

git clone git@github.com:sterinaldi/evidence_pop.git
cd evidence_pop
pip install .

evidence_pop is meant to be used via its CLI, evidence. The complete list of options can be displayed with evidence -h: here we summarise the most important ones:

• -s path/to/samples.txt: file storing the posterior samples $x_i$;
• -l path/to/log_p.txt: file storing the evaluated $\log\Pi(x_i) + \log\mathcal{L}(x_i)$;
• -b "[[xmin, xmax],[ymin, ymax], ...]": domain bounds for $P(x)$ (remember the quotation marks);
• --n_parallel N: use N parallel threads for the analysis
• -r: skip the $P(x)$ reconstruction (if already done);
• -z: skip the individual $p(Z(x_i))$ reconstructions (if already done);
• -p: postprocessing;
• --save_posterior: produce a plot with the inferred $P(x)$ (it may take a long time for more than 3 dimensions).

Assuming that the folder you are working in is

folder
├─ samples.txt
└─ log_p.txt

the basic instruction for running the evidence inference from start to finish looks somewhat like this:

evidence -s samples.txt -l log_p.txt -b "[[0,1],[0,1]]"

Once the run is finished, the folder will look like this:

folder
├─ draws_evidence.json
├─ draws_samples.json
├─ evidence.pdf (optional)
├─ evidence.txt
├─ log_evidence.txt
├─ options.ini
├─ prob_evidence.txt (optional)
├─ prob_log_evidence.txt
├─ realisations_Zi.json
├─ samples_Z.txt
├─ samples.pdf (optional)
├─ samples.txt
└─ log_p.txt

Among these files, the most important ones are:

• log_evidence.pdf: plot showing the posterior distribution for $Z$;
• evidence.txt: stores the expected value and credible interval for $Z$;
• draws_evidence.json: stores the FIGARO reconstructions for $p(Z)$ (see here for a tutorial on how to use them).

Acknowledgments

If you use this code in your research, please cite Rinaldi et al. (2024):

@ARTICLE{2024arXiv240507504R,
       author = {{Rinaldi}, Stefano and {Demasi}, Gabriele and {Del Pozzo}, Walter and {Hannuksela}, Otto A.},
        title = "{Hierarchical inference of evidence using posterior samples}",
      journal = {arXiv e-prints},
     keywords = {Statistics - Methodology, 62F15, 62C10},
         year = 2024,
        month = may,
          eid = {arXiv:2405.07504},
        pages = {arXiv:2405.07504},
          doi = {10.48550/arXiv.2405.07504},
archivePrefix = {arXiv},
       eprint = {2405.07504},
 primaryClass = {stat.ME},
       adsurl = {https://ui.adsabs.harvard.edu/abs/2024arXiv240507504R},
      adsnote = {Provided by the SAO/NASA Astrophysics Data System}
}