Learning Stochastic Majority Votes by Minimizing a PAC-Bayes Generalization Bound

Dependencies

Running with python3.6.

Install PyTorch, following the guidelines. Then install the requirements:

pip3 install -r requirements.txt

Train on toy datasets

python3 toy.py

Default configuration is stored in 'config/toy.yaml'. You can edit directly the config file or change values from the command line, e.g. as follows:

python3 toy.py dataset.N_train=1000 dataset.noise=0.1 model.M=16

See Hydra for a tutorial.

Reproduce main results

To reproduce the main results of the paper on real benchmarks, run:

bash real.sh

You can also run a specific experiment, passing the chosen values for the hyper-parameters as follows:

python3 real.py dataset=SENSORLESS model.M=100 training.risk=MC model.pred=rf model.prior=2 model.tree_depth=5

Minimal script

To run a simplified script that supports only the optimization of the proposed "exact" and "MC" bounds, run:

python3 minimal.py exact
python3 minimal.py MC

Bibtex

If you find this work useful, please cite:

@article{zantedeschi2021learning,
  title={Learning Stochastic Majority Votes by Minimizing a PAC-Bayes Generalization Bound},
  author={Zantedeschi, Valentina and Viallard, Paul and Morvant, Emilie and Emonet, R{\'e}mi and Habrard, Amaury and Germain, Pascal and Guedj, Benjamin},
  journal={NeurIPS},
  year={2021}
}