MI-NEE

Mutual Information Neural Entropic Estimation.

• Presentation slides
• arXiv paper

Dependencies

See binder/requirements.txt

Experimental results in Jupyter notebook

• Mixed Gaussian with MINE
• Mixed Gaussian with MI-NEE
• Mixed Gaussian with cross training by MI-NEE followed by MINE
• Higher dimensional Gaussian with MINE
• Higher dimensional Gaussian with MI-NEE
• Higher dimensional Gaussian with crossing training by MI-NEE followed by MINE
• Other experimental results

Directory structure

• data folder contains the code that generates samples for the experiments.
  • mix_gaussian.py contains the code for the mixed Gaussian distribution.
  • gaussian.py contains the code for gaussian distribution.
• model folder contains the code of the mutual information estimators.
  • mine.py contains the code for MINE.
  • minee.py contains the code for MI-NEE.
  • minee_mine.py contains the code for cross training by MI-NEE followed by MINE.

Reference

Chung Chan, Ali Al-Bashabsheh, Hing Pang Huang, Michael Lim, Da Sun Handason Tam, and Chao Zhao. "Neural Entropic Estimation: A faster path to mutual information estimation." arXiv preprint arXiv:1905.12957 (2019).