lecture_8

This lecture:

Introduction to information theory.

Information bottleneck method (https://arxiv.org/abs/physics/0004057).

Connection between minimization of infobottleneck lagrangian and learning sufficient statistics (http://www.cs.huji.ac.il/labs/learning/Papers/ibgen.pdf).

Phases of learning (https://arxiv.org/abs/1703.00810), critics (https://openreview.net/forum?id=ry_WPG-A-&noteId=ry_WPG-A-).

No slides this time; video.

Next lecture:

Representation learning, cross-entropy decomposition (https://arxiv.org/abs/1706.01350).