Layer-Wise Relevance Propagation

PyTorch implementation of Layer-wise Relevance Propagation (LRP) algorithm together with quantitative evaluation metrics to compare heatmap explanations objectively part of master's thesis In-Depth Hyperparameter Selection For Layer-Wise Relevance Propagation at TU Berlin.

Special thanks to Dr. Grégoire Montavon for his insights, which shaped the development of this project.

Features

Explainability:

• Layer-wise Relevance Propagation (LRP)

Quantitative Evaluation:

• Pixel Flipping (PF), also known as Region Perturbation (RP)
  • Perturbation modes:
    • inpainting
    • random
  • Sort objectives:
    • most relevant first (MoRF), also known as activation curve
    • least relevant first (LRF), also known as pruning curve
    • random
• AUC (Area Under the Curve), also known as AUAC (Area Under the Activation Curve) or AU-MSE (Area Under the Mean Squared Error Curve) depending on the sort objective

Showcase

LRP (lrp-tutorial composite) and PF with inpainting perturbation mode and sort objective MoRF.

Classification scores of castle image with inpainting perturbation mode

Number of simultaneous flips per perturbation step during Pixel-Flipping.

---

PF perturbation mode random

LRP (lrp-tutorial composite) and PF with random perturbation mode and sort objective MoRF.

Classification scores of castle image with random perturbation mode

Requirements

• python3 >= 3.9

Installation

python3 -m pip install lrp-pf-auc

The PyPI distribution lrp-pf-auc provides the following two packages: lrp and pf.

The name lrp-pf-auc stands for Layer-wise Relevance Propagation (LRP), Pixel Flipping (PF), and Area Under the Curve (AUC) respectively.

Usage

Refer to demo.ipynb for an example of Layer-wise Relevance Propagation (LRP), Pixel-Flipping (PF) and Area under the Curve (AUC).

Feel free to check out the Jupyter notebooks under experiments/notebooks for a chronological overview of the project.

Related Projects

• Sequential LRP implementation: gmontavon/lrp-tutorial

  Tutorial on how to implement LRP

• Updated version of gmontavon/lrp-tutorial: rodrigobdz/lrp-tutorial
• Forward-hook LRP implementation: chr5tphr/zennit

  Implementation of LRP-based methods in PyTorch

• innvestigate-based LRP implementation: moboehle/Pytorch-LRP
• Caffe-based LRP implementation: lrp_toolbox
• Pixel-Flipping and Region Perturbation implementation: understandable-machine-intelligence-lab/Quantus

Citation

Cite as:

• Plaintext:

  Rodrigo Bermúdez Schettino. (2022). rodrigobdz/lrp: v0.1.6 (v0.1.6). Zenodo. https://doi.org/10.5281/zenodo.6821295

• BibTeX:

  @software{rodrigo_bermudez_schettino_2022_6821295,
    author       = {Rodrigo Bermúdez Schettino},
    title        = {rodrigobdz/lrp: v0.1.6},
    month        = jul,
    year         = 2022,
    publisher    = {Zenodo},
    version      = {v0.1.6},
    doi          = {10.5281/zenodo.6821295},
    url          = {https://doi.org/10.5281/zenodo.6821295}
  }
  

Credits

• The structure of this readme is based on minimal-readme

• Scripts follow Personal Shell Style Guide

• The lrp package uses two customized files originally from chr5tphr/zennit.

• The syntax for defining custom composites is inspired by this discussion on zennit's repo.

This implementation is based on insights from:

• LRP overview paper

  G. Montavon, A. Binder, S. Lapuschkin, W. Samek, K.-R. Müller Layer-wise Relevance Propagation: An Overview in Explainable AI: Interpreting, Explaining and Visualizing Deep Learning, Springer LNCS, vol. 11700, 2019

• Original LRP paper

  S. Bach, A. Binder, G. Montavon, F. Klauschen, K.-R. Müller, W. Samek On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation PloS ONE 10 (7), e0130140, 2015

• ECML/PKDD 2020 Tutorial: Explainable AI for Deep Networks: Basics and Extensions (Part 3)

License

MIT © rodrigobdz