AryaXAI: DLBacktrace

This repository contains the source code for the paper: DLBacktrace: A Model Agnostic Explainability for any Deep Learning Models.

Introducing DLBacktrace, a method for analyzing neural networks by tracing the relevance of each component from output to input, to understand how each part contributes to the final prediction. Developed by AryaXAI Team. It offers two modes: Default and Contrast, and is compatible with TensorFlow and PyTorch.

Overview

The DlBacktrace Module, a patent-pending algorithm developed by AryaXAI, is designed to significantly enhance the explainability of Deep Learning AI models.

Features

• Explainability: Gain deep insights into your AI models by using the Backtrace algorithm, providing multiple explanations for their decisions.

• Consistency: Ensure consistent and accurate explanations across different scenarios and use cases.

• Mission-Critical Support: Tailored for mission-critical AI use cases where transparency is paramount.

Installation

To integrate the Backtrace Module into your project, follow these simple steps:

pip install dl-backtrace

Usage

Tensoflow-Keras based models

from dl_backtrace.tf_backtrace import Backtrace as B

Pytorch based models

from dl_backtrace.pytorch_backtrace import Backtrace as B

Evalauting using Backtrace:

1. Step - 1: Initialize a Backtrace Object using your Model

backtrace = B(model=model)

2. Step - 2: Calculate layer-wise output using a data instance

layer_outputs = backtrace.predict(test_data[0])

3. Step - 3: Calculate layer-wise Relevance using Evaluation

relevance = backtrace.eval(layer_outputs,mode='default',scaler=1,thresholding=0.5,task="binary-classification")

Depending on Task we have several attributes for Relevance Calculation in Evalaution:

Attribute	Description	Values
mode	evaluation mode of algorithm	{ default, contrastive}
scaler	Total / Starting Relevance at the Last Layer	Integer ( Default: None, Preferred: 1)
thresholding	Thresholding Model Prediction in Segemntation Task to select Pixels predicting the actual class. (Only works in Segmentation Tasks)	Default:0.5
task	The task of the Model	{ binary-classification, multi-class classification, bbox-regression, binary-segmentation}
model-type	Type of the Model	{Encoder}

Example Notebooks :

Tensorflow-Keras :

Name	Task	Link
Backtrace Loan Classification Tabular Dataset	Binary Classification	Colab Link
Backtrace Image FMNIST Dataset	Multi-Class Classification	Colab Link
Backtrace CUB Bounding Box Regression Image Dataset	Single Object Detection	Colab Link
Backtrace Next Word Generation Textual Dataset	Next Word Generation	Colab Link
Backtrace ImDB Sentiment Classification Textual Dataset	Sentiment Classification	Colab Link
Backtrace Binary Classification Textual Dataset	Binary Classification	Colab Link
Backtrace Multi-Class NewsGroup20 Classification Textual Dataset	Multi-Class Classification	Colab Link
Backtrace CVC-ClinicDB Colonoscopy Binary Segmentation	Organ Segmentation	Colab Link
Backtrace CamVid Road Car Binary Segmentation	Binary Segmentation	Colab Link
Backtrace Transformer Encoder for Sentiment Analysis	Binary Classification	Colab Link

Pytorch :

Name	Task	Link
Backtrace Tabular Dataset	Binary Classification	Colab Link
Backtrace Image Dataset	Multi-Class Classification	Colab Link

Supported Layers and Future Work :

Tensorflow-Keras:

• Dense (Fully Connected) Layer
• Convolutional Layer (Conv2D,Conv1D)
• Transpose Convolutional Layer (Conv2DTranspose,Conv1DTranspose)
• Reshape Layer
• Flatten Layer
• Global Max Pooling (2D & 1D) Layer
• Global Average Pooling (2D & 1D) Layer
• Max Pooling (2D & 1D) Layer
• Average Pooling (2D & 1D) Layer
• Concatenate Layer
• Add Layer
• Long Short-Term Memory (LSTM) Layer
• Dropout Layer
• Embedding Layer
• TextVectorization Layer
• Self-Attention Layer
• Cross-Attention Layer
• Feed-Forward Layer
• Pooler Layer
• Decoder LM (Language Model) Head
• Other Custom Layers

Pytorch :

(Note: Currently we only Support Binary and Multi-Class Classification in Pytorch, Segmentation and Single Object Detection will be supported in the next release.)

• Linear (Fully Connected) Layer
• Convolutional Layer (Conv2D)
• Reshape Layer
• Flatten Layer
• Global Average Pooling 2D Layer (AdaptiveAvgPool2d)
• Max Pooling 2D Layer (MaxPool2d)
• Average Pooling 2D Layer (AvgPool2d)
• Concatenate Layer
• Add Layer
• Long Short-Term Memory (LSTM) Layer
• 1d Convolution Layer (Conv1d)
• 1d Pooling Layers (AvgPool1d,MaxPool1d,AdaptiveAvgPool1d,AdaptiveMaxPool1d)
• Transpose Convolution Layers (ConvTranspose2d,ConvTranspose1d)
• Global Max Pooling 2D Layer (AdaptiveMaxPool2d)
• Dropout Layer
• Embedding Layer
• EmbeddingBag Layer
• Other Custom Layers

Limitations

Limitations for both TensorFlow and PyTorch

1. Custom Layers Not Supported: The implementation of custom layers is currently not supported in the framework.
2. Model Requirements: All models must be developed entirely from scratch, adhering to the framework's specifications.
3. Layer Abstraction: Custom layers using class-based abstractions are not supported. Instead, layers should be directly utilized within tf.Sequential for TensorFlow and nn.Sequential for PyTorch.

Additional Limitations Torch

1. Torch Functional Not Supported: The use of torch.functional for activation functions is not supported. Instead, define them as attributes in the constructor.
2. Single Function Per Line: Avoid using multiple functions in a single line within the forward pass.
3. Tensor Operations in Forward Pass: Operations like transpose and other tensor manipulations should not be used directly in the forward pass.

We hope to address these limitations soon in the future versions.

Getting Started

If you're new to Backtrace, check out our Example Notebooks for a quick and comprehensive guide covering a variety of tasks.

Contributing

We welcome contributions from the community.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Citations

This code is free. So, if you use this code anywhere, please cite us:

@misc{sankarapu2024dlbacktracemodelagnosticexplainability,
      title={DLBacktrace: A Model Agnostic Explainability for any Deep Learning Models}, 
      author={Vinay Kumar Sankarapu and Chintan Chitroda and Yashwardhan Rathore and Neeraj Kumar Singh and Pratinav Seth},
      year={2024},
      eprint={2411.12643},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2411.12643}, 
}

Get in touch

Contanct us at AryaXAI.