This project aims to address the multi-label black-box outcome explanation problem. MARLENA (Multi-lAbel RuLe-based ExplaNAtions) is based on our paper:
@inproceedings{panigutti2019explaining,
title={Explaining multi-label black-box classifiers for health applications},
author={Panigutti, Cecilia and Guidotti, Riccardo and Monreale, Anna and Pedreschi, Dino},
booktitle={International Workshop on Health Intelligence},
pages={97--110},
year={2019},
organization={Springer}
}
You can run the experiments of the paper following the instructions provided here.
-
Install the requirements.txt listed packages
-
Install marlena: In order to run MARLENA you first have to locally install the pyhton module marlena. You can do this by running the following command into the module directory:
$ cd marlena
$ pip install .
Here an example on how to use MARLENA.
%matplotlib inline
import numpy as np
import pandas as pd
import matplotlib.pyplot as pltAs a toy example we will use the make_multilabel_classification
from sklearn.datasets import make_multilabel_classification
n_classes=2
n_features=2
avg_n_labels_per_instance = 1
X, Y = make_multilabel_classification(n_samples=200,
n_features=n_features,
n_classes=n_classes,
n_labels=avg_n_labels_per_instance,
length=50,
allow_unlabeled=False,
sparse=False,
return_indicator='dense',
return_distributions=False,
random_state=0)In this toy example we will explain the decision of a OneVsRest-Adaboost classifier
from sklearn.model_selection import train_test_split
from sklearn.multiclass import OneVsRestClassifier
from sklearn.ensemble import AdaBoostClassifier
from sklearn.metrics import f1_score
X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.20, random_state=42)
bb = OneVsRestClassifier(AdaBoostClassifier(n_estimators=2000, random_state=0))
bb.fit(X_train, y_train)
y_pred = bb.predict(X_test)
print(f'prediction performance of the black-box, F1-score: {round(f1_score(y_test, y_pred, average="macro"),3)}')prediction performance of the black-box, F1-score: 0.789
Suppose we want to explain the black box decision of this instance:
i2e_id = 10
instance_to_be_explained = X[i2e_id].reshape(1, -1)
bb.predict(instance_to_be_explained)array([[1, 1]])
Since this is a 2D example we can visualize the points and their classification according to the black box.
The one we want to explain is annotated using the black arrow
x_min, x_max = X[:, 0].min() - .5, X[:, 0].max() + .5
y_min, y_max = X[:, 1].min() - .5, X[:, 1].max() + .5
h = .02
xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))
colors = ['#F07818', '#45CCA1']
sizes = [40, 120]
plt.figure(figsize=(6,5))
plt.scatter(X[:, 0], X[:, 1], s=5, c='gray')
labels = ['Class1','Class2']
Y_bb = bb.predict(X)
for i in range(0, n_classes):
plt.scatter(X[np.where(Y_bb[:, i]), 0], X[np.where(Y_bb[:, i]), 1], s=sizes[i], edgecolors=colors[i],
facecolors='none', linewidths=2, label='class %d' % (i+1))
plt.annotate("", xy=(X[i2e_id][0], X[i2e_id][1]), xytext=(35, 35), arrowprops=dict(facecolor='black', shrink=0.05))
plt.xlim(xx.min(), xx.max())
plt.ylim(yy.min(), yy.max())
plt.xticks(())
plt.yticks(())
plt.xlabel(r'$X_0$', fontsize=16)
plt.ylabel(r'$X_1$', fontsize=16)
plt.legend(loc=(0,1), fontsize=16, ncol=3, labelspacing=0, handlelength=0.2, frameon=False)
plt.tick_params(axis='both', which='major', labelsize=16)
from marlena import marlenaIstantiate the MARLENA object
m1 = marlena.MARLENA(neigh_type='mixed',random_state=42)To extract the explanation, MARLENA needs:
- the black box
bb - the instance whose black box decision we want to explain
i2e = instance_to_be_explained - the name of the classes
labels_name = ['class1', 'class2'] - the names of categorical and numerical variables, in our case they are all numerical:
numerical_vars = ['feature_one','feature_two'],categorical_vars = [] - a set of instances representative of the dataset that was used to train the black box, e.g. the training set
X2E = X_train
labels_name = ['class1', 'class2']
numerical_vars = ['feature_one','feature_two']
categorical_vars = []
i2e = pd.Series(instance_to_be_explained[0],index=numerical_vars)
X2E = pd.DataFrame(X_train,columns=numerical_vars)- k: number of real neighbors to use to generate the synthetic ones
- alpha: fraction of neighbors to be sampled from the feature space
- k_synth: number of synthetic neighbors
k = 10
k_synt = 500
alpha = 0.7rule, instance_imporant_feat, fidelity, hit, DT = m1.extract_explanation(i2e,
X2E,
bb,
numerical_vars,
categorical_vars,
labels_name,
k=k,
size=k_synt,
alpha=alpha)MARLENA provides rule-based explanations. The explanation is the decision tree path from root to leaf that matches the features of the instance whose black-box classification we want to explain.
print(rule){feature_two <= 23.03,
feature_one > 24.0,
feature_two > 16.53,
feature_two > 17.49,
feature_one > 24.46,
feature_one <= 35.46,
feature_two <= 21.51,
feature_two <= 20.43,
feature_two <= 20.39,
feature_two > 18.7,
feature_two > 18.75,
feature_two <= 19.48,
feature_two <= 19.44,
feature_two <= 19.08,
feature_one <= 31.32,
feature_two > 18.88,
feature_one <= 28.26,
feature_two <= 19.03} -> ['class1' 'class2']
Together with the rule-based explanation, MARLENA also provides some metrics to evaluate the quality of the explanation: fidelity and hit.
The fidelity compares the decisions of the DT to those of the black-box on the synthetic neighborhood. It is measured using the F1-score.
The hit compares the predictions of the DT and the black-box on the instance x under analysis. It is measured using the simple match similarity
print(fidelity)1.0
print(hit)1.0