Add doc for interpretability

doc/reference.rst

@@ -27,6 +27,7 @@ Private Module Reference
  econml._ortho_learner
  econml._cate_estimator
  econml._causal_tree
+ econml._shap
  econml.dml._rlearner
  econml.grf._base_grf
  econml.grf._base_grftree

doc/spec/interpretability.rst

@@ -0,0 +1,95 @@
+Interpretability
+================
+
+Our package offers multiple interpretability tools to better understand the final model CATE.
+
+
+Tree Interpreter
+----------------
+
+Tree Interpreter provides a presentation-ready summary of the key features that explain the biggest differences in responsiveness to an intervention.
+
+:class:`.SingleTreeCateInterpreter` trains a single shallow decision tree for the treatment effect :math:`\theta(X)` you learned from any of
+our available CATE estimators on a small set of feature :math:`X` that you are interested to learn heterogeneity from. The model will split on the cutoff
+points that maximize the treatment effect difference in each leaf. Finally each leaf will be a subgroup of samples that respond to a treatment differently
+from other leaves. 
+
+For instance: 
+
+.. testsetup::
+
+ import numpy as np
+ X = np.random.choice(np.arange(5), size=(100,3))
+ Y = np.random.normal(size=(100,2))
+ y = np.random.normal(size=(100,))
+ T = np.random.choice(np.arange(3), size=(100,2))
+ t = T[:,0]
+ W = np.random.normal(size=(100,2))
+
+
+.. testcode::
+
+ from econml.cate_interpreter import SingleTreeCateInterpreter
+ from econml.dml import LinearDML
+ est = LinearDML()
+ est.fit(y, t, X=X, W=W)
+ intrp = SingleTreeCateInterpreter(include_model_uncertainty=True, max_depth=2, min_samples_leaf=10)
+ # We interpret the CATE model's behavior based on the features used for heterogeneity
+ intrp.interpret(est, X)
+ # Plot the tree
+ intrp.plot(feature_names=['A', 'B', 'C'], fontsize=12)
+
+Policy Interpreter
+------------------
+Policy Interpreter offers similar functionality but taking cost into consideration. 
+
+Instead of training a tree model regressor on :math:`\theta(X)`, :class:`.SingleTreePolicyInterpreter` trains a tree model classifier by using whether
+the effect is above the cost as label. This results in simple rules to segment the samples in order to maximize the outcome of interest.
+
+
+For instance: 
+
+.. testcode::
+
+ from econml.cate_interpreter import SingleTreePolicyInterpreter
+ # We find a tree-based treatment policy based on the CATE model
+ # sample_treatment_costs is the cost of treatment. Policy will treat if effect is above this cost.
+ intrp = SingleTreePolicyInterpreter(risk_level=0.05, max_depth=2, min_samples_leaf=1,min_impurity_decrease=.001)
+ intrp.interpret(est, X, sample_treatment_costs=0.05)
+ # Plot the tree
+ intrp.plot(feature_names=['A', 'B', 'C'], fontsize=12)
+
+
+SHAP
+----
+
+`SHAP <https://shap.readthedocs.io/en/latest/>`_ is a popular open source library for interpreting black-box machine learning
+models using the Shapley values methodology (see e.g. [Lundberg2017]_).
+
+Similar to how black-box predictive machine learning models can be explained with SHAP, we can also explain black-box effect
+heterogeneity models. This approach provides an explanation as to why a heterogeneous causal effect model produced larger or
+smaller effect values for particular segments of the population. Which were the features that lead to such differentiation?
+This question is easy to address when the model is succinctly described, such as the case of linear heterogneity models, 
+where one can simply investigate the coefficients of the model. However, it becomes hard when one starts using more expressive
+models, such as Random Forests and Causal Forests to model effect hetergoeneity. SHAP values can be of immense help to
+understand the leading factors of effect hetergoeneity that the model picked up from the training data.
+
+Our package offers seamless integration with the SHAP library. Every CATE estimator has a method `shap_values`, which returns the
+SHAP value explanation of the estimators output for every treatment and outcome pair. These values can then be visualized with
+the plethora of visualizations that the SHAP library offers. Moreover, whenever possible our library invokes fast specialized
+algorithms from the SHAP library, for each type of final model, which can greatly reduce computation times.
+
+For instance:
+
+.. testcode::
+
+ import shap
+ from econml.dml import CausalForestDML
+ est = CausalForestDML()
+ est.fit(Y, T, X=X, W=W)
+ shap_values = est.shap_values(X)
+ # local view: explain hetergoeneity for a given observation
+ ind=0
+ shap.plots.force(shap_values["Y0"]["T0"][ind], matplotlib=True)
+ # global view: explain hetergoeneity for a sample of dataset
+ shap.summary_plot(shap_values['Y0']['T0'])

doc/spec/references.rst

@@ -108,4 +108,9 @@ References
 .. [Friedberg2018]
  Friedberg, R., Tibshirani, J., Athey, S., & Wager, S. (2018).
  Local linear forests.
- arXiv preprint arXiv:1807.11408.
+ arXiv preprint arXiv:1807.11408.
+
+.. [Lundberg2017]
+ Lundberg, S., Lee, S. (2017).
+ A Unified Approach to Interpreting Model Predictions.
+ URL https://arxiv.org/abs/1705.07874

doc/spec/spec.rst

@@ -20,6 +20,7 @@ The EconML Python SDK, developed by the ALICE team at MSR New England, incorpora
  estimation
  estimation_iv
  inference
+ interpretability
  references
 
 .. todo::

econml/_shap.py

@@ -1,6 +1,17 @@
 # Copyright (c) Microsoft Corporation. All rights reserved.
 # Licensed under the MIT License.
 
+"""Helper functions to get shap values for different cate estimators.
+
+References
+----------
+Scott Lundberg, Su-In Lee (2017)
+ A Unified Approach to Interpreting Model Predictions.
+ NeurIPS, https://arxiv.org/abs/1705.07874
+
+
+"""
+
 import shap
 from collections import defaultdict
 import numpy as np