From 8d4ed8cf3553b35f39b3b5142fb85610157079a1 Mon Sep 17 00:00:00 2001 From: Clive Cox Date: Thu, 25 Mar 2021 09:09:43 +0000 Subject: [PATCH] Update alibi docs --- doc/source/analytics/explainers.md | 15 ++++++++++ doc/source/examples/notebooks.rst | 2 +- doc/source/python/api/seldon_core.rst | 16 +++++------ notebooks/explainer_examples.ipynb | 40 ++++++++++++++------------- 4 files changed, 45 insertions(+), 28 deletions(-) diff --git a/doc/source/analytics/explainers.md b/doc/source/analytics/explainers.md index ddc2aed958..b4830cd3ee 100644 --- a/doc/source/analytics/explainers.md +++ b/doc/source/analytics/explainers.md @@ -7,6 +7,21 @@ Seldon provides model explanations using its [Alibi](https://github.com/SeldonIO We provide [an example notebook](../examples/explainer_examples.html) showing how to deploy an explainer for Tabular, Text and Image models. + +## Available Methods + +Seldon Core supports a subset of the methods currently available in [Alibi](https://github.com/SeldonIO/alibi). Presently this the following: + + +| Method | Explainer Key | Example | +|--------|---------------|---------|| +| [Anchor Tabular](https://docs.seldon.io/projects/alibi/en/latest/methods/Anchors.html) | `AnchorTabular` | [Example](https://docs.seldon.io/projects/seldon-core/en/latest/examples/explainer_examples.html#Income-Prediction-Model-with-Anchors-Explainer) | +| [Anchor Text](https://docs.seldon.io/projects/alibi/en/latest/methods/Anchors.html) | `AnchorText` | [Example](https://docs.seldon.io/projects/seldon-core/en/latest/examples/explainer_examples.html#Movie-Sentiment-Model) | +| [Anchor Images](https://docs.seldon.io/projects/alibi/en/latest/methods/Anchors.html) | `AnchorImages` | [Example](https://docs.seldon.io/projects/seldon-core/en/latest/examples/explainer_examples.html#Tensorflow-CIFAR10-Model) | +| [kernel Shap](https://docs.seldon.io/projects/alibi/en/latest/methods/KernelSHAP.html) | `KernelShap` | [Example](https://docs.seldon.io/projects/seldon-core/en/latest/examples/explainer_examples.html#Wine-Prediction-Model-with-Shap-Explainer) | +| [Integrated Gradients](https://docs.seldon.io/projects/alibi/en/latest/methods/IntegratedGradients.html) | `IntegratedGradients` | [Example](https://docs.seldon.io/projects/seldon-core/en/latest/examples/explainer_examples.html#MNIST-Model-with-Integrated-Gradients-Explainer) | +| [Tree Shap](https://docs.seldon.io/projects/alibi/en/latest/methods/TreeSHAP.html) | `TreeShap` | [Example](https://docs.seldon.io/projects/seldon-core/en/latest/examples/explainer_examples.html#XGBoost-Model-with-TreeShap-Explainer) | + ## Creating your explainer For Alibi explainers that need to be trained you should diff --git a/doc/source/examples/notebooks.rst b/doc/source/examples/notebooks.rst index 705f1ac596..618d85bb9b 100644 --- a/doc/source/examples/notebooks.rst +++ b/doc/source/examples/notebooks.rst @@ -83,7 +83,7 @@ Advanced Machine Learning Insights :titlesonly: Real time monitoring of statistical metrics - Tabular, Text and Image Model Explainers + Model Explainers Outlier Detection on CIFAR10 Batch Processing with Seldon Core diff --git a/doc/source/python/api/seldon_core.rst b/doc/source/python/api/seldon_core.rst index 1cad136ed7..7211fd01ff 100644 --- a/doc/source/python/api/seldon_core.rst +++ b/doc/source/python/api/seldon_core.rst @@ -25,14 +25,6 @@ seldon\_core.api\_tester module :undoc-members: :show-inheritance: -seldon\_core.app module ------------------------ - -.. automodule:: seldon_core.app - :members: - :undoc-members: - :show-inheritance: - seldon\_core.batch\_processor module ------------------------------------ @@ -49,6 +41,14 @@ seldon\_core.flask\_utils module :undoc-members: :show-inheritance: +seldon\_core.gunicorn\_utils module +----------------------------------- + +.. automodule:: seldon_core.gunicorn_utils + :members: + :undoc-members: + :show-inheritance: + seldon\_core.imports\_helper module ----------------------------------- diff --git a/notebooks/explainer_examples.ipynb b/notebooks/explainer_examples.ipynb index d75cdc69c4..0ea7c8968c 100644 --- a/notebooks/explainer_examples.ipynb +++ b/notebooks/explainer_examples.ipynb @@ -5,29 +5,31 @@ "metadata": {}, "source": [ "# Example Model Explanations with Seldon\n", + "\n", "Seldon core supports various out-of-the-box explainers that leverage the [Alibi ML Expalinability](https://github.com/SeldonIO/alibi) open source library.\n", "\n", "In this notebook we show how you can use the pre-packaged explainer functionality that simplifies the creation of advanced AI model explainers.\n", "\n", "Seldon provides the following out-of-the-box pre-packaged explainers:\n", - "* Anchor Tabular Explainer \n", - " * A black box Explainer that uses the [anchor technique](https://docs.seldon.io/projects/alibi/en/latest/methods/Anchors.html) for tabular data\n", - " * It basically answers the question of what are the most \"powerul\" or \"important\" features in a tabular prediction\n", - "* Anchor Image Explainer\n", - " * A black box Explainer that uses the [anchor technique](https://docs.seldon.io/projects/alibi/en/latest/methods/Anchors.html) for image data\n", - " * It basically answers the question of what are the most \"powerul\" or \"important\" pixels in an image prediction\n", - "* Anchor Text Explainer\n", - " * A black box Explainer that uses the [anchor technique](https://docs.seldon.io/projects/alibi/en/latest/methods/Anchors.html) for text data\n", - " * It basically answers the question of what are the most \"powerul\" or \"important\" tokens in a text prediction\n", - "* Kernel Shap Explainer\n", - " * A black box Explainer that uses the [kernel shap technique](https://docs.seldon.io/projects/alibi/en/latest/methods/Anchors.html) for tabular data\n", - " * It provides postive and negative feature attributions that contributed to the predictions\n", - "* Integrated Gradient Explainer\n", - " * A white box explainer that uses the [Integrated Gradients technique](https://docs.seldon.io/projects/alibi/en/latest/methods/Anchors.html) for Keras models\n", - " * It provides importance values for each feature\n", - "* Tree Shap Explainer\n", - " * A white box explainer that uses the [TreeShap technqiue](https://docs.seldon.io/projects/alibi/en/latest/methods/Anchors.html) for tree based models\n", - " * It provides positive and negative feature attributions that contributed to the predictions" + "\n", + " - Anchor Tabular Explainer \n", + " - A black box Explainer that uses the [anchor technique](https://docs.seldon.io/projects/alibi/en/latest/methods/Anchors.html) for tabular data\n", + " - It basically answers the question of what are the most \"powerul\" or \"important\" features in a tabular prediction\n", + " - Anchor Image Explainer\n", + " - A black box Explainer that uses the [anchor technique](https://docs.seldon.io/projects/alibi/en/latest/methods/Anchors.html) for image data\n", + " - It basically answers the question of what are the most \"powerul\" or \"important\" pixels in an image prediction\n", + " - Anchor Text Explainer\n", + " - A black box Explainer that uses the [anchor technique](https://docs.seldon.io/projects/alibi/en/latest/methods/Anchors.html) for text data\n", + " - It basically answers the question of what are the most \"powerul\" or \"important\" tokens in a text prediction\n", + " - Kernel Shap Explainer\n", + " - A black box Explainer that uses the [kernel shap technique](https://docs.seldon.io/projects/alibi/en/latest/methods/Anchors.html) for tabular data\n", + " - It provides postive and negative feature attributions that contributed to the predictions\n", + " - Integrated Gradient Explainer\n", + " - A white box explainer that uses the [Integrated Gradients technique](https://docs.seldon.io/projects/alibi/en/latest/methods/Anchors.html) for Keras models\n", + " - It provides importance values for each feature\n", + " - Tree Shap Explainer\n", + " - A white box explainer that uses the [TreeShap technique](https://docs.seldon.io/projects/alibi/en/latest/methods/Anchors.html) for tree based models\n", + " - It provides positive and negative feature attributions that contributed to the predictions" ] }, { @@ -2474,7 +2476,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.7.9" + "version": "3.6.8" }, "varInspector": { "cols": {