cannot import MimicExplainer

[Dewei-Wang-xx]

I have recently met this issue:

ImportError: cannot import name 'MimicExplainer' from 'interpret.ext.blackbox' (C:\Users\xxx\Anaconda3\lib\site-packages\interpret\ext\blackbox_init_.py)

but I didn't update the library or related ones. Do anyone see this error as well?

[imatiach-msft]

@Dewei-Wang-xx can you try to run:

pip uninstall interpret
pip uninstall interpret-core
pip install --upgrade interpret-community

and print the output of those two commands? Usually there is some issue with the extension logic in interpret/interpret-core package that causes this.

[Dewei-Wang-xx]

@imatiach-msft
thanks for your response. I tried your commands but still cannot import the libraries, including TabularExplainer, MimicExplainer, etc.

with pip show interpret-community, it shows

Name: interpret-community
Version: 0.30.0
Summary: Microsoft Interpret Extensions SDK for Python
Home-page: https://github.com/interpretml/interpret-community
Author: Microsoft Corp
Author-email: ilmat@microsoft.com
License: MIT License
Location: c:\users\wang109\anaconda3\envs\ensemble\lib\site-packages
Requires: interpret-core, ml-wrappers, numpy, packaging, pandas, raiutils, scikit-learn, scipy, shap
Required-by:

There are no script except for "init.py" in \Lib\site-packages\interpret\ext\blackbox

[imatiach-msft]

@Dewei-Wang-xx it seems there is some issue with the extension system on your particular machine/environment, but without debugging with you I can't tell which package in the environment is causing it. I would recommend to setup a meeting to debug this together. If that is not possible, the simple workaround is to use the interpret_community namespace instead of the interpret.ext.* namespace.
For example, instead of using:

from interpret.ext.blackbox import MimicExplainer

You can just import using:

from interpret_community.mimic import MimicExplainer

Please see the setup.py file for the full mapping of all classes:

https://github.com/interpretml/interpret-community/blob/main/python/setup.py#L90

[imatiach-msft]

The whole extension system isn't really needed - it's just there to make it look like the interpret-community package is more integrated with the interpret package and the interpretml ecosystem of python packages. But anyone can just import using the original interpret_community namespace instead of using the interpret.ext.* namespace.

[imatiach-msft]

Another way this extension system can break is if there is some package version conflict. For example, I ran the command:

pip install --upgrade interpret

And then I see the failures on import telling me that there is a package version conflict and the version of interpret is too new - downgrading via "pip install interpret-core[required]<=0.4.3" fixes this issue (see the very last line of error):

(sh) PS C:\interpret-community> python
Python 3.8.17 | packaged by conda-forge | (default, Jun 16 2023, 07:01:59) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from interpret.ext.blackbox import MimicExplainer
C:\Users\ilmat\AppData\Local\miniconda3\envs\sh\lib\site-packages\numpy\_distributor_init.py:30: UserWarning: loaded more than 1 DLL from .libs:
C:\Users\ilmat\AppData\Local\miniconda3\envs\sh\lib\site-packages\numpy\.libs\libopenblas.EL2C6PLE4ZYW3ECEVIV3OXXGRN2NRFM2.gfortran-win_amd64.dll
C:\Users\ilmat\AppData\Local\miniconda3\envs\sh\lib\site-packages\numpy\.libs\libopenblas64__v0.3.21-gcc_10_3_0.dll
  warnings.warn("loaded more than 1 DLL from .libs:"
C:\Users\ilmat\AppData\Local\miniconda3\envs\sh\lib\site-packages\interpret\ext\extension_utils.py:74: UserWarning: Failure while loading interpret_ext_blackbox. Failed to load entrypoint KernelExplainer = interpret_community.shap:KernelExplainer with exception Traceback (most recent call last):
  File "C:\Users\ilmat\AppData\Local\miniconda3\envs\sh\lib\site-packages\interpret\ext\extension_utils.py", line 46, in load_class_extensions
    extension_class = entrypoint.load()
  File "C:\Users\ilmat\AppData\Local\miniconda3\envs\sh\lib\site-packages\pkg_resources\__init__.py", line 2517, in load
    self.require(*args, **kwargs)
  File "C:\Users\ilmat\AppData\Local\miniconda3\envs\sh\lib\site-packages\pkg_resources\__init__.py", line 2540, in require
    items = working_set.resolve(reqs, env, installer, extras=self.extras)
  File "C:\Users\ilmat\AppData\Local\miniconda3\envs\sh\lib\site-packages\pkg_resources\__init__.py", line 829, in resolve
    dist = self._resolve_dist(
  File "C:\Users\ilmat\AppData\Local\miniconda3\envs\sh\lib\site-packages\pkg_resources\__init__.py", line 875, in _resolve_dist
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (interpret-core 0.4.4 (c:\users\ilmat\appdata\local\miniconda3\envs\sh\lib\site-packages), Requirement.parse('interpret-core[required]<=0.4.3,>=0.1.20'))

[Dewei-Wang-xx]

@Dewei-Wang-xx it seems there is some issue with the extension system on your particular machine/environment, but without debugging with you I can't tell which package in the environment is causing it. I would recommend to setup a meeting to debug this together. If that is not possible, the simple workaround is to use the interpret_community namespace instead of the interpret.ext.* namespace. For example, instead of using:

from interpret.ext.blackbox import MimicExplainer

You can just import using:

from interpret_community.mimic import MimicExplainer

Please see the setup.py file for the full mapping of all classes:

https://github.com/interpretml/interpret-community/blob/main/python/setup.py#L90

@imatiach-msft thanks a lot! this works for me.

But how to get the " from interpret.ext.blackbox import MimicExplainer" working?
In previous scripts, I have imported LGBMExplainableModel from interpret.ext.glassbox, TabularExplainer from interpret.ext.blackbox, etc., it may cause some issues to fix them one by one. Thank you!

[imatiach-msft]

@Dewei-Wang-xx I'm honestly not sure - there should be some error message like above or during install but you are not seeing one. I think the best way to debug this would be to setup a call since you are not seeing anything.

The code to setup the extension system is not in this package but in interpret - specifically this method:

https://github.com/interpretml/interpret/blob/develop/python/interpret-core/interpret/ext/extension_utils.py#L33

I think the best way to debug would be to install interpret-core package locally, call this method here in a jupyter notebook:

https://github.com/interpretml/interpret/blob/develop/python/interpret-core/interpret/ext/glassbox/__init__.py#L8

And then either add print statements or use pdb to debug through that extension_utils method above to see what error is thrown or why those extensions aren't getting registered.