Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failure to install msticpy module via MITRE ATT&CK for Azure Sentinel.ipynb #3422

Closed
ktb-jcm opened this issue Nov 10, 2021 · 13 comments
Closed
Assignees
Labels
Config issue Issue is related to a configuration problem.

Comments

@ktb-jcm
Copy link
Contributor

ktb-jcm commented Nov 10, 2021

Using an AzureML environment with a STANDARD_DS3_V2 sized compute instance, attempting to execute the initial notebook set-up results in a cascade of errors related to the msticpy module.

  1. Create a brand new AzureML compute instance
  2. Load the xyz notebook
  3. Specify the new compute instance for the notebook and select the Python 3.8 AzureML kernel (NOTE: This error also occurs with the Python 3.6 AzureML kernel).
  4. Execute the first code cell
  5. Observe the failure to install msticpy due to supposed dependency issues.

Collecting msticpy
Downloading msticpy-1.4.5-py3-none-any.whl (588 kB)
|████████████████████████████████| 588 kB 29.9 MB/s eta 0:00:01
Collecting folium>=0.9.0
Downloading folium-0.12.1-py2.py3-none-any.whl (94 kB)
|████████████████████████████████| 94 kB 3.8 MB/s eta 0:00:01
Requirement already satisfied, skipping upgrade: requests>=2.21.1 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from msticpy) (2.26.0)
Collecting tldextract>=2.2.2
Downloading tldextract-3.1.2-py2.py3-none-any.whl (87 kB)
|████████████████████████████████| 87 kB 7.0 MB/s eta 0:00:01
Requirement already satisfied, skipping upgrade: ipython>=7.1.1 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from msticpy) (7.16.1)
Requirement already satisfied, skipping upgrade: attrs>=18.2.0 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from msticpy) (21.2.0)
Collecting lxml>=4.6.3
Downloading lxml-4.6.4-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (6.3 MB)
|████████████████████████████████| 6.3 MB 84.9 MB/s eta 0:00:01
Collecting dnspython<=2.0.0
Downloading dnspython-2.0.0-py3-none-any.whl (208 kB)
|████████████████████████████████| 208 kB 91.9 MB/s eta 0:00:01
Requirement already satisfied, skipping upgrade: ipywidgets>=7.4.2 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from msticpy) (7.6.4)
Requirement already satisfied, skipping upgrade: cryptography>=3.1 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from msticpy) (3.4.8)
Collecting idna<3
Downloading idna-2.10-py2.py3-none-any.whl (58 kB)
|████████████████████████████████| 58 kB 5.3 MB/s eta 0:00:01
Collecting html5lib
Downloading html5lib-1.1-py2.py3-none-any.whl (112 kB)
|████████████████████████████████| 112 kB 92.5 MB/s eta 0:00:01
Requirement already satisfied, skipping upgrade: numpy>=1.15.4 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from msticpy) (1.18.5)
Requirement already satisfied, skipping upgrade: msrest>=0.6.0 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from msticpy) (0.6.21)
Requirement already satisfied, skipping upgrade: tqdm>=4.36.1 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from msticpy) (4.62.2)
Requirement already satisfied, skipping upgrade: azure-core>=1.2.2 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from msticpy) (1.18.0)
Requirement already satisfied, skipping upgrade: azure-common>=1.1.18 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from msticpy) (1.1.27)
Collecting azure-mgmt-subscription>=1.0.0
Downloading azure_mgmt_subscription-1.0.0-py2.py3-none-any.whl (40 kB)
|████████████████████████████████| 40 kB 6.9 MB/s eta 0:00:01
Requirement already satisfied, skipping upgrade: python-dateutil>=2.8.1 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from msticpy) (2.8.2)
Requirement already satisfied, skipping upgrade: bokeh>=1.4.0 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from msticpy) (2.3.3)
Collecting ipwhois>=1.1.0
Downloading ipwhois-1.2.0-py2.py3-none-any.whl (73 kB)
|████████████████████████████████| 73 kB 2.5 MB/s eta 0:00:01
Collecting azure-identity>=1.5.0
Downloading azure_identity-1.7.1-py2.py3-none-any.whl (129 kB)
|████████████████████████████████| 129 kB 89.4 MB/s eta 0:00:01
Requirement already satisfied, skipping upgrade: matplotlib>=3.0.0 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from msticpy) (3.2.1)
Requirement already satisfied, skipping upgrade: pytz>=2019.2 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from msticpy) (2021.1)
Requirement already satisfied, skipping upgrade: setuptools>=40.6.3 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from msticpy) (50.3.0)
Requirement already satisfied, skipping upgrade: pyyaml>=3.13 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from msticpy) (5.4.1)
Requirement already satisfied, skipping upgrade: msrestazure>=0.6.0 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from msticpy) (0.6.4)
Requirement already satisfied, skipping upgrade: networkx>=2.2 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from msticpy) (2.5)
Collecting geoip2>=2.9.0
Downloading geoip2-4.4.0-py2.py3-none-any.whl (25 kB)
Requirement already satisfied, skipping upgrade: urllib3>=1.23 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from msticpy) (1.25.11)
Collecting deprecated>=1.2.4
Downloading Deprecated-1.2.13-py2.py3-none-any.whl (9.6 kB)
Collecting pandas>=1.1.5
Downloading pandas-1.1.5-cp36-cp36m-manylinux1_x86_64.whl (9.5 MB)
|████████████████████████████████| 9.5 MB 88.5 MB/s eta 0:00:01
Collecting KqlmagicCustom[auth_code_clipboard,jupyter-basic]>=0.1.114.dev26
Downloading KqlmagicCustom-0.1.114.post3-py3-none-any.whl (230 kB)
|████████████████████████████████| 230 kB 86.1 MB/s eta 0:00:01
Requirement already satisfied, skipping upgrade: jinja2>=2.9 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from folium>=0.9.0->msticpy) (2.11.2)
Collecting branca>=0.3.0
Downloading branca-0.4.2-py3-none-any.whl (24 kB)
Requirement already satisfied, skipping upgrade: charset-normalizer~=2.0.0; python_version >= "3" in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from requests>=2.21.1->msticpy) (2.0.4)
Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from requests>=2.21.1->msticpy) (2021.5.30)
Requirement already satisfied, skipping upgrade: filelock>=3.0.8 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from tldextract>=2.2.2->msticpy) (3.0.12)
Collecting requests-file>=1.4
Downloading requests_file-1.5.1-py2.py3-none-any.whl (3.7 kB)
Requirement already satisfied, skipping upgrade: decorator in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from ipython>=7.1.1->msticpy) (5.0.9)
Requirement already satisfied, skipping upgrade: pickleshare in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from ipython>=7.1.1->msticpy) (0.7.5)
Requirement already satisfied, skipping upgrade: backcall in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from ipython>=7.1.1->msticpy) (0.2.0)
Requirement already satisfied, skipping upgrade: prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from ipython>=7.1.1->msticpy) (3.0.20)
Requirement already satisfied, skipping upgrade: pygments in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from ipython>=7.1.1->msticpy) (2.10.0)
Requirement already satisfied, skipping upgrade: jedi>=0.10 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from ipython>=7.1.1->msticpy) (0.17.2)
Requirement already satisfied, skipping upgrade: pexpect; sys_platform != "win32" in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from ipython>=7.1.1->msticpy) (4.8.0)
Requirement already satisfied, skipping upgrade: traitlets>=4.2 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from ipython>=7.1.1->msticpy) (4.3.3)
Requirement already satisfied, skipping upgrade: nbformat>=4.2.0 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from ipywidgets>=7.4.2->msticpy) (5.1.3)
Requirement already satisfied, skipping upgrade: jupyterlab-widgets>=1.0.0; python_version >= "3.6" in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from ipywidgets>=7.4.2->msticpy) (1.0.1)
Requirement already satisfied, skipping upgrade: widgetsnbextension~=3.5.0 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from ipywidgets>=7.4.2->msticpy) (3.5.1)
Requirement already satisfied, skipping upgrade: ipython-genutils~=0.2.0 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from ipywidgets>=7.4.2->msticpy) (0.2.0)
Requirement already satisfied, skipping upgrade: ipykernel>=4.5.1 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from ipywidgets>=7.4.2->msticpy) (5.5.5)
Requirement already satisfied, skipping upgrade: cffi>=1.12 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from cryptography>=3.1->msticpy) (1.14.6)
Requirement already satisfied, skipping upgrade: six>=1.9 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from html5lib->msticpy) (1.16.0)
Requirement already satisfied, skipping upgrade: webencodings in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from html5lib->msticpy) (0.5.1)
Requirement already satisfied, skipping upgrade: requests-oauthlib>=0.5.0 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from msrest>=0.6.0->msticpy) (1.3.0)
Requirement already satisfied, skipping upgrade: isodate>=0.6.0 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from msrest>=0.6.0->msticpy) (0.6.0)
Requirement already satisfied, skipping upgrade: azure-mgmt-core<2.0.0,>=1.2.0 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from azure-mgmt-subscription>=1.0.0->msticpy) (1.3.0)
Requirement already satisfied, skipping upgrade: tornado>=5.1 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from bokeh>=1.4.0->msticpy) (6.1)
Requirement already satisfied, skipping upgrade: pillow>=7.1.0 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from bokeh>=1.4.0->msticpy) (8.0.0)
Requirement already satisfied, skipping upgrade: packaging>=16.8 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from bokeh>=1.4.0->msticpy) (21.0)
Requirement already satisfied, skipping upgrade: typing-extensions>=3.7.4 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from bokeh>=1.4.0->msticpy) (3.10.0.2)
Collecting msal-extensions~=0.3.0
Downloading msal_extensions-0.3.0-py2.py3-none-any.whl (16 kB)
Requirement already satisfied, skipping upgrade: msal<2.0.0,>=1.12.0 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from azure-identity>=1.5.0->msticpy) (1.14.0)
Requirement already satisfied, skipping upgrade: cycler>=0.10 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from matplotlib>=3.0.0->msticpy) (0.10.0)
Requirement already satisfied, skipping upgrade: kiwisolver>=1.0.1 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from matplotlib>=3.0.0->msticpy) (1.3.1)
Requirement already satisfied, skipping upgrade: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from matplotlib>=3.0.0->msticpy) (2.4.7)
Requirement already satisfied, skipping upgrade: adal<2.0.0,>=0.6.0 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from msrestazure>=0.6.0->msticpy) (1.2.7)
Requirement already satisfied, skipping upgrade: aiohttp<4.0.0,>=3.6.2 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from geoip2>=2.9.0->msticpy) (3.7.4.post0)
Collecting maxminddb<3.0.0,>=2.2.0
Downloading maxminddb-2.2.0.tar.gz (330 kB)
|████████████████████████████████| 330 kB 92.0 MB/s eta 0:00:01
Requirement already satisfied, skipping upgrade: wrapt<2,>=1.10 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from deprecated>=1.2.4->msticpy) (1.11.1)
Collecting pyperclip>=1.7.0; extra == "auth_code_clipboard"
Downloading pyperclip-1.8.2.tar.gz (20 kB)
Collecting prettytable>=0.7.2; extra == "jupyter-basic"
Downloading prettytable-2.4.0-py3-none-any.whl (24 kB)
Requirement already satisfied, skipping upgrade: MarkupSafe>=0.23 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from jinja2>=2.9->folium>=0.9.0->msticpy) (2.0.1)
Requirement already satisfied, skipping upgrade: wcwidth in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0->ipython>=7.1.1->msticpy) (0.2.5)
Collecting parso<0.8.0,>=0.7.0
Downloading parso-0.7.1-py2.py3-none-any.whl (109 kB)
|████████████████████████████████| 109 kB 87.4 MB/s eta 0:00:01
Requirement already satisfied, skipping upgrade: ptyprocess>=0.5 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from pexpect; sys_platform != "win32"->ipython>=7.1.1->msticpy) (0.7.0)
Requirement already satisfied, skipping upgrade: jsonschema!=2.5.0,>=2.4 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from nbformat>=4.2.0->ipywidgets>=7.4.2->msticpy) (3.2.0)
Requirement already satisfied, skipping upgrade: jupyter-core in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from nbformat>=4.2.0->ipywidgets>=7.4.2->msticpy) (4.7.1)
Requirement already satisfied, skipping upgrade: notebook>=4.4.1 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from widgetsnbextension~=3.5.0->ipywidgets>=7.4.2->msticpy) (6.4.3)
Requirement already satisfied, skipping upgrade: jupyter-client in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from ipykernel>=4.5.1->ipywidgets>=7.4.2->msticpy) (6.1.12)
Requirement already satisfied, skipping upgrade: pycparser in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from cffi>=1.12->cryptography>=3.1->msticpy) (2.20)
Requirement already satisfied, skipping upgrade: oauthlib>=3.0.0 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from requests-oauthlib>=0.5.0->msrest>=0.6.0->msticpy) (3.1.1)
Requirement already satisfied, skipping upgrade: portalocker~=1.0; platform_system != "Windows" in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from msal-extensions~=0.3.0->azure-identity>=1.5.0->msticpy) (1.7.1)
Requirement already satisfied, skipping upgrade: PyJWT[crypto]<3,>=1.0.0 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from msal<2.0.0,>=1.12.0->azure-identity>=1.5.0->msticpy) (2.1.0)
Requirement already satisfied, skipping upgrade: yarl<2.0,>=1.0 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from aiohttp<4.0.0,>=3.6.2->geoip2>=2.9.0->msticpy) (1.6.3)
Requirement already satisfied, skipping upgrade: chardet<5.0,>=2.0 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from aiohttp<4.0.0,>=3.6.2->geoip2>=2.9.0->msticpy) (4.0.0)
Requirement already satisfied, skipping upgrade: multidict<7.0,>=4.5 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from aiohttp<4.0.0,>=3.6.2->geoip2>=2.9.0->msticpy) (5.1.0)
Requirement already satisfied, skipping upgrade: idna-ssl>=1.0; python_version < "3.7" in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from aiohttp<4.0.0,>=3.6.2->geoip2>=2.9.0->msticpy) (1.1.0)
Requirement already satisfied, skipping upgrade: async-timeout<4.0,>=3.0 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from aiohttp<4.0.0,>=3.6.2->geoip2>=2.9.0->msticpy) (3.0.1)
Requirement already satisfied, skipping upgrade: importlib-metadata; python_version < "3.8" in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from prettytable>=0.7.2; extra == "jupyter-basic"->KqlmagicCustom[auth_code_clipboard,jupyter-basic]>=0.1.114.dev26->msticpy) (4.8.1)
Requirement already satisfied, skipping upgrade: pyrsistent>=0.14.0 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from jsonschema!=2.5.0,>=2.4->nbformat>=4.2.0->ipywidgets>=7.4.2->msticpy) (0.18.0)
Requirement already satisfied, skipping upgrade: terminado>=0.8.3 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets>=7.4.2->msticpy) (0.12.1)
Requirement already satisfied, skipping upgrade: Send2Trash>=1.5.0 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets>=7.4.2->msticpy) (1.8.0)
Requirement already satisfied, skipping upgrade: pyzmq>=17 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets>=7.4.2->msticpy) (22.2.1)
Requirement already satisfied, skipping upgrade: nbconvert in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets>=7.4.2->msticpy) (5.6.1)
Requirement already satisfied, skipping upgrade: prometheus-client in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets>=7.4.2->msticpy) (0.11.0)
Requirement already satisfied, skipping upgrade: argon2-cffi in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets>=7.4.2->msticpy) (21.1.0)
Requirement already satisfied, skipping upgrade: zipp>=0.5 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from importlib-metadata; python_version < "3.8"->prettytable>=0.7.2; extra == "jupyter-basic"->KqlmagicCustom[auth_code_clipboard,jupyter-basic]>=0.1.114.dev26->msticpy) (3.5.0)
Requirement already satisfied, skipping upgrade: mistune<2,>=0.8.1 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets>=7.4.2->msticpy) (0.8.4)
Requirement already satisfied, skipping upgrade: testpath in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets>=7.4.2->msticpy) (0.5.0)
Requirement already satisfied, skipping upgrade: bleach in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets>=7.4.2->msticpy) (4.1.0)
Requirement already satisfied, skipping upgrade: defusedxml in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets>=7.4.2->msticpy) (0.7.1)
Requirement already satisfied, skipping upgrade: entrypoints>=0.2.2 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets>=7.4.2->msticpy) (0.3)
Requirement already satisfied, skipping upgrade: pandocfilters>=1.4.1 in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets>=7.4.2->msticpy) (1.4.3)
Building wheels for collected packages: maxminddb, pyperclip
Building wheel for maxminddb (setup.py) ... done
Created wheel for maxminddb: filename=maxminddb-2.2.0-py2.py3-none-any.whl size=16344 sha256=c9e688dc1edcf04af692d0a933d0a2aa3c9cf669d8bc3ed28b5424f36a401e49
Stored in directory: /home/azureuser/.cache/pip/wheels/4a/01/2e/0219dd35e909a917dea3da2113250fa684693406bbcfd64dbe
Building wheel for pyperclip (setup.py) ... done
Created wheel for pyperclip: filename=pyperclip-1.8.2-py3-none-any.whl size=11106 sha256=5f7008bc478525dc2414bfde8cae5ca200c021b13d91eb2aa43431d8dd713a71
Stored in directory: /home/azureuser/.cache/pip/wheels/95/38/95/e30a7f0b44cb90642de3469f211a3218f93f871789b4f4b46c
Successfully built maxminddb pyperclip
ERROR: responsibleai 0.10.0 has requirement dice-ml<0.8,>=0.7.1, but you'll have dice-ml 0.6.1 which is incompatible.
ERROR: raiwidgets 0.10.0 has requirement jinja2==2.11.3, but you'll have jinja2 2.11.2 which is incompatible.
ERROR: datasets 1.8.0 has requirement tqdm<4.50.0,>=4.27, but you'll have tqdm 4.62.2 which is incompatible.
ERROR: azureml-train-automl-runtime 1.34.0 has requirement pandas<1.0.0,>=0.21.0, but you'll have pandas 1.1.5 which is incompatible.
ERROR: azureml-responsibleai 1.34.0 has requirement responsibleai==0.9.4, but you'll have responsibleai 0.10.0 which is incompatible.
ERROR: azureml-dataprep 2.22.2 has requirement azure-identity<1.5.0,>=1.2.0, but you'll have azure-identity 1.7.1 which is incompatible.
ERROR: azureml-automl-runtime 1.34.0 has requirement pandas<1.0.0,>=0.21.0, but you'll have pandas 1.1.5 which is incompatible.
ERROR: azure-cli 2.24.0 has requirement azure-graphrbac~=0.60.0, but you'll have azure-graphrbac 0.61.1 which is incompatible.
ERROR: azure-cli 2.24.0 has requirement azure-mgmt-containerregistry==3.0.0rc17, but you'll have azure-mgmt-containerregistry 8.1.0 which is incompatible.
ERROR: azure-cli 2.24.0 has requirement azure-mgmt-keyvault==9.0.0, but you'll have azure-mgmt-keyvault 9.1.0 which is incompatible.
ERROR: azure-cli 2.24.0 has requirement azure-mgmt-resource==18.0.0, but you'll have azure-mgmt-resource 13.0.0 which is incompatible.
ERROR: azure-cli 2.24.0 has requirement azure-mgmt-storage~=18.0.0, but you'll have azure-mgmt-storage 11.2.0 which is incompatible.
ERROR: azure-cli 2.24.0 has requirement packaging~=20.9, but you'll have packaging 21.0 which is incompatible.
ERROR: azure-cli 2.24.0 has requirement pytz==2019.1, but you'll have pytz 2021.1 which is incompatible.
ERROR: azure-cli 2.24.0 has requirement websocket-client~=0.56.0, but you'll have websocket-client 1.2.1 which is incompatible.
ERROR: azure-cli-core 2.24.0 has requirement azure-mgmt-core<1.3.0,>=1.2.0, but you'll have azure-mgmt-core 1.3.0 which is incompatible.
ERROR: azure-cli-core 2.24.0 has requirement cryptography<3.4,>=3.2, but you'll have cryptography 3.4.8 which is incompatible.
ERROR: azure-cli-core 2.24.0 has requirement PyJWT==1.7.1, but you'll have pyjwt 2.1.0 which is incompatible.
ERROR: autokeras 1.0.16 has requirement tensorflow<=2.5.0,>=2.3.0, but you'll have tensorflow 2.1.0 which is incompatible.
Installing collected packages: branca, folium, idna, requests-file, tldextract, lxml, dnspython, html5lib, azure-mgmt-subscription, ipwhois, msal-extensions, azure-identity, maxminddb, geoip2, deprecated, pandas, pyperclip, prettytable, KqlmagicCustom, msticpy, parso
Attempting uninstall: idna
Found existing installation: idna 3.2
Uninstalling idna-3.2:
Successfully uninstalled idna-3.2
Attempting uninstall: msal-extensions
Found existing installation: msal-extensions 0.2.2
Uninstalling msal-extensions-0.2.2:
Successfully uninstalled msal-extensions-0.2.2
Attempting uninstall: azure-identity
Found existing installation: azure-identity 1.4.1
Uninstalling azure-identity-1.4.1:
Successfully uninstalled azure-identity-1.4.1
Attempting uninstall: pandas
Found existing installation: pandas 0.25.3
Uninstalling pandas-0.25.3:
Successfully uninstalled pandas-0.25.3
Attempting uninstall: parso
Found existing installation: parso 0.8.2
Uninstalling parso-0.8.2:
Successfully uninstalled parso-0.8.2
Successfully installed KqlmagicCustom-0.1.114.post3 azure-identity-1.7.1 azure-mgmt-subscription-1.0.0 branca-0.4.2 deprecated-1.2.13 dnspython-2.0.0 folium-0.12.1 geoip2-4.4.0 html5lib-1.1 idna-2.10 ipwhois-1.2.0 lxml-4.6.4 maxminddb-2.2.0 msal-extensions-0.3.0 msticpy-1.4.5 pandas-1.1.5 parso-0.7.1 prettytable-2.4.0 pyperclip-1.8.2 requests-file-1.5.1 tldextract-3.1.2
Show less

ModuleNotFoundError Traceback (most recent call last)
/tmp/ipykernel_5770/2450406361.py in
30 try:
---> 31 check_mp_ver(min_msticpy_ver=REQ_MSTICPY_VER)
32 except ImportError:

/mnt/batch/tasks/shared/LS_root/mounts/clusters/ml2/code/Users/admin/utils/nb_check.py in check_mp_ver(min_msticpy_ver)
219 try:
--> 220 import msticpy
221

ModuleNotFoundError: No module named 'msticpy'

During handling of the above exception, another exception occurred:

ModuleNotFoundError Traceback (most recent call last)
/tmp/ipykernel_5770/2450406361.py in
35 importlib.reload(sys.modules["msticpy"])
36 else:
---> 37 import msticpy
38 check_mp_ver(REQ_MSTICPY_VER)
39

ModuleNotFoundError: No module named 'msticpy'

@github-actions
Copy link
Contributor

Thank you for submitting an Issue to the Azure Sentinel GitHub repo! You should expect an initial response to your Issue from the team within 5 business days. Note that this response may be delayed during holiday periods. For urgent, production-affecting issues please raise a support ticket via the Azure Portal.

@ktb-jcm
Copy link
Contributor Author

ktb-jcm commented Nov 10, 2021

I've been researching this and it may relate to issues with the version of PIP in the AzureML compute instance and/or the wrapper used for pip. The AzureML compute instance reports pip 20.1.1 is installed, however it is installed as a python PIP itself, rather than leveraging the Ubuntu python3-pip package.

See also:
Azure/azure-sdk-for-python#9583
microsoft/nlp-recipes#465
Azure/MachineLearningNotebooks#516
pypa/pip#5599

@Azure Azure deleted a comment from github-actions bot Nov 15, 2021
@Azure Azure deleted a comment from github-actions bot Nov 15, 2021
@ashwin-patil
Copy link
Member

Could you please restart the kernel and try the notebook again? Please ignore any dependency errors as long as the last status show successfully installed msticpy. it looks like msticpy library is installed successfully from the above trace and need to restart the kernel to try import again.

I would also recommend starting with template A Getting Started Guide For Azure Sentinel ML Notebooks to set up your environment and msticpyconfig files.

@ktb-jcm
Copy link
Contributor Author

ktb-jcm commented Nov 15, 2021 via email

@github-actions
Copy link
Contributor

Thank you for submitting an Issue to the Azure Sentinel GitHub repo! You should expect an initial response to your Issue from the team within 5 business days. Note that this response may be delayed during holiday periods. For urgent, production-affecting issues please raise a support ticket via the Azure Portal.

@ktb-jcm
Copy link
Contributor Author

ktb-jcm commented Nov 17, 2021

I was able to resolve this issue by replacing the msticpyconfig.yaml file I had in my filespace with an earlier version.

@dragoscosma
Copy link

give it a try with: %pip install msticpy; otherwise this might be also a good idea to install it by default on an aml compute: https://github.com/Azure/Azure-Sentinel-Notebooks/wiki/Can-I-install-MSTICPy-by-default-on-a-new-AML-compute%3F

@ktb-jcm
Copy link
Contributor Author

ktb-jcm commented Nov 29, 2021

Thanks, I'll give that a shot and report back.

@ktb-jcm
Copy link
Contributor Author

ktb-jcm commented Nov 30, 2021

@dragoscosma I tried that Shell script (the Wiki page needs the link updated), and it seemed to work fine. However, importing the resulting mticpy.py library generates a new error. I'll open a separate issue for that.

@v-amolpatil v-amolpatil added the Config issue Issue is related to a configuration problem. label Jan 10, 2023
@github-actions
Copy link
Contributor

Thank you for submitting an Issue to the Azure Sentinel GitHub repo! You should expect an initial response to your Issue from the team within 5 business days. Note that this response may be delayed during holiday periods. For urgent, production-affecting issues please raise a support ticket via the Azure Portal.

@v-amolpatil
Copy link
Contributor

Closing this issue as it is fixed. Please feel free to reopen if need any further assistance.

@github-actions
Copy link
Contributor

Thank you for submitting an Issue to the Azure Sentinel GitHub repo! You should expect an initial response to your Issue from the team within 5 business days. Note that this response may be delayed during holiday periods. For urgent, production-affecting issues please raise a support ticket via the Azure Portal.

@scottpendlebury
Copy link

For those with a problem using this MITRE ATT&CK notebook, change line 37 of the setup code block:

"from msticpy.data.azure.sentinel_core import MicrosoftSentinel"

to

"from msticpy.context.azure.sentinel_core import MicrosoftSentinel"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Config issue Issue is related to a configuration problem.
Projects
None yet
Development

No branches or pull requests

6 participants