Installation Instructions for Pymc3 on Windows 10 using Anaconda 3

[sreedat]

Hi @michaelosthege - here is the first version of the Installation Reference. Let me know what changes may be needed.

For Pymc3 Windows users who may not have a programming background or have comfort with tool chains and such, the installation instructions on this page (https://github.com/pymc-devs/pymc3/wiki/Installation-Guide-(Windows)) may not be sufficient. The instructions posted below go beyond the basic installation process posted at the above link.

In addition, there are a large community of users, who use both R and Python (Anaconda). The RTools mingw tool chain has to be first on the System Environment variables so that R packages that need compilation such as rstan, brms will run correctly. In this situation, the compilation of pymc3 models will break and additional post-install User Environment Variables has to be done so pymc3 works correctly.

Versions and main components

• PyMC3 Version: 3.11.2
• Aesara/Theano Version: 1.1.2 ( using theano-pymc)
• Python Version: 3.7 / 3.8 using Anaconda3 64-bit
• Operating system: Windows 10 64-bit, with 1904 Update
• How did you install PyMC3: pip
• C & C++ compilers: Installed m2w64 tool chain from conda-forge
• Microsoft VS C++ compiler(s) present? No
• Any Competing C++ compiler(s) present? Yes - RTools mingw tool chain on the System PATH

---

The essence of a solid PyMC3 installation on Windows is to install most of the dependencies through conda. The reason installation via PyPI is difficult is that Theano/Aesara require compilation against MKL, which is difficult to set up, while Conda comes with its own compilers and MKL installation.

⚠ Do not pip install without first installing dependencies with conda. ⚠

Method 1: Run conda env create -f environment.yml to create a fresh environment in one step - use Notepad++, if possible to create the said environment.yml file.

environment.yml (copy from name: pm3env and ending with pymc3 and save it in C:\Users\Your_User_Name )

channels:
  - conda-forge
  - defaults
dependencies:
  - libpython
  - blas
  - mkl-service
  - m2w64-toolchain
  - numba
  - pip
  - python=3.8
  - python-graphviz
  - scipy
  - pip:
    - pymc3

You can change name to something meaningful for you such as pym3 or env_pym3. You do not have to use pm3env. Keep it simple and meaningful so you can use the environment with ease when using the command line.

Method 2: You can create pymc3 specific environment also directly from the Anaconda3 Command Prompt using the following command:
conda create -n pm3env -c conda-forge "python=3.8" libpython mkl-service m2w64-toolchain numba python-graphviz scipy

After you have created the environment, you can test it has been created successfully by typing:
conda info --envs - the output of the command will show all the environments that exist in the current Anaconda3 install. Hopefully you will see your environment for pymc3

Activate your environment by typing the command: conda activate pm3env (or whatever name you chose)
Your command prompt will look like: (pm3env) C:\Users\Your_User_Name

Check Packages Installed using the command conda list and all packages installed in pm3env will be shown. Check if either one of theano or theano-pymc has been installed. Make note of it (It should not be)

Install Pymc3: Now you can install Pymc3 using the command: pip install pymc3 and then if all requirements are met, all packages from pip and their dependencies will be installed,. Of critical importance to note is whether theano-pymc has been installed along with pymc3 - pymc3 will not run without this - Here theano-pymc should be installed. In addition, in the pip output check that dependencies such as arviz (for working with pymc3 objects) and matplotlib (for general purpose data graphing) have also been installed.

Sometimes these packages may not be installed in your new environment but will be installed likely in the location below on Windows 10: c:\users\your_user_name\appdata\roaming\python\python38\site-packages - note the number 38 next to python38 in the folder name. This means that versions relevant for python3.8 have been installed here. These packages will not appear in the output of conda list. Ensure that the python you specified (python=3.8) matches what you see in the folder name here (python38)

[@michaelosthege The original instructions ask to install theano-pymc using conda-forge. However, I found that in the new dependency installation of pymc3, theano-pymc is being automatically being installed and it is working correctly. This is one where I need your input on whether we remove the note to update theano-pymc. I think this is outdated]

Now there are additional ways to install ``pymc3and additional variants ofpymc3```. Refer to the next section, which may be more appropriate for advanced users.

Developer Installation

If you want to tinker with PyMC3 itself, first clone the repository and then make an "editable" installation:
(You need to have already installed git for this to work, if not install git first)

cd pymc3
pip install --editable .

Upgrading from Theano to Theano-PyMC - Just in case when you run conda list at the pm3env prompt and find that you have theano instead of theano-pymc (stranger things have and will happen!)

1. Make a note of the channel where theano was installed from: it will show pypi or conda-forge.
2. If you see pypi then use the command, pip uninstall theano - you should see a message stating theano has been uninstalled
3. If you see conda-forge use the command, conda remove theano - you should answer with a y if prompted for the removal of theano
4. Install theano-pymc using the command, conda install -c conda-forge theano-pymc.
5. Once the installation is complete, run the command conda list and verify pymc3 & theano-pymc are installed.

Optional Dependencies (before you install any packages, first check they have not been already installed using conda list

1. The GLM submodule relies on Patsy. Patsy brings the convenience of "R-style formulas" to Python.
2. pm.model_to_graphviz depends on Graphviz and pydot:
3. Use the command conda install -c conda-forge python-graphviz and pip install pydot-ng
4. In the package installations done so far, Jupyter Notebook or Jupyter Lab are not installed. If you are working with Anaconda3, you install these two Jupyter tools from Anaconda3 Navigator or from the pm3env command prompt using the command: conda install -c conda-forge notebook

Do Not Close the Command Prompt window of the pm3env - we have to use it later on.

Post Installation Checks - Do Not Skip This Step on Windows
A. Assumes you have installed Jupyter Notebook
B. Make note of whether you have R and in particular RTools installed on your laptop and location of its' install.
C. Assumes you have either Admin or Power-User rights on your laptop so that you can make changes to the environment variables at the User level.

1. Windows does not come pre-installed with C and C++ compilers (as Mac and Linux Distros do) so it is important to ensure your Anaconda3 environments are pointing to the correct internal compilers.

2. Going back to the original creation / installation of pm3env environment, one of the packages installed is m2w64-toolchain. Here things can get complex in terms of having compiler tools specific to the version of Python and to that of the environment. So generally speaking, try to keep your Anaconda3 environments to a minimum, when you are starting out with tools such as pymc3. Also make sure you install a m2w64-toolchain in each environment you create. Conda will ensure that you have the most appropriate version of the compilers for the version of Python in that environment. There are exceptions, but it gets beyond the scope of this document.

3. On Windows, search for "Edit System Environment Variables". You will be taken to the screen below
   

4. Click on the "Environment Variables" on the bottom right-hand corner and you will see a new window pop-up with the top Window for User variables and the bottom window for the System variables. Click on the entry labeled Path under User Variables for Your_User_Name and click edit. Here you should add the following Anaconda paths specific to your environment:

Note the location of where Anaconda3 is installed by default (as shown below). If you have changed the location during the installation of Anaconda3, please make the changes accordingly.

First: C:\ProgramData\Anaconda3\Library\mingw-w64\bin
Second: C:\ProgramData\Anaconda3\Library\bin
Third: C:\ProgramData\Anaconda3\Scripts

Move each of the entries so that they appear in the exact order as shown above at the top of the User Variables Path using the Move Up and Move Down buttons

Then click "OK" to accept the changes all the way.

5. Go back to the pm3env Command Prompt window that is already open that displays (pm3env) C:\Users\Your_User_Name>

6. Type jupyter-notebook at the prompt and (hopefully) a page should be opened in your Default Browser.
   

7. On the top right corner, click on the drop-down under New
   

8. Select Python 3 (ipykernel)

9. You should see a new Jupyter Notebook open that looks as follows:
   

10. Make sure that at the top right corner, the button says Trusted. If this is the first time you are using a Jupyter Notebook, it might show up as Not Trusted. Click on it if it says "Not Trusted" and select the option to make it "Trusted".

11. In the first Notebook cell, type import theano as tp and click on the Run button.

12. You may see a warning as shown in the picture below (WARNING (theano.tensor.blas): Using NumPy C-API based implementation for BLAS functions.)
    

13. In the next cell type the following commands and click Run
    import pymc3 as pm
print(f"Running on PyMC3 v{pm.__version__}")

14. You should see an output as follows: Running on PyMC3 v3.11.2

15. You can now begin testing the full capabilities of Pymc3 by starting with the three examples from the Getting Started with Pymc3 - linked here - http://docs.pymc.io/notebooks/getting_started

Good luck with using Pymc3!

[michaelosthege]

Before I go through the details, here are two suggestions:

1. Instead of using a Jupyter notebook, you can test the PyMC3 import with the line
   python -c "import pymc3; print(pymc3.__version__)
2. You can use the following code in Markdown to make collapsible sections:

<details><summary>Click here to show detailed instructions</summary>

Inside the details you can have Markdown formatting (including code sections) and images.

</details>

[ricardoV94]

Is that theano.tensor.blas warning an indication that the installation is not optimal?

[sreedat]

Thanks @michaelosthege for the initial feedback - I will include those changes. Collapsible sections are a great idea and thanks for showing how it is done!

@ricardoV94 yes it is (and thanks for the question) - what this means (upon researching I found) is that theano is unable to find the optimized libraries such as mkl and is calling indirectly via numpy. From what I read, this will slow down the computation and also increase the memory utilization.

I found a solution: install blas from conda-forge: conda install -c conda-forge blas. After the installation of blas, the warning went away.

@michaelosthege I will add blas to the requirements in the environment.yml file.

Update: ``blashas been added to theenvironment.yml``` file

[mjhajharia]

this looks like a really detailed and helpful way of explaining things!! thanks! we do have a lot of windows installation problems. A lil suggestion that you can think about, although wait to see what others think but: a few of the steps seem to be things that are more to do with requirements for a working "anaconda and jupyter" which can probably be a separate subsection. as in "here's the guide to install pymc3 on windows: 1. If you don't have a working anaconda/are not familiar with condo envs check {insert link}" and perhaps after some suggestions we can make a small installation troubleshooting guide, I think there are some common import errors that happen, so users don't have to dig through old issues and are aware of common troubles, it can be an FAQ/ troubleshooting guide thing, a lot of libraries nowadays have those I think. the docs team might have a different approach about it, here's just my two cents!!

[sreedat]

@mjhajharia thank you for the feedback and the suggestions. I included Jupyter as a way to test as it reflects my Windows bias rather than my comfort with command line. @michaelosthege has already proposed a way to test the Pymc3 installation using the command line and I will change that in the document. I agree that the sub-sections to installing Pymc3 and testing it are many. I hope that with my own experience, I can help others avoid the mistakes that I did. Please share any additional feedback.

[sreedat]

@michaelosthege @ricardoV94 Just an update on the Warning associated with Theano and Blas:

What I found is the folliwing:

1. The solution I provided of installing blas worked only in the environment with Python=3.7.9
2. The same approach did not work and the warning persisted in a different environment with Python=3.8.8. Where the warning persists, the pm.sample() command samples lot more slowly compared to scenario 1 above without the warning.

@michaelosthege Any idea why this is happening?

Sree

[michaelosthege]

My experience with this whole installation stuff is just empirical. I have very little understanding of how these dependencies interact with each other.

Over here (and linked there) are ongoing activities to fix the Aesara and PyMC3 conda recipes: #4932.

Again I'm not involved with making these changes right now. I just happen to be someone with a Windows machine who found a local optimum of more or less reproducible installation instructions.

[sreedat]

@michaelosthege thanks for getting back Michael! I will check the link and understand more. It is good that there is a very good working setup with 3.7.9 that I found on Windows 10 and you have some thing similar. Hopefully between these two variations we can have some stability. Might be wishful thinking on my part! In the documentation, the Python=3.7.9 and Python=3.8.8 and their respective environments can be provided which may help out other users.

[sreedat]

@michaelosthege @ricardoV94 the installation instructions are no good. On a new machine I tried to follow these instructions and create a working installation of pymc3 (v=3.11.2 as well as v=3.11.3). I cannot create the environment using Python=3.7.9 that was working on one machine with Windows 10 and replicate this on another Windows 10 machine by making use of the "environment.yml``` I had created from a fully working version. Following the user path that @michaelosthege posted, which worked on one machine, is not helping on another. This is a completely fresh install. I get the same error below:

Exception: ('Compilation failed (return status=1): C:\Users\SreeDatta\AppData\Local\Temp\ccEWCiGI.s: Assembler messages:\r. C:\Users\SreeDatta\AppData\Local\Temp\ccEWCiGI.s:367: Error: invalid register for .seh_savexmm\r. ', 'FunctionGraph(Elemwise{eq,no_inplace}(<TensorType(int64, vector)>, <TensorType(int8, (True,))>))')

I have to do more investigation into why this is failing for Windows 10 and understand this seemingly random behavior

[sreedat]

@michaelosthege @ricardoV94 I think I have discovered a working solution to installing and running pymc3 on Windows 10. I have now replicated this approach on 3 laptops running Windows 10 and it works without any errors so far. The problem appears to be:

m2w64-toolchain uses gcc 5.3.0 and some part of that, on Windows 10, is incompatible with theano-pymc being used in pymc3 v=3.11.2 & v=3.11.3 (since I have only tested these versions).

On a whim, I installed RTools mingw tool chain and used it, since it uses gcc 8.3.0. I checked on Anaconda and found that the gcc 8.3.0 based m2w64-toolchain is not available for Windows. It is available for Linux. In addition to installing RTools, in the specific pymc3 environmentI also installedblasandlibblas(after installing RTools) fromconda-forge```.

When RTools was installed, I allowed that to be on the System PATH and to be discovered as the main mingw compiler source for conda environments and disabled the installed m2w64-toolchain by changing the directory name in the conda envs.

After I restarted each of the Windows 10 machines, I could run all of the examples from the Pymc3 Getting Started page successfully without any theano errors. I tested and made sure that the only gcc & g++ accessed in conda envs were those from RTools.

These invalid register errors crop up only with theano-pymc. In testing pymc3 v4 with aesara, these invalid register errors do not appear for now. I was ready to give up on using pymc3 on Windows 10. With this serendipitous solution, I have hope. I will see how long this lasts.

Lastly, is there anything that can be done to precisely identify the problem and find a proper solution? I have a solution, but this is a stop gap at best.

[twiecki]

@sreedat can you post the exact install commands you used?

[sreedat]

@twiecki After posting this I could make my pymc3 work without installing m2w64-toolchain in the first place any where, either the base Anaconda or one of the envs. Here are the steps I followed.

1. Downloaded the latest 64-bit RTools from this page https://cran.r-project.org/bin/windows/Rtools/ and installed it with Admin rights so that the mingw compilers are on the System PATH and also First on the PATH. I installed RTools to the default location installer prefers C:\rtools40

2. Note: RTools on WIndows has to be installed in directory paths without any space in the directory names.

3. Restarted the machine.

4. All of the next steps are in Anaconda.

5. Used the following command to create a basic environment:
   conda create -n pm3env -c conda forge "python=3.8" libpython mkl mkl-service numba scipy python-graphviz

6. conda activate pm3env

7. pip install pymc3==3.11.2 (I'm running everything with Admin rights)

8. (pm3env) C:\Users\SreeDatta> which gcc and then which g++ (to ensure C:\rtools40\mingw64\bin is referenced as the primary C and C++ compiler.

9. (pm3env) C:\Users\SreeDatta>conda install -c conda-forge blas libblas (I did this a preemptive install. I got tired of seeing the invalid register theano error.

10. For reference these were the versions `installed:
    blas 1.0 mkl conda-forge
    ```libblas 3.9.0 1_h8933c1f_netlib conda-forge```

11. After this step, I stalled jupyter using conda install -c conda-forge notebook as that is my preferred way to interact with pymc3

12. Then I could run the basic and some advanced hierarchical models without the theano-pymc error.

13. I do not know how long this working environment will last. I also verified that on my WSL Ubuntu as well as my dedicated Ubuntu laptop, I've gcc 8.3.0 - compared to Windows 10, Linux / Ubuntu has been a breeze, but all of my team members work on WIndows 10 and I needed to get a decent Windows 10 environment for pymc3.

14. I will post more here as I see what happens with different users.

[sreedat]

@twiecki I have successfully repeated the exact process of using RTools as the preferred gcc compiler and created additional environments across the same 3 machines using python=3.9 and pymc3=3.11.3. I installed blas and libblas in these environments as I specified in the steps above. I have a good working pymc3 environment without theano-pymc errors.

Here are the two environment.yml files for both of what I have tested to date when using RTools gcc 8.3.0

Environment_pymc3_py38RT.yml

name: pm3py38 channels: - conda-forge - defaults dependencies: - anyio=3.3.0=py38haa244fe_0 - argon2-cffi=20.1.0=py38h294d835_2 - async_generator=1.10=py_0 - attrs=21.2.0=pyhd8ed1ab_0 - babel=2.9.1=pyh44b312d_0 - backcall=0.2.0=pyh9f0ad1d_0 - backports=1.0=py_2 - backports.functools_lru_cache=1.6.4=pyhd8ed1ab_0 - blas=1.0=mkl - bleach=4.0.0=pyhd8ed1ab_0 - brotlipy=0.7.0=py38h294d835_1001 - ca-certificates=2021.5.30=h5b45459_0 - cairo=1.16.0=hb19e0ff_1008 - certifi=2021.5.30=py38haa244fe_0 - cffi=1.14.6=py38hd8c33c5_0 - chardet=4.0.0=py38haa244fe_1 - charset-normalizer=2.0.0=pyhd8ed1ab_0 - colorama=0.4.4=pyh9f0ad1d_0 - cryptography=3.4.7=py38hd7da0ea_0 - debugpy=1.4.1=py38h885f38d_0 - decorator=5.0.9=pyhd8ed1ab_0 - defusedxml=0.7.1=pyhd8ed1ab_0 - entrypoints=0.3=pyhd8ed1ab_1003 - expat=2.4.1=h39d44d4_0 - font-ttf-dejavu-sans-mono=2.37=hab24e00_0 - font-ttf-inconsolata=3.000=h77eed37_0 - font-ttf-source-code-pro=2.038=h77eed37_0 - font-ttf-ubuntu=0.83=hab24e00_0 - fontconfig=2.13.1=h1989441_1005 - fonts-conda-ecosystem=1=0 - fonts-conda-forge=1=0 - freetype=2.10.4=h546665d_1 - fribidi=1.0.10=h8d14728_0 - getopt-win32=0.1=h8ffe710_0 - gettext=0.19.8.1=h1a89ca6_1005 - graphite2=1.3.13=1000 - graphviz=2.48.0=hefbd956_0 - gts=0.7.6=h7c369d9_2 - harfbuzz=2.8.2=hc601d6f_0 - icu=68.1=h0e60522_0 - idna=3.1=pyhd3deb0d_0 - importlib-metadata=4.6.3=py38haa244fe_0 - intel-openmp=2021.3.0=h57928b3_3372 - ipykernel=6.0.3=py38h43734a8_0 - ipython=7.26.0=py38h595d716_0 - ipython_genutils=0.2.0=py_1 - jbig=2.1=h8d14728_2003 - jedi=0.18.0=py38haa244fe_2 - jinja2=3.0.1=pyhd8ed1ab_0 - jpeg=9d=h8ffe710_0 - json5=0.9.5=pyh9f0ad1d_0 - jsonschema=3.2.0=pyhd8ed1ab_3 - jupyter_client=6.1.12=pyhd8ed1ab_0 - jupyter_core=4.7.1=py38haa244fe_0 - jupyter_server=1.10.2=pyhd8ed1ab_0 - jupyterlab=3.1.4=pyhd8ed1ab_0 - jupyterlab_pygments=0.1.2=pyh9f0ad1d_0 - jupyterlab_server=2.6.2=pyhd8ed1ab_0 - lerc=2.2.1=h0e60522_0 - libblas=3.9.0=1_h8933c1f_netlib - libcblas=3.9.0=5_hd5c7e75_netlib - libdeflate=1.7=h8ffe710_5 - libffi=3.3=h0e60522_2 - libgd=2.3.2=h138e682_0 - libglib=2.68.3=h1e62bf3_0 - libiconv=1.16=he774522_0 - liblapack=3.9.0=5_hd5c7e75_netlib - libpng=1.6.37=h1d00b33_2 - libpython=2.0=py38haa244fe_1 - libsodium=1.0.18=h8d14728_1 - libtiff=4.3.0=h0c97f57_1 - libwebp=1.2.0=h57928b3_0 - libwebp-base=1.2.0=h8ffe710_2 - libxcb=1.13=hcd874cb_1003 - libxml2=2.9.12=hf5bbc77_0 - llvmlite=0.36.0=py38h57a6900_0 - lz4-c=1.9.3=h8ffe710_1 - m2w64-gcc-libgfortran=5.3.0=6 - m2w64-gcc-libs=5.3.0=7 - m2w64-gcc-libs-core=5.3.0=7 - m2w64-gmp=6.1.0=2 - m2w64-libwinpthread-git=5.0.0.4634.697f757=2 - markupsafe=2.0.1=py38h294d835_0 - matplotlib-inline=0.1.2=pyhd8ed1ab_2 - mistune=0.8.4=py38h294d835_1004 - mkl=2021.3.0=haa95532_524 - mkl-service=2.4.0=py38h294d835_0 - msys2-conda-epoch=20160418=1 - nbclassic=0.3.1=pyhd8ed1ab_1 - nbclient=0.5.3=pyhd8ed1ab_0 - nbconvert=6.1.0=py38haa244fe_0 - nbformat=5.1.3=pyhd8ed1ab_0 - nest-asyncio=1.5.1=pyhd8ed1ab_0 - notebook=6.4.2=pyha770c72_0 - numba=0.53.1=py38h99fac5f_1 - numpy=1.21.1=py38h09042cb_0 - openssl=1.1.1k=h8ffe710_1 - packaging=21.0=pyhd8ed1ab_0 - pandoc=2.14.1=h8ffe710_0 - pandocfilters=1.4.2=py_1 - pango=1.48.7=hd84fcdd_0 - parso=0.8.2=pyhd8ed1ab_0 - pcre=8.45=h0e60522_0 - pickleshare=0.7.5=py_1003 - pip=21.2.3=pyhd8ed1ab_0 - pixman=0.40.0=h8ffe710_0 - prometheus_client=0.11.0=pyhd8ed1ab_0 - prompt-toolkit=3.0.19=pyha770c72_0 - pthread-stubs=0.4=hcd874cb_1001 - pycparser=2.20=pyh9f0ad1d_2 - pygments=2.9.0=pyhd8ed1ab_0 - pyopenssl=20.0.1=pyhd8ed1ab_0 - pyparsing=2.4.7=pyh9f0ad1d_0 - pyrsistent=0.17.3=py38h294d835_2 - pysocks=1.7.1=py38haa244fe_3 - python=3.8.10=h7840368_1_cpython - python-dateutil=2.8.2=pyhd8ed1ab_0 - python-graphviz=0.17=pyhaef67bd_0 - python_abi=3.8=2_cp38 - pytz=2021.1=pyhd8ed1ab_0 - pywin32=301=py38h294d835_0 - pywinpty=1.1.3=py38hd3f51b4_0 - pyzmq=22.2.1=py38h09162b1_0 - requests=2.26.0=pyhd8ed1ab_0 - requests-unixsocket=0.2.0=py_0 - scipy=1.7.1=py38ha1292f7_0 - send2trash=1.7.1=pyhd8ed1ab_0 - setuptools=49.6.0=py38haa244fe_3 - six=1.16.0=pyh6c4a22f_0 - sniffio=1.2.0=py38haa244fe_1 - sqlite=3.36.0=h8ffe710_0 - terminado=0.10.1=py38haa244fe_0 - testpath=0.5.0=pyhd8ed1ab_0 - tornado=6.1=py38h294d835_1 - traitlets=5.0.5=py_0 - ucrt=10.0.20348.0=h57928b3_0 - urllib3=1.26.6=pyhd8ed1ab_0 - vc=14.2=hb210afc_5 - vs2015_runtime=14.29.30037=h902a5da_5 - wcwidth=0.2.5=pyh9f0ad1d_2 - webencodings=0.5.1=py_1 - websocket-client=0.57.0=py38haa244fe_4 - wheel=0.36.2=pyhd3deb0d_0 - win_inet_pton=1.1.0=py38haa244fe_2 - wincertstore=0.2=py38haa244fe_1006 - winpty=0.4.3=4 - xorg-kbproto=1.0.7=hcd874cb_1002 - xorg-libice=1.0.10=hcd874cb_0 - xorg-libsm=1.2.3=hcd874cb_1000 - xorg-libx11=1.7.2=hcd874cb_0 - xorg-libxau=1.0.9=hcd874cb_0 - xorg-libxdmcp=1.1.3=hcd874cb_0 - xorg-libxext=1.3.4=hcd874cb_1 - xorg-libxpm=3.5.13=hcd874cb_0 - xorg-libxt=1.2.1=hcd874cb_2 - xorg-xextproto=7.3.0=hcd874cb_1002 - xorg-xproto=7.0.31=hcd874cb_1007 - xz=5.2.5=h62dcd97_1 - zeromq=4.3.4=h0e60522_0 - zipp=3.5.0=pyhd8ed1ab_0 - zlib=1.2.11=h62dcd97_1010 - zstd=1.5.0=h6255e5f_0 - pip: - arviz==0.11.2 - cachetools==4.2.2 - cftime==1.5.0 - cycler==0.10.0 - dill==0.3.4 - fastprogress==1.0.0 - filelock==3.0.12 - formulae==0.1.3 - kiwisolver==1.3.1 - matplotlib==3.4.2 - netcdf4==1.5.7 - pandas==1.3.1 - patsy==0.5.1 - pillow==8.3.1 - pymc3==3.11.2 - semver==2.13.0 - theano-pymc==1.1.2 - typing-extensions==3.10.0.0 - xarray==0.19.0

Environment_pymc3_py39RT.yml

name: pm3py39 channels: - conda-forge - defaults dependencies: - argon2-cffi=20.1.0=py39hb82d6ee_2 - async_generator=1.10=py_0 - attrs=21.2.0=pyhd8ed1ab_0 - backcall=0.2.0=pyh9f0ad1d_0 - backports=1.0=py_2 - backports.functools_lru_cache=1.6.4=pyhd8ed1ab_0 - blas=1.0=mkl - bleach=4.0.0=pyhd8ed1ab_0 - ca-certificates=2021.5.30=h5b45459_0 - cairo=1.16.0=hb19e0ff_1008 - cffi=1.14.6=py39h0878f49_0 - colorama=0.4.4=pyh9f0ad1d_0 - debugpy=1.4.1=py39h415ef7b_0 - decorator=5.0.9=pyhd8ed1ab_0 - defusedxml=0.7.1=pyhd8ed1ab_0 - entrypoints=0.3=pyhd8ed1ab_1003 - expat=2.4.1=h39d44d4_0 - font-ttf-dejavu-sans-mono=2.37=hab24e00_0 - font-ttf-inconsolata=3.000=h77eed37_0 - font-ttf-source-code-pro=2.038=h77eed37_0 - font-ttf-ubuntu=0.83=hab24e00_0 - fontconfig=2.13.1=h1989441_1005 - fonts-conda-ecosystem=1=0 - fonts-conda-forge=1=0 - freetype=2.10.4=h546665d_1 - fribidi=1.0.10=h8d14728_0 - getopt-win32=0.1=h8ffe710_0 - gettext=0.19.8.1=h1a89ca6_1005 - graphite2=1.3.13=1000 - graphviz=2.48.0=hefbd956_0 - gts=0.7.6=h7c369d9_2 - harfbuzz=2.9.0=hc601d6f_0 - icu=68.1=h0e60522_0 - importlib-metadata=4.6.4=py39hcbf5309_0 - intel-openmp=2021.3.0=h57928b3_3372 - ipykernel=6.2.0=py39h832f523_0 - ipython=7.26.0=py39h832f523_0 - ipython_genutils=0.2.0=py_1 - jbig=2.1=h8d14728_2003 - jedi=0.18.0=py39hcbf5309_2 - jinja2=3.0.1=pyhd8ed1ab_0 - jpeg=9d=h8ffe710_0 - jsonschema=3.2.0=pyhd8ed1ab_3 - jupyter_client=6.1.12=pyhd8ed1ab_0 - jupyter_core=4.7.1=py39hcbf5309_0 - jupyterlab_pygments=0.1.2=pyh9f0ad1d_0 - lerc=2.2.1=h0e60522_0 - libblas=3.9.0=1_h8933c1f_netlib - libcblas=3.9.0=5_hd5c7e75_netlib - libdeflate=1.7=h8ffe710_5 - libffi=3.3=h0e60522_2 - libgd=2.3.2=h138e682_0 - libglib=2.68.4=h1e62bf3_0 - libiconv=1.16=he774522_0 - liblapack=3.9.0=5_hd5c7e75_netlib - libpng=1.6.37=h1d00b33_2 - libpython=2.0=py39hcbf5309_1 - libsodium=1.0.18=h8d14728_1 - libtiff=4.3.0=h0c97f57_1 - libwebp=1.2.1=h57928b3_0 - libwebp-base=1.2.1=h8ffe710_0 - libxcb=1.13=hcd874cb_1003 - libxml2=2.9.12=hf5bbc77_0 - llvmlite=0.36.0=py39ha0cd8c8_0 - lz4-c=1.9.3=h8ffe710_1 - m2w64-gcc-libgfortran=5.3.0=6 - m2w64-gcc-libs=5.3.0=7 - m2w64-gcc-libs-core=5.3.0=7 - m2w64-gmp=6.1.0=2 - m2w64-libwinpthread-git=5.0.0.4634.697f757=2 - markupsafe=2.0.1=py39hb82d6ee_0 - matplotlib-inline=0.1.2=pyhd8ed1ab_2 - mistune=0.8.4=py39hb82d6ee_1004 - mkl=2021.3.0=haa95532_524 - mkl-service=2.4.0=py39hb82d6ee_0 - msys2-conda-epoch=20160418=1 - nbclient=0.5.4=pyhd8ed1ab_0 - nbconvert=6.1.0=py39hcbf5309_0 - nbformat=5.1.3=pyhd8ed1ab_0 - nest-asyncio=1.5.1=pyhd8ed1ab_0 - notebook=6.4.3=pyha770c72_0 - numba=0.53.1=py39hb8cd55e_1 - numpy=1.21.2=py39h6635163_0 - openssl=1.1.1k=h8ffe710_1 - packaging=21.0=pyhd8ed1ab_0 - pandoc=2.14.1=h8ffe710_0 - pandocfilters=1.4.2=py_1 - pango=1.48.9=hd84fcdd_0 - parso=0.8.2=pyhd8ed1ab_0 - pcre=8.45=h0e60522_0 - pickleshare=0.7.5=py_1003 - pip=21.2.4=pyhd8ed1ab_0 - pixman=0.40.0=h8ffe710_0 - prometheus_client=0.11.0=pyhd8ed1ab_0 - prompt-toolkit=3.0.19=pyha770c72_0 - pthread-stubs=0.4=hcd874cb_1001 - pycparser=2.20=pyh9f0ad1d_2 - pygments=2.10.0=pyhd8ed1ab_0 - pyparsing=2.4.7=pyh9f0ad1d_0 - pyrsistent=0.17.3=py39hb82d6ee_2 - python=3.9.6=h7840368_1_cpython - python-dateutil=2.8.2=pyhd8ed1ab_0 - python-graphviz=0.17=pyhaef67bd_0 - python_abi=3.9=2_cp39 - pywin32=301=py39hb82d6ee_0 - pywinpty=1.1.3=py39h99910a6_0 - pyzmq=22.2.1=py39he46f08e_0 - scipy=1.7.1=py39hc0c34ad_0 - send2trash=1.8.0=pyhd8ed1ab_0 - setuptools=57.4.0=py39hcbf5309_0 - six=1.16.0=pyh6c4a22f_0 - sqlite=3.36.0=h8ffe710_0 - terminado=0.11.1=py39hcbf5309_0 - testpath=0.5.0=pyhd8ed1ab_0 - tornado=6.1=py39hb82d6ee_1 - traitlets=5.0.5=py_0 - tzdata=2021a=he74cb21_1 - ucrt=10.0.20348.0=h57928b3_0 - vc=14.2=hb210afc_5 - vs2015_runtime=14.29.30037=h902a5da_5 - wcwidth=0.2.5=pyh9f0ad1d_2 - webencodings=0.5.1=py_1 - wheel=0.37.0=pyhd8ed1ab_1 - winpty=0.4.3=4 - xorg-kbproto=1.0.7=hcd874cb_1002 - xorg-libice=1.0.10=hcd874cb_0 - xorg-libsm=1.2.3=hcd874cb_1000 - xorg-libx11=1.7.2=hcd874cb_0 - xorg-libxau=1.0.9=hcd874cb_0 - xorg-libxdmcp=1.1.3=hcd874cb_0 - xorg-libxext=1.3.4=hcd874cb_1 - xorg-libxpm=3.5.13=hcd874cb_0 - xorg-libxt=1.2.1=hcd874cb_2 - xorg-xextproto=7.3.0=hcd874cb_1002 - xorg-xproto=7.0.31=hcd874cb_1007 - xz=5.2.5=h62dcd97_1 - zeromq=4.3.4=h0e60522_0 - zipp=3.5.0=pyhd8ed1ab_0 - zlib=1.2.11=h62dcd97_1010 - zstd=1.5.0=h6255e5f_0 - pip: - arviz==0.11.2 - cachetools==4.2.2 - cftime==1.5.0 - cycler==0.10.0 - dill==0.3.4 - fastprogress==1.0.0 - filelock==3.0.12 - kiwisolver==1.3.1 - matplotlib==3.4.3 - netcdf4==1.5.7 - pandas==1.3.2 - patsy==0.5.1 - pillow==8.3.1 - pymc3==3.11.3 - pytz==2021.1 - semver==2.13.0 - theano-pymc==1.1.2 - typing-extensions==3.10.0.0 - xarray==0.19.0

[sreedat]

@twiecki upon further testing I did notice the following warning in pymc3 environment built with python=3.8:
WARNING (theano.tensor.blas): We did not find a dynamic library in the library_dir of the library we use for blas. If you use ATLAS, make sure to compile it with dynamics library.

I do not get this warning in pymc3 environment built with python=3.9 - I have to test some more!

[sreedat]

@twiecki I was able to resolve this warning, although I do not understand why, by installing m2w64-openblas using

conda install -c msys2 m2w64-openblas. I was getting this error in the Python=3.9 env too. I was not seeing in the correct place. Sorry for the confusion.

The time for sampling is interesting to note:
pymc3=3.11.2 took 5 seconds to sample 2 chains on 1 core; pymc3=3.11.3 took 3 seconds for the same. There clearly seem to be some improvements in version 3.11.3

[curtywang]

Hi All,

Just wanted to add some additional instructions for the Windows Installation Guide. I just wanted to add that even using RTools with gcc 8.3.0, there seems to be a problem with compilation (same error as above, "invalid register ...") for Xeon users on Windows 10. GCC seems to have fixed this bug in version 9.0+ (see the discourse post that gave me the idea to try other compilers (including clang and VS Tools).

To get gcc working properly on Windows 10 with Xeon processors:

1. Do not install m2w64-toolchain from conda-forge.
2. Instead, install MSYS2 following the install directions: https://www.msys2.org/
3. Add your path for MSYS2's MinGW64 bin folder (for instance, mine is C:\msys64\mingw64\bin) to the user PATH.
4. Then, close and re-open the conda/mamba prompt.
5. Then, install pymc3 and theano-pymc using pip.
6. Open ipython/python/jupyter and check that theano is using the correct CXX by running the following:

import theano
print(theano.config.cxx)

[twiecki]

Thanks @curtywang. msys2 should also be available through condo-forge, have you tried installing that?

[curtywang]

Thanks @curtywang. msys2 should also be available through condo-forge, have you tried installing that?

Yes, but the msys2 version of gcc on conda repos for Windows has stalled at 5.3.0.

[twiecki]

Any idea why m2w64 doesn't work for you?

[curtywang]

Any idea why m2w64 doesn't work for you?

It's the same compilation error as above, even if I remove everything except the m2w64 paths from PATH and check the CXX being used. If I use clang or MSVC, theano-pymc is able to compile models (but not use any optimizations). I suspect it has to do with GCC and Windows more than anything, given that even numpy seems to have faced this problem for GCC 8.3.0. The version of GCC on my MSYS2 install now is 10.3.0.

[zperko]

@sreedat this was the only approach that managed to produce at least a half working pymc3 installation on my Windows 10. However, it seems even this approach produces an unstable installation, with some things working some not. Doing a quick check reproducing https://towardsdatascience.com/bayesian-statistics-101-4c4bc5fde1e1 examples the standard linear regressions work, but the binomial examples do not, neither in the GLM mode nor when implementing from scratch. Both give 'Compilation failed (return status=1): C:\Users\\AppData\Local\Temp\ccKUWQjU.s: Assembler messages:\r. C:\Users\\AppData\Local\Temp\ccKUWQjU.s:367: Error: invalid register for .seh_savexmm\r. ', 'FunctionGraph(Elemwise{eq,no_inplace}(TensorConstant{[0 0 1 1 0..1 1 1 0 1]}, <TensorType(int8, (True,))>))'" error.

Install:

• theano-pymc 1.1.2
• pymc3 3.11.2
• python 3.8.12
• numba 0.54.1
• arviz 0.11.4

[curtywang]

@sreedat this was the only approach that managed to produce at least a half working pymc3 installation on my Windows 10. However, it seems even this approach produces an unstable installation, with some things working some not. Doing a quick check reproducing https://towardsdatascience.com/bayesian-statistics-101-4c4bc5fde1e1 examples the standard linear regressions work, but the binomial examples do not, neither in the GLM mode nor when implementing from scratch. Both give 'Compilation failed (return status=1): C:\Users*\AppData\Local\Temp\ccKUWQjU.s: Assembler messages:\r. C:\Users*\AppData\Local\Temp\ccKUWQjU.s:367: Error: invalid register for .seh_savexmm\r. ', 'FunctionGraph(Elemwise{eq,no_inplace}(TensorConstant{[0 0 1 1 0..1 1 1 0 1]}, <TensorType(int8, (True,))>))'" error.

Install:

• theano-pymc 1.1.2
• pymc3 3.11.2
• python 3.8.12
• numba 0.54.1
• arviz 0.11.4

Let me try the binomial example and see if it works for the MSYS2 GCC 10.3.0 setup.

[curtywang]

@sreedat this was the only approach that managed to produce at least a half working pymc3 installation on my Windows 10. However, it seems even this approach produces an unstable installation, with some things working some not. Doing a quick check reproducing https://towardsdatascience.com/bayesian-statistics-101-4c4bc5fde1e1 examples the standard linear regressions work, but the binomial examples do not, neither in the GLM mode nor when implementing from scratch. Both give 'Compilation failed (return status=1): C:\Users*\AppData\Local\Temp\ccKUWQjU.s: Assembler messages:\r. C:\Users*\AppData\Local\Temp\ccKUWQjU.s:367: Error: invalid register for .seh_savexmm\r. ', 'FunctionGraph(Elemwise{eq,no_inplace}(TensorConstant{[0 0 1 1 0..1 1 1 0 1]}, <TensorType(int8, (True,))>))'" error.
Install:

• theano-pymc 1.1.2
• pymc3 3.11.2
• python 3.8.12
• numba 0.54.1
• arviz 0.11.4

Let me try the binomial example and see if it works for the MSYS2 GCC 10.3.0 setup.

@zperko On the MSYS2 GCC 10.3.0 setup, the Bernoulli model seems to have worked:

import arviz as az
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import pymc3 as pm
# Setting a random seed for reproducibility
np.random.seed(23)
# True parameter values
alpha = -0.8
beta = [-1, 1.5]
sigma = 1
# Number of observations
size = 500
# Predictor variables
X1 = np.random.randn(size)
X2 = np.random.randn(size) * 0.8
# Outcome variable
Y = alpha + beta[0] * X1 + beta[1] * X2 + np.random.randn(size) * sigma
# Putting our original data into a dataframe (we'll use it later on)
df = pd.DataFrame(
    data = np.array([X1, X2, Y]),
    index=['X1', 'X2', 'Y']
).T

outputs just the usual WARNING (theano.tensor.blas): Using NumPy C-API based implementation for BLAS functions..

# Creating a binary variable
df['Y_bin']=df['Y']>df['Y'].mean()

with pm.Model() as model_log:
    # Priors
    alpha = pm.Normal("alpha", mu=0, sigma=10)
    beta = pm.Normal("beta", mu=0, sigma=10, shape=2)
    # Likelihood    
    p = pm.Deterministic('p', pm.math.sigmoid(alpha + beta[0] * X1 + beta[1] * X2))
    Y_obs = pm.Bernoulli("Y_obs", p, observed=df['Y_bin'])
    
    # Posterior
    trace = pm.sample(100, return_inferencedata=False, chains=4)
    az.plot_trace(trace)

Is able to run the sampler in whole:

Only 100 samples in chain.
Auto-assigning NUTS sampler...
Initializing NUTS using jitter+adapt_diag...
Multiprocess sampling (4 chains in 4 jobs)
NUTS: [beta, alpha]

100.00% [4400/4400 00:07<00:00 Sampling 4 chains, 0 divergences]

Sampling 4 chains for 1_000 tune and 100 draw iterations (4_000 + 400 draws total) took 47 seconds.

[zperko]

@curtywang Thanks for the fast reply and try. I've tried to replicate your solution, installing msys2, but in runtime now all examples give a "cc1plux.exe system error", complaining about not finding zlib1.dll. Despite the dll file being in /mingw64/bin. Really weird.

[zperko]

@curtywang Managed to solve it, thanks for the help. Apparently the issue was with theano config not using the msys g++ compiler, was still pointing to my Strawberry install.
One sidenote though, seems like msys2 is pretty slow, at least for the regression problems that both are able to run it's factor 3-4 slower.

[curtywang]

@twiecki I just confirmed that it might just be a newer Intel microarch problem for Windows m2w64-toolchain. I had another user who has a Ice Lake (Core i5-1035G4) and I myself have a Xeon Silver 4216 (Cascade Lake). It seems as though the msys2 g++ will solve the problem.

@zperko I have the same problem, Windows multiprocessing is slower than just running the sampler singlethreaded, sometimes.

[fonnesbeck]

I've been trying to get this working on Win11 using the advice above, but I run into compilation errors:

~\miniforge3\envs\pymc_env\lib\site-packages\theano\link\c\cmodule.py in compile_str(module_name, src_code, location, include_dirs, lib_dirs, libs, preargs, py_module, hide_symbols)
   2544             # difficult to read.
   2545             compile_stderr = compile_stderr.replace("\n", ". ")
-> 2546             raise Exception(
   2547                 f"Compilation failed (return status={status}): {compile_stderr}"
   2548             )

Exception: ('Compilation failed (return status=1): C:\\Users\\fonne\\AppData\\Local\\Temp\\ccabwyOj.s: Assembler messages:\r. C:\\Users\\fonne\\AppData\\Local\\Temp\\ccabwyOj.s:114: Error: invalid register for .seh_savexmm\r. ', 'FunctionGraph(Elemwise{le,no_inplace}(TensorConstant{[0 0 1 0 0..0 0 1 1 1]}, <TensorType(int8, (True,))>))')

This is having installed m2w64-toolchain from the msys2 repository. This is a Surface Pro 8, so 11th gen core i7.

[fonnesbeck]

Sorry accidentally closed the issue. Reopening.

[hectormz]

@jedludlow love to hear it!

[twiecki]

We now support the most recent aesara version on main: #5757

@jedludlow Is there a way for you to test that this fix isn't required anymore without a pip/conda release?

[jedludlow]

@twiecki, yes, there is. Here's how to do it.

First, make sure to rename any .aesararc file you have in your home directory to something else so that it will be ignored by Aesara. Next, this environment.yml file will produce the minimal conda environment plus install pymc directly from the tip of the main branch on GitHub:

name: pymc_env
channels:
  - conda-forge
dependencies:
  - python
  - libpython
  - m2w64-toolchain
  - mkl-service
  - numba
  - pip
  - pip:
    - "git+https://github.com/pymc-devs/pymc.git"

Save this environment.yml file to a directory of your choosing, and from that directory invoke

conda env create

which will create the environment and install pymc from the tip of the main branch over pip.

Creating this environment and activating it produces a working PyMC install for me with no compiler failures or the WARNING (aesara.tensor.blas): Using NumPy C-API based implementation for BLAS functions message. I can also verify that Aesara 2.6.6 was installed in the process.

Additional diagnostics can be obtained from this command (after activating the pymc_env environment you've created):

python -m aesara.misc.check_blas

That will produce a long printout of BLAS diagnostics and a performance check, but the important line for us today is the one that talks about Aesara BLAS flags. Look for something like like this:

Some Aesara flags:
    blas__ldflags= -L"D:\miniforge3\envs\pymc_env\Library\bin" -lmkl_core -lmkl_intel_thread -lmkl_rt

If MKL is being auto-discovered appropriately, the -l entries should all point to MKL libraries.

[twiecki]

@jedludlow Thanks for confirming, so we can simplify the installation instructions even more. Want to edit them accordingly?

[jedludlow]

@twiecki, I'd be happy to update the installation instructions, but I would appreciate some input on which simplifications the community would suggest.

• I used an environment.yml file above, but the current instructions create the conda environment and install pymc with a couple of command line calls. Is the use of an environment file considered a simplification?
• In theory we shouldn't need to create .aesararc files to solve BLAS warnings now. Would you suggest removing the workaround instructions for BLAS warnings from the installation instructions as a simplification? Or would you rather preserve those tricks to help people out when things don't go as planned?

[twiecki]

I used an environment.yml file above, but the current instructions create the conda environment and install pymc with a couple of command line calls. Is the use of an environment file considered a simplification?

They should be written assuming 4.0.0 is out on conda-forge.

In theory we shouldn't need to create .aesararc files to solve BLAS warnings now. Would you suggest removing the workaround instructions for BLAS warnings from the installation instructions as a simplification? Or would you rather preserve those tricks to help people out when things don't go as planned?

I would remove them.

[twiecki]

@jedludlow I gave it a shot.

[jedludlow]

@twiecki I made just a couple of additional edits to remove any traces of pip or the PyMC3 option. I assume that was your goal.

[hectormz]

@jedludlow are the changes to aesara that remove the need to specify a BLAS flag universal? I know Anaconda ships a flavor of BLAS with numpy, but if I'm using pymc with vanilla python (and I'm using intel-numpy or no BLAS at all) am I still covered? I acknowledge that Anaconda makes things easier for most people, but still like the ability to have things mostly work if I'm using vanilla python and come with my own mingw etc.

[jedludlow]

@hectormz, I suspect there will be complex and custom system configurations that confuse Aesara's automatic BLAS detection logic, so I would expect that sometimes manual BLAS flag setting will still be required. I would suggest trying things out on your configuration, and let us know how it goes!

[sreedta8]

@jedludlow I can report successful installation of pymc on Windows 10 laptop using the new environment.yml instructions (that do not need .aesararc creation). No BLAS warnings appeared.

[sreedta8]

@jedludlow python -m aesara.misc.check_blas shows blas being linked to mkl

[hectormz]

@jedludlow I'm always happy to be a guinea pig!

[sreedta8]

@jedludlow @twiecki @fonnesbeck see this issue I have opened on testing the latest version of pymc v4 and aesara-2.6.6 on both Windows10 and an Ubuntu20.10 laptops

To provide a brief synopsis: Using the same package environments (as nearly exact as possible) I tested the latest versions of pymc on Windows and Ubuntu laptops. On the Stochastic Volatility exercise I found that the time taken for Sampling on Windows (average13 minutes) is a mere fraction of the time taken on Ubuntu ( average 1 hour 30 minutes).

Both laptops have the same RAM (8GB) and Ananconda was the only application running at that time. If any of you could try to replicate and see what happens it would be very helpful

[twiecki]

We now have a conda-forge package that should install mkl if you just do conda install -c conda-forge pymc. Can someone with windows confirm that it installs and uses mkl?

[sreedta8]

@twiecki I can confirm installation and binding of BLAS to MKL on Windows10 using the instructions @jedludlow provided

[twiecki]

@sreedat I'm interested in the case that's not following @jedludlow's instrucitons but jsut creating a fresh env and running:
conda install -c conda-forge pymc
And making sure MKL is isntalled and used.

[jessegrabowski]

I tried a fresh install on Windows 10 by running conda create -n pymc_env -c conda-forge pymc libpython mkl-service numba. python -m aesara.misc.check_blas shows MKL is installed and used. So I can confirm this all seems to work.

The "fancy install" instructions currently on the windows installation guide don't seem to work, though. I tried omitting m2w64-toolchain as suggested and instead put g++ version 12.1.0 built from msys2 on my system path. m2w64-toolchain ends up getting installed along with everything, whether or not I ask for it. Asking conda to remove it removes everything related to pymc and aesara, so it seems to be a requirement somewhere.

I pointed aesara to the msys2 g++ with cxx = C:/msys64/mingw64/bin/g++.exe in aesararc.txt, but when I run pm.sample I get the following output:

Using built-in specs.
COLLECT_GCC=C:\Users\Jesse\miniconda3\envs\pymc_env\Library\mingw-w64\bin\gfortran.exe
COLLECT_LTO_WRAPPER=C:/Users/Jesse/miniconda3/envs/pymc_env/Library/mingw-w64/bin/../lib/gcc/x86_64-w64-mingw32/5.3.0/lto-wrapper.exe
Target: x86_64-w64-mingw32
Configured with: ../gcc-5.3.0/configure --prefix=/mingw64 --with-local-prefix=/mingw64/local --build=x86_64-w64-mingw32 --host=x86_64-w64-mingw32 --target=x86_64-w64-mingw32 --with-native-system-header-dir=/mingw64/x86_64-w64-mingw32/include --libexecdir=/mingw64/lib --with-gxx-include-dir=/mingw64/include/c++/5.3.0 --enable-bootstrap --with-arch=x86-64 --with-tune=generic --enable-languages=c,lto,c++,objc,obj-c++,fortran,ada --enable-shared --enable-static --enable-libatomic --enable-threads=posix --enable-graphite --enable-fully-dynamic-string --enable-libstdcxx-time=yes --disable-libstdcxx-pch --disable-libstdcxx-debug --enable-version-specific-runtime-libs --disable-isl-version-check --enable-lto --enable-libgomp --disable-multilib --enable-checking=release --disable-rpath --disable-win32-registry --disable-nls --disable-werror --disable-symvers --with-libiconv --with-system-zlib --with-gmp=/mingw64 --with-mpfr=/mingw64 --with-mpc=/mingw64 --with-isl=/mingw64 --with-pkgversion='Rev5, Built by MSYS2 project' --with-bugurl=https://sourceforge.net/projects/msys2 --with-gnu-as --with-gnu-ld
Thread model: posix
gcc version 5.3.0 (Rev5, Built by MSYS2 project)

So it's still pointing to the mingw32 installation version 5.3.0 associated with the conda environment rather than the newer msys2 version. I am also getting some new warnings pop up when I import aesara:

Could not locate executable g77
Could not locate executable f77
Could not locate executable ifort
Could not locate executable ifl
Could not locate executable f90
Could not locate executable efl

These issues don't seem to affect my ability to sample models, but they are also new and exciting.

[twiecki]

@jessegrabowski Can you try with a new environment and instead:

conda create -n pymc_env2 -c conda-forge pymc

[twiecki]

@jessegrabowski I also changed the fancy installation instructions, they should work again (pymc-base is a package that does not specify compilers to install).

[jessegrabowski]

@jessegrabowski Can you try with a new environment and instead:

conda create -n pymc_env2 -c conda-forge pymc

Output of check_blas:

Some Aesara flags:
    blas__ldflags= -L"C:\Users\Jesse\miniconda3\envs\pymc_env2\Library\bin" -lmkl_core -lmkl_intel_thread -lmkl_rt
    compiledir= C:\Users\Jesse\AppData\Local\Aesara\compiledir_Windows-10-10.0.19044-SP0-Intel64_Family_6_Model_158_Stepping_13_GenuineIntel-3.10.4-64
    floatX= float64
    device= cpu
Some OS information:
    sys.platform= win32
    sys.version= 3.10.4 | packaged by conda-forge | (main, Mar 24 2022, 17:32:50) [MSC v.1929 64 bit (AMD64)]
    sys.prefix= C:\Users\Jesse\miniconda3\envs\pymc_env2
Some environment variables:
    MKL_NUM_THREADS= None
    OMP_NUM_THREADS= None
    GOTO_NUM_THREADS= None

Looks like it works. I still got the "could not locate" warnings as aesara was imported in the test script. Not sure if that's a problem on my end, though; it's new since I did this msys2 stuff. Will try a fancy-install with pymc-base and see what happens there.

[jessegrabowski]

m2w64-toolchain still gets installed when using pymc-base. Would be nice if another windows user can confirm so I know I'm not doing something wrong on my end.

[sreedta8]

@twiecki got it. I will test accordingly and confirm.

@jessegrabowski I will share details of my Windows 10 install

[jedludlow]

I believe Aesara's dependency list may be the reason that m2w64-toolchain is getting installed even when asking for pymc-base under the Fancy Install instructions. Here's the output of conda search aesara=2.6.6 --info:

aesara 2.6.6 py39h7fcc1c2_0                                                                             
---------------------------                                                                             
file name   : aesara-2.6.6-py39h7fcc1c2_0.tar.bz2                                                       
name        : aesara                                                                                    
version     : 2.6.6                                                                                     
build       : py39h7fcc1c2_0                                                                            
build number: 0                                                                                         
size        : 1.9 MB                                                                                    
license     : BSD-3-Clause                                                                              
subdir      : win-64                                                                                    
url         : https://conda.anaconda.org/conda-forge/win-64/aesara-2.6.6-py39h7fcc1c2_0.tar.bz2         
md5         : a90f08442ce38874f568b5a0fad63ec0                                                          
timestamp   : 2022-05-10 02:05:04 UTC                                                                   
dependencies:                                                                                           
  - cons                                                                                                
  - etuples                                                                                             
  - filelock                                                                                            
  - libblas >=3.8.0,<4.0a0                                                                              
  - liblapack >=3.8.0,<4.0a0                                                                            
  - libpython >=2.0                                                                                     
  - logical-unification                                                                                 
  - m2w64-toolchain_win-64                                                                              
  - minikanren                                                                                          
  - numpy >=1.9.1                                                                                       
  - python >=3.9,<3.10.0a0                                                                              
  - python_abi 3.9.* *_cp39                                                                             
  - scipy >=0.14,<1.8.0                                                                                 
  - setuptools                                                                                          
  - six >=1.9.0                                                                                         
  - typing_extensions

Note the inclusion of m2w64-toolchain_win-64 in the list.

[twiecki]

Yes, we could do the same trick with base there too.

…

On Mon, Jun 6, 2022, 17:04 Jed Ludlow ***@***.***> wrote: I believe Aesara's dependency list may be the reason that m2w64-toolchain is getting installed even when asking for pymc-base under the Fancy Install instructions. Here's the output of conda search aesara=2.6.6 --info: aesara 2.6.6 py39h7fcc1c2_0 --------------------------- file name : aesara-2.6.6-py39h7fcc1c2_0.tar.bz2 name : aesara version : 2.6.6 build : py39h7fcc1c2_0 build number: 0 size : 1.9 MB license : BSD-3-Clause subdir : win-64 url : https://conda.anaconda.org/conda-forge/win-64/aesara-2.6.6-py39h7fcc1c2_0.tar.bz2 md5 : a90f08442ce38874f568b5a0fad63ec0 timestamp : 2022-05-10 02:05:04 UTC dependencies: - cons - etuples - filelock - libblas >=3.8.0,<4.0a0 - liblapack >=3.8.0,<4.0a0 - libpython >=2.0 - logical-unification - m2w64-toolchain_win-64 - minikanren - numpy >=1.9.1 - python >=3.9,<3.10.0a0 - python_abi 3.9.* *_cp39 - scipy >=0.14,<1.8.0 - setuptools - six >=1.9.0 - typing_extensions Note the inclusion of m2w64-toolchain_win-64 in the list. — Reply to this email directly, view it on GitHub <#4937 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFETGBQIS5PR3U26TF762LVNYHQDANCNFSM5CJBRBYA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[ghost]

@twiecki installing pymc from conda-forge works fine for me on Win11

[jedludlow]

@twiecki, installing pymc from conda-forge using the Simple Install instructions works fine for me on Windows 10, and I can verify that MKL was automatically discovered by Aesara to link against.

[twiecki]

Thanks everyone! I think we can close this then.