Windows builds

[jjhelmus]

Currently conda-forge does not build NumPy on Windows. I'm guessing this is due to OpenBLAS's poor support of the platform.

[jjhelmus]

One options for a BLAS / LAPACK on Windows might be ATLAS. I believe this is what the NumPy PyPI wheels files use on Windows. These will not give optimal performance on may systems but may be "good enough".

[jakirkham]

xref: conda-forge/openblas-feedstock#2

[jakirkham]

Actually OpenBLAS has pretty good support for Windows.

It provides CMakeLists.txt. Downsides are Visual Studio does not understand AT&T assembly. Meaning many kernels would be much slower than they would be with assembly. Also Visual Studio does not do Fortran, which impacts LAPACK. Though I believe OpenBLAS has a C implementation of LAPACK (LAPACKE) that could be used.

Alternatively (and maybe this is what we should do) is use MSYS2/gcc to build on Windows. This would allow AT&T assembly and Fortran to be used. There are concerns raised in the referenced thread about UCRT compatibility with MinGW-w64. Though it remains unclear how important these are generally vs for OpenBLAS alone. When talking with @mingwandroid, it sounded like we may not need to worry about this at all. Though I don't really understand the issues well enough to say and it still seems like others are concerned.

So those are the main options for OpenBLAS. Please feel free to weigh in on which one is used.

[jjhelmus]

Yes, if MSYS2/gcc is an option on Windows I would be in favor of that options. My recollection is the with the C implementation, i.e. without assembly, the performance of OpenBLAS is significantly diminished. I'm a bit concerned about the UCRT issue, especially on Python 3.5 and 3.6, but I do not understand the details enough to voice anything more than concern.

[rgommers]

• The NumPy PyPI wheels are built with MSVC and use ATLAS indeed: https://github.com/numpy/windows-wheel-builder (build script) and https://github.com/MacPython/numpy-wheels/blob/master/appveyor.yml (automation)
• UCRT compatibility for OpenBLAS may not be critical, but IIRC it's not going to work for NumPy itself due to issues with file streams (and maybe more).
• The other MinGW issue for SciPy and other packages with Fortran code is where to put the gcclib DLL (Mingw Gfortran on Windows? scipy/scipy#2829 (comment)). This is a pain for wheels, but not an issue for conda I'd think.

[jakirkham]

UCRT compatibility for OpenBLAS may not be critical, but IIRC it's not going to work for NumPy itself due to issues with file streams (and maybe more).

Hmm...that's a good point. Though we should still be able to build NumPy with MSVC. Would the file streams point be a concern with SciPy (given things in scipy.io for instance)? I'd expect we would want to use MinGW there for Fortran.

[rgommers]

Would the file streams point be a concern with SciPy

I suspect so, the problematic file will be scipy/io/matlab/streams.pyx.

[jakirkham]

Is there some way to mix and match MSVC and MinGW-w64? Perhaps just use the Fortran compiler from MinGW-w64 and otherwise use MSVC when building SciPy. Does this even make sense or is this just a recipe for disaster?

[kain88-de]

I've just check the language distribution of scipy with sloccount. There is no fortran code in scipy/io. There is only fortran code in sparse, stats, linalg, optimize, odr, _build_utils, integrate and fftpack. If these fortran files are compiled to individual python modules like c-extensions are then mixing the compilers might work.

[rgommers]

Is there some way to mix and match MSVC and MinGW-w64? Perhaps just use the Fortran compiler from MinGW-w64 and otherwise use MSVC when building SciPy. Does this even make sense or is this just a recipe for disaster?

IIRC this was discussed a while ago, as an alternative to MingwPy. It's possible in theory but required quite a bit of work (can't remember what exactly). Right now the only options are:

• MSVC + ifort
• mingw-w64 for everything

[jakirkham]

Do you happen to have any references to this discussion?

[rgommers]

Can't find anything public - @cournape had experimented with having gfortran interface directly with MSVC, maybe he can comment.

[cournape]

The idea was based on the precept that scipy used very little of the fortran runtime, which was true 7 years ago, I don't know about now.

[rgommers]

I don't think a whole lot changed on that front in the last 7 years. We haven't added many new Fortran extensions, the only one I can think of is linalg.interpolative

[rolk]

@rgommers If you are really experimental and adventurous you could perhaps use the compiler at https://github.com/flang-compiler/flang (it's an LLVM fork of the PGI compiler) with somehing like:

flang --target=x86_64-w64-windows-msvc -emit-llvm -c foo.f95 -o foo.bc
llc -filetype=obj foo.bc -o foo.obj

to generate COFF and then use the MSVC linker.

[rgommers]

@rolk based on recent feedback from the Flang devs it's not yet Windows compatible (they do seem interested, but no concrete plans yet). Would be really great if someone started work on that, but that won't be me.

[ocefpaf]

Done in #77