Set use_blas64 to false to fix MKL.jl interaction numpy problem #35

fgerick · 2020-03-06T10:35:52Z

same as the other pr, updated to latest master.

carstenbauer · 2020-03-06T10:54:48Z

Would be good to link to the other PR.

aminya · 2020-03-12T21:40:35Z

This builds now with the new version. I don't see any reason for not merging this.

carstenbauer · 2020-03-13T07:57:37Z

What are the side effects of USE_BLAS64=false again? (apart form fixing the numpy interplay)

UPDATE: Oh, I asked this before. Anyways, I guess this 32-bit "limitation" should be mentioned in the README if this gets merged, no?

fgerick · 2020-03-13T08:34:04Z

is it easy to include a flag in the build for the individual user to choose?

KristofferC · 2020-03-13T12:55:45Z

src/install.jl


    @assert libblas_idx !== nothing && liblapack_idx !== nothing

    lines[libblas_idx] = "const libblas_name = $(repr(name))"
    lines[liblapack_idx] = "const liblapack_name = $(repr(name))"
+    if useblas64_idx != nothing
+        lines[useblas64_idx] = "const USE_BLAS64 = false"


This will set it to use 32 bit integers even when reverting to normal sysimage? Anyway, I can fix it up.

Bump. @KristofferC Not sure I understand this - but it would be nice to get this PR merged.

carstenbauer · 2020-03-13T16:31:53Z

is it easy to include a flag in the build for the individual user to choose?

It's probably fine to simply make it the default. But I think it's worth mentioning in the README.md (or somewhere else?).

aminya · 2020-05-26T06:55:13Z

Some time ago I did some comparison between BLAS 64 and 32, and the performance gain of BLAS 64 was noticeable. I think BLAS 32 should not be the default behavior, but only be enabled by an option,

ViralBShah · 2020-05-30T15:19:47Z

Some time ago I did some comparison between BLAS 64 and 32, and the performance gain of BLAS 64 was noticeable. I think BLAS 32 should not be the default behavior, but only be enabled by an option,

It's a bit odd. Can you share the benchmarks?

ViralBShah · 2020-05-30T15:20:35Z

Even if there is a performance issue, we probably still need to ship 32-bit as the default, given the number of crash complaints we receive. We should file the performance issue with Intel.

ViralBShah · 2020-05-30T15:23:01Z

What is holding this PR from being merged?

ViralBShah · 2020-05-30T15:29:01Z

@stevengj If we merge this - I assume it will solve all the issues we are having on the python side.

stevengj · 2020-05-30T17:39:48Z

Probably.

aminya · 2020-05-30T21:16:48Z

@ViralBShah I created an issue to remind myself to do some benchmarks
#50

If switching to 64 bit version be a matter of a special flag or option, definitely that is easy enough for everyone (like me) who does not use Python to use the 64bit version.

fgerick · 2020-06-13T17:44:53Z

I have added the option to build MKL.jl with the 64bit version of MKL by setting ENV["USE_BLAS64"]=true before building. Also, the install script reverts the alteration by the installation. I added a small statement into the README, but maybe someone wants to say something about performance issues...

fgerick · 2020-06-13T18:19:17Z

I guess once this is merged the comment in the readme: "frequently encountered compatibility issues" is already deprecated...

fgerick · 2020-07-06T21:01:26Z

Is there anything left to be done here? I'd be happy to see this move on and make MKL with Julia more end-user friendly. I'm still avoiding MKL because of Arpack.jl but I'm hoping if MKL.jl has less struggles steps will be taken there also.

ViralBShah · 2020-07-06T21:45:01Z

We need @andreasnoack to look at this - whenever he has a moment.

ViralBShah · 2020-08-21T18:29:39Z

@KristofferC Do you understand this well enough to know if it will work right? It would be nice to use MKL as 32-bit by default so as to not cause trouble with python.

ViralBShah · 2020-10-12T17:44:50Z

Failing on nightly.

andreasnoack · 2020-10-12T19:15:43Z

@KristofferC Is the failure on nightly known?

ViralBShah · 2020-10-12T20:17:30Z

https://travis-ci.org/github/JuliaComputing/MKL.jl/jobs/735062657#L262

ERROR: ArgumentError: Package Artifacts not found in current path:
- Run `import Pkg; Pkg.add("Artifacts")` to install the Artifacts package.

andreasnoack · 2020-10-13T08:00:05Z

I suppose it's an issue with PackageCompiler on 1.6? I'll merge, as tests are passing on 1.5.

KristofferC · 2020-10-13T09:22:19Z

How will this work with MKLSparse? Will we lose the ability to do Int64 sparse matmuls? I'm not really sure how this interacts with the sparse MKL part.

andreasnoack · 2020-10-13T09:44:15Z

How will this work with MKLSparse? Will we lose the ability to do Int64 sparse matmuls?

@fgerick Would you be able to test this?

KristofferC · 2020-10-13T09:46:41Z

It seems MKL at compile time uses BlasInt: https://github.com/JuliaSparse/MKLSparse.jl/blob/6060cd08a2586b275c95aac650c379ff48130ef7/src/BLAS/level_2_3/generator.jl#L36

So I guess that means it would be restricted to Int32 (and silently for Int64 use the Julia native one)?

ViralBShah · 2020-10-13T13:53:25Z

Yes, presumably. I think that is also the behaviour we need by default, otherwise it leads to all those crashes in matplotlib.

KristofferC · 2020-10-13T14:16:35Z

Yeah, all I'm saying is that right now, if someone installs this it most likely effectively turns off MKLSparse since everyone is using the default Int64 for their sparse matrices. And I guess many people that use MKLSparse use MKL (and vice versa).

ViralBShah · 2020-10-13T21:24:37Z

Oh I didn't quite understand that. Good to know. Is there a way to opt into the Int64 version?

use_blas64 false to fix mkl.jl interaction numpy problem

c61ada2

aminya mentioned this pull request Mar 6, 2020

package not registered? #13

Closed

KristofferC mentioned this pull request Mar 12, 2020

Segmentation Fault #39

Closed

aminya approved these changes Mar 13, 2020

View reviewed changes

andreasnoack mentioned this pull request Mar 13, 2020

Use LP64 isntead of ILP64 to avoid issues when linking to libraries #11

Closed

KristofferC reviewed Mar 13, 2020

View reviewed changes

aminya mentioned this pull request Apr 18, 2020

Where did the caveats section go? #41

Closed

aminya mentioned this pull request May 26, 2020

PyPlot crashes after Julia MKL is installed #46

Closed

ViralBShah changed the title ~~use_blas64 false to fix MKL.jl interaction numpy problem~~ Set use_blas64 to false to fix MKL.jl interaction numpy problem May 30, 2020

make 32bit MKL optional

bfb14b6

jakebolewski mentioned this pull request Jun 20, 2020

Unit-tests with CI Update CliMA/CalibrateEmulateSample.jl#53

Merged

KristofferC mentioned this pull request Jul 13, 2020

Depend on MKL_jll JuliaSparse/MKLSparse.jl#22

Merged

andreasnoack closed this Oct 12, 2020

andreasnoack reopened this Oct 12, 2020

andreasnoack closed this Oct 13, 2020

andreasnoack reopened this Oct 13, 2020

andreasnoack merged commit 84ae4d4 into JuliaLinearAlgebra:master Oct 13, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Set use_blas64 to false to fix MKL.jl interaction numpy problem #35

Set use_blas64 to false to fix MKL.jl interaction numpy problem #35

fgerick commented Mar 6, 2020

carstenbauer commented Mar 6, 2020

aminya commented Mar 12, 2020

carstenbauer commented Mar 13, 2020 •

edited

Loading

fgerick commented Mar 13, 2020

KristofferC Mar 13, 2020

ViralBShah May 30, 2020

carstenbauer commented Mar 13, 2020 •

edited

Loading

aminya commented May 26, 2020 •

edited

Loading

ViralBShah commented May 30, 2020

ViralBShah commented May 30, 2020 •

edited

Loading

ViralBShah commented May 30, 2020

ViralBShah commented May 30, 2020

stevengj commented May 30, 2020

aminya commented May 30, 2020 •

edited

Loading

fgerick commented Jun 13, 2020

fgerick commented Jun 13, 2020

fgerick commented Jul 6, 2020

ViralBShah commented Jul 6, 2020

ViralBShah commented Aug 21, 2020

ViralBShah commented Oct 12, 2020

andreasnoack commented Oct 12, 2020

ViralBShah commented Oct 12, 2020

andreasnoack commented Oct 13, 2020

KristofferC commented Oct 13, 2020 •

edited

Loading

andreasnoack commented Oct 13, 2020

KristofferC commented Oct 13, 2020

ViralBShah commented Oct 13, 2020

KristofferC commented Oct 13, 2020 •

edited

Loading

ViralBShah commented Oct 13, 2020

Set use_blas64 to false to fix MKL.jl interaction numpy problem #35

Set use_blas64 to false to fix MKL.jl interaction numpy problem #35

Conversation

fgerick commented Mar 6, 2020

carstenbauer commented Mar 6, 2020

aminya commented Mar 12, 2020

carstenbauer commented Mar 13, 2020 • edited Loading

fgerick commented Mar 13, 2020

KristofferC Mar 13, 2020

Choose a reason for hiding this comment

ViralBShah May 30, 2020

Choose a reason for hiding this comment

carstenbauer commented Mar 13, 2020 • edited Loading

aminya commented May 26, 2020 • edited Loading

ViralBShah commented May 30, 2020

ViralBShah commented May 30, 2020 • edited Loading

ViralBShah commented May 30, 2020

ViralBShah commented May 30, 2020

stevengj commented May 30, 2020

aminya commented May 30, 2020 • edited Loading

fgerick commented Jun 13, 2020

fgerick commented Jun 13, 2020

fgerick commented Jul 6, 2020

ViralBShah commented Jul 6, 2020

ViralBShah commented Aug 21, 2020

ViralBShah commented Oct 12, 2020

andreasnoack commented Oct 12, 2020

ViralBShah commented Oct 12, 2020

andreasnoack commented Oct 13, 2020

KristofferC commented Oct 13, 2020 • edited Loading

andreasnoack commented Oct 13, 2020

KristofferC commented Oct 13, 2020

ViralBShah commented Oct 13, 2020

KristofferC commented Oct 13, 2020 • edited Loading

ViralBShah commented Oct 13, 2020

carstenbauer commented Mar 13, 2020 •

edited

Loading

carstenbauer commented Mar 13, 2020 •

edited

Loading

aminya commented May 26, 2020 •

edited

Loading

ViralBShah commented May 30, 2020 •

edited

Loading

aminya commented May 30, 2020 •

edited

Loading

KristofferC commented Oct 13, 2020 •

edited

Loading

KristofferC commented Oct 13, 2020 •

edited

Loading