openblas 0.3.6 #27847

vbraun · 2019-05-18T10:56:05Z

wget -O upstream/openblas-0.3.6.tar.gz https://github.com/xianyi/OpenBLAS/archive/v0.3.6.tar.gz

CC: @embray

Component: packages: standard

Author: Volker Braun

Branch: 26ce230

Reviewer: Dima Pasechnik, Matthias Koeppe

Issue created by migration from https://trac.sagemath.org/ticket/27847

The text was updated successfully, but these errors were encountered:

vbraun · 2019-05-18T11:07:48Z

Branch: u/vbraun/openblas_0_3_6

vbraun · 2019-05-18T11:08:13Z

Author: Volker Braun

vbraun · 2019-05-18T11:08:13Z

Commit: 26ce230

vbraun · 2019-05-18T11:08:13Z

New commits:

`26ce230`	`Upgrade to openblas 0.3.6`

dimpase · 2019-05-18T11:24:05Z

comment:4

these patches you removed - they are upstream, or they just don't apply?

vbraun · 2019-05-18T12:05:18Z

comment:5

Upstreamed

dimpase · 2019-05-18T12:11:00Z

Reviewer: Dima Pasechnik

dimpase · 2019-05-18T12:11:00Z

comment:6

OK, so Cygwin should not be a problem then.

dimpase · 2019-05-18T17:23:27Z

comment:7

On Gentoo Linux withg gcc 9.1 I am getting here from make ptest

sage: B.<a,b,c> = BooleanPolynomialRing() ## line 4733 ##

out of memory allocating 8388640 bytes

as well as

sage -t src/sage/rings/polynomial/multi_polynomial_sequence.py
    Killed due to abort
**********************************************************************
Tests run before process (pid=19424) failed:
sage: sr = mq.SR(2,1,2,4,gf2=True,polybori=True) ## line 26 ##
sage: sr ## line 27 ##
SR(2,1,2,4)
sage: set_random_seed(1) ## line 33 ##
sage: F,s = sr.polynomial_system() ## line 34 ##
sage: F ## line 35 ##

out of memory allocating 8388640 bytes

and one more polybori-related failure, in total 3

----------------------------------------------------------------------
sage -t src/sage/crypto/mq/sr.py  # Killed due to abort
sage -t src/sage/rings/polynomial/multi_polynomial_sequence.py  # Killed due to abort
sage -t src/sage/rings/polynomial/pbori.pyx  # Killed due to abort
----------------------------------------------------------------------

vbraun · 2019-05-18T22:51:29Z

comment:8

Yes, see #27676 for the brial failures with gcc9

dimpase · 2019-05-19T07:10:16Z

comment:9

anyhow, I also tested with gcc 8.3 on Linux and on OSX with clang, these are all good.
Waiting for Cygwin confirmation I suppose, from Erik.

mkoeppe · 2019-05-20T01:18:50Z

comment:10

This new version also fixes a compile problem unrelated to #27676 that I encountered with 0.3.5.p1, on a CentOS Linux release 7.6.1810 running on Intel(R) Xeon(R) Gold 6152 CPU. 0.3.6 from this ticket compiles and passes its test suite on this system.

dimpase · 2019-05-20T12:15:25Z

comment:11

while we are at it, can we drop write_pc_file.py ? With this version, if you run make install then OpenBLAS does generate openblas.pc by itself.

dimpase · 2019-05-21T19:33:11Z

comment:12

Replying to @dimpase:

while we are at it, can we drop write_pc_file.py ? With this version, if you run make install then OpenBLAS does generate openblas.pc by itself.

In fact, I don't know where write_pc_file.py writes anything. If I remove it I still get local/lib/pkgconfig/openblas.pc written just fine.

dimpase · 2019-05-22T07:04:45Z

New commits:

`3821a45`	`openblas writes openblas.pc itself`

dimpase · 2019-05-22T07:04:45Z

Changed commit from 26ce230 to 3821a45

dimpase · 2019-05-22T07:04:45Z

Changed branch from u/vbraun/openblas_0_3_6 to public/packages/openblas_0_3_6

dimpase · 2019-05-22T15:41:20Z

comment:14

OK, I see there is a small problem - one still needs to make these (c)blas.pc and lapack.pc from openblas.pc. Not sure whether Name: line needs to be adjusted, or merely a copy/link will do.

dimpase · 2019-05-22T17:21:23Z

comment:15

Let us postpone this then, so back to postive review from me.

dimpase · 2019-05-22T17:21:23Z

Changed commit from 3821a45 to 26ce230

dimpase · 2019-05-22T17:21:23Z

Changed branch from public/packages/openblas_0_3_6 to u/vbraun/openblas_0_3_6

embray · 2019-05-23T11:19:12Z

comment:16

I'll give it a look. I want to review this very carefully, as I've been practically traumatized by OpenBLAS in the last year. I know this release should include many of my existing fixes but I want to look over the other diffs between versions to make sure no other obvious regressions crept in.

vbraun · 2019-05-25T09:46:12Z

comment:17

Can you do this sometime soon, I can't build Sage on my own PC...

dimpase · 2019-05-25T10:39:36Z

comment:18

How crucial is Openblas for Cygwin? Can't one use SAGE_ATLAS_LIB and whatever is supplied by Cygwin? (but we should work on #27870 anyway, to resolve these things in a meaningful way)

vbraun · 2019-05-30T22:37:54Z

comment:19

Any updates?

embray · 2019-05-31T13:54:09Z

comment:20

Replying to @vbraun:

What exactly do you mean by this?

embray · 2019-05-31T14:11:39Z

comment:21

I will test this as soon as I can get a working Sage, which now seems to be a problem due to some other reason (possibly related to #27901 but unclear).

vbraun · 2019-06-02T13:31:32Z

comment:23

That I'm on gcc9 and while I can technically build Sage, the testsuite fails with heaps of errors.

I'm in favor of releasing this if Eric can't try out cygwin any time soon. Its a must fix for currently supported platforms.

mkoeppe · 2019-06-02T22:33:44Z

comment:24

Replying to @vbraun:

I'm in favor of releasing this if Eric can't try out cygwin any time soon. Its a must fix for currently supported platforms.

+1

mkoeppe · 2019-06-02T22:39:35Z

comment:25

On a different Linux system (Ubuntu 18.04.2 LTS, running on some virtualized AMD system), I have the following problem with both 0.3.5.p1 and 0.3.6:

[openblas-0.3.6.p0] perl ./gensymbol linktest  x86_64 _ 0 0 0 0 0 0 "" "" 1 > linktest.c
[openblas-0.3.6.p0] gcc -O2 -DMAX_STACK_ALLOC=2048 -Wall -m64 -DF_INTERFACE_GFORT -fPIC -DSMP_SERVER -DNO_WARMUP -DMAX_CPU_NUMBER=24 -DMAX_PARALLEL_NUMBER=1 -DUSE_TLS -DVERSION=\"0.3.6\" -DASMNAME= -DASMFNAME=_ -DNAME=_ -DCNAME= -DCHAR_NAME=\"_\" -DCHAR_CNAME=\"\" -DNO_AFFINITY -I.. -L/homes/home05/m/mkoeppe/sage/local/lib -Wl,-rpath,/homes/home05/m/mkoeppe/sage/local/lib  -shared -o ../libopenblas_atomp-r0.3.6.so \
[openblas-0.3.6.p0] -Wl,--whole-archive ../libopenblas_atomp-r0.3.6.a -Wl,--no-whole-archive \
[openblas-0.3.6.p0] -Wl,-soname,libopenblas.so.0 -lm -lpthread -lgfortran -lm -lpthread -lgfortran
[openblas-0.3.6.p0] /usr/bin/ld: ../libopenblas_atomp-r0.3.6.a(ssymv_U.o): relocation R_X86_64_32S against undefined symbol `PREFETCHSIZE' can not be used when making a shared object; recompile with -fPIC
[openblas-0.3.6.p0] /usr/bin/ld: ../libopenblas_atomp-r0.3.6.a(ssymv_L.o): relocation R_X86_64_32S against undefined symbol `PREFETCHSIZE' can not be used when making a shared object; recompile with -fPIC
[openblas-0.3.6.p0] /usr/bin/ld: ../libopenblas_atomp-r0.3.6.a(dsymv_U.o): relocation R_X86_64_32S against undefined symbol `PREFETCHSIZE' can n
ot be used when making a shared object; recompile with -fPIC
[openblas-0.3.6.p0] /usr/bin/ld: ../libopenblas_atomp-r0.3.6.a(dsymv_L.o): relocation R_X86_64_32S against undefined symbol `PREFETCHSIZE' can not be used when making a shared object; recompile with -fPIC
[openblas-0.3.6.p0] /usr/bin/ld: ../libopenblas_atomp-r0.3.6.a(zsymv_U.o): relocation R_X86_64_32S against undefined symbol `PREFETCHSIZE' can not be used when making a shared object; recompile with -fPIC
[openblas-0.3.6.p0] /usr/bin/ld: ../libopenblas_atomp-r0.3.6.a(zsymv_L.o): relocation R_X86_64_32S against undefined symbol `PREFETCHSIZE' can not be used when making a shared object; recompile with -fPIC
[openblas-0.3.6.p0] /usr/bin/ld: ../libopenblas_atomp-r0.3.6.a(zhemv_U.o): relocation R_X86_64_32S against undefined symbol `PREFETCHSIZE' can not be used when making a shared object; recompile with -fPIC
[openblas-0.3.6.p0] /usr/bin/ld: ../libopenblas_atomp-r0.3.6.a(zhemv_L.o): relocation R_X86_64_32S against undefined symbol `PREFETCHSIZE' can not be used when making a shared object; recompile with -fPIC
[openblas-0.3.6.p0] /usr/bin/ld: final link failed: Nonrepresentable section on output
[openblas-0.3.6.p0] collect2: error: ld returned 1 exit status
[openblas-0.3.6.p0] Makefile:153: recipe for target '../libopenblas_atomp-r0.3.6.so' failed
[openblas-0.3.6.p0] make[3]: *** [../libopenblas_atomp-r0.3.6.so] Error 1
[openblas-0.3.6.p0] make[3]: Leaving directory '/homes/home05/m/mkoeppe/sage/local/var/tmp/sage/build/openblas-0.3.6.p0/src/exports'
[openblas-0.3.6.p0] Makefile:101: recipe for target 'shared' failed
[openblas-0.3.6.p0] make[2]: *** [shared] Error 2
[openblas-0.3.6.p0] make[2]: Leaving directory '/homes/home05/m/mkoeppe/sage/local/var/tmp/sage/build/openblas-0.3.6.p0/src'
[openblas-0.3.6.p0] ********************************************************************************
[openblas-0.3.6.p0] Error building openblas-0.3.6.p0

Nevertheless, I'm setting the ticket to positive review.

mkoeppe · 2019-06-02T22:39:48Z

Changed reviewer from Dima Pasechnik to Dima Pasechnik, Matthias Koeppe

embray · 2019-06-03T06:15:18Z

comment:27

What exactly is the issue with gcc9 that this is resolving?

dimpase · 2019-06-03T07:57:33Z

comment:28

Replying to @embray:

What exactly is the issue with gcc9 that this is resolving?

see #27676 - basically, openblas 0.3.5 is too old for gcc9, which miscompiles it.

vbraun · 2019-06-06T22:26:50Z

Changed branch from u/vbraun/openblas_0_3_6 to 26ce230

embray · 2019-06-07T17:23:10Z

comment:30

Replying to @dimpase:

Replying to @embray:

What exactly is the issue with gcc9 that this is resolving?

see #27676 - basically, openblas 0.3.5 is too old for gcc9, which miscompiles it.

I think not being able to use the most bleeding-edge version of a compiler is not such a blocker if it means forcing dependency updates that could lead to instabilities in other ways (in general being always at the bleeding edge lends some instability; as does falling too far behind).

In this case, at least, I haven't been able to identify any major problems with this openblas release, and it fixes all the problems i'm aware of from 0.3.5, which was a mess.

embray · 2019-06-07T17:23:10Z

Changed commit from 26ce230 to none

jhpalmieri · 2019-06-09T19:39:16Z

comment:31

This seems to cause Sage to crash with some machines using OS X. From the end of Sage_crash_report.txt:

ImportError: dlopen(/Users/palmieri/Desktop/Sage_stuff/git/sage/local/lib/python2.7/site-packages/sage/matrix/matrix_rational_dense.so, 2): Library not loaded: /Users/palmieri/Desktop/Sage_stuff/git/sage/local/lib/libopenblas_haswellp-r0.3.5.dylib
  Referenced from: /Users/palmieri/Desktop/Sage_stuff/git/sage/local/lib/python2.7/site-packages/sage/matrix/matrix_rational_dense.so
  Reason: image not found

jhpalmieri · 2019-06-09T19:45:57Z

comment:32

Followup at #27961.

dimpase · 2019-06-09T20:31:08Z

comment:33

why does the error message mention 0.3.5 as the version number of the library which cannot be loaded?

and not 0.3.6?

jhpalmieri · 2019-06-09T20:58:52Z

comment:34

I don't know, but that's the error I get with this ticket plus 8.8.beta7, incremental build.

jhpalmieri · 2019-06-10T22:37:26Z

comment:35

A second issue, only present on one OS X machine so far:

sage: M = ModularSymbols(GammaH(13,[3]), weight=4)
sage: M.cuspidal_subspace()
Fatal: Memory exhausted.

This occurs with 8.8.beta7 + this ticket. For now, let's track both issues at #27961. If they need to be separated into two tickets, we can do this later.

vbraun added this to the sage-8.8 milestone May 18, 2019

vbraun added the p: major / 3 label May 18, 2019

This comment has been minimized.

Sign in to view

vbraun added c: packages: standard labels May 18, 2019

vbraun added the s: needs review label May 18, 2019

This comment has been minimized.

Sign in to view

vbraun added p: blocker / 1 and removed p: major / 3 labels Jun 2, 2019

mkoeppe added s: positive review and removed s: needs review labels Jun 2, 2019

vbraun removed the s: positive review label Jun 6, 2019

vbraun closed this as completed in f276d4b Jun 6, 2019

jhpalmieri mentioned this issue Jun 18, 2019

openblas 0.3.6 vs. OS X on some processors #27961

Closed

mkoeppe mentioned this issue Apr 5, 2020

Update OpenBLAS to 0.3.9 - fixes build error of iml on fedora-32-standard #29378

Closed

openblas 0.3.6 #27847

openblas 0.3.6 #27847

Comments

vbraun commented May 18, 2019

This comment has been minimized.

vbraun commented May 18, 2019

vbraun commented May 18, 2019

vbraun commented May 18, 2019

vbraun commented May 18, 2019

dimpase commented May 18, 2019

vbraun commented May 18, 2019

dimpase commented May 18, 2019

dimpase commented May 18, 2019

dimpase commented May 18, 2019

vbraun commented May 18, 2019

dimpase commented May 19, 2019

This comment has been minimized.

mkoeppe commented May 20, 2019

dimpase commented May 20, 2019

dimpase commented May 21, 2019

dimpase commented May 22, 2019

dimpase commented May 22, 2019

dimpase commented May 22, 2019

dimpase commented May 22, 2019

dimpase commented May 22, 2019

dimpase commented May 22, 2019

dimpase commented May 22, 2019

embray commented May 23, 2019

vbraun commented May 25, 2019

dimpase commented May 25, 2019

vbraun commented May 30, 2019

embray commented May 31, 2019

embray commented May 31, 2019

vbraun commented Jun 2, 2019

mkoeppe commented Jun 2, 2019

mkoeppe commented Jun 2, 2019

mkoeppe commented Jun 2, 2019

embray commented Jun 3, 2019

dimpase commented Jun 3, 2019

vbraun commented Jun 6, 2019

embray commented Jun 7, 2019

embray commented Jun 7, 2019

jhpalmieri commented Jun 9, 2019

jhpalmieri commented Jun 9, 2019

dimpase commented Jun 9, 2019

jhpalmieri commented Jun 9, 2019

jhpalmieri commented Jun 10, 2019