Upgrade to OpenBLAS 0.3.13

[omus]

Fixes #39201. Requires JuliaPackaging/Yggdrasil#2385.

Possibly this PR also should include the neoverse-generic-kernels.patch used with the BinaryBuilder version. (the additional patches are for GCC and don't apply here)

[omus]

I'll mention that the patches here and in the BB version have minor differences but should be functionally identical

[dkarrasch]

Should close #36976 and #38865.

[omus]

I've successfully ran the Julia test suite locally with OpenBLAS 0.3.13 using dependencies built from source. I just need to get the BinaryBuilder version working.

[omus]

Testers seem to have this error (I just spot checked a couple):

precompile                         (1) |        started at 2021-01-21T15:46:04.840
julia: /workspace/srcdir/llvm-project/llvm/lib/IR/Value.cpp:460: void llvm::Value::doRAUW(llvm::Value*, llvm::Value::ReplaceMetadataUses): Assertion `New->getType() == getType() && "replaceAllUses of value with new value of different type!"' failed.
signal (6): Aborted
in expression starting at /buildworker/worker/tester_linux64/build/share/julia/test/precompile.jl:850

[omus]

I had some strange building issues locally that have since disappeared. I'll try a CI re-run as I'm unable to reproduce locally

[omus]

I've been waiting on JuliaPackaging/Yggdrasil#2431 before investigating these CI issues further.

[pablosanjose]

There is a fix to a LAPACK-netlib bug for OpenBLAS here. Is there any chance it could be included here as a patch? Or do we need to wait for OpenBLAS 0.3.14 to be released?

[omus]

There is a fix to a LAPACK-netlib bug for OpenBLAS here. Is there any chance it could be included here as a patch? Or do we need to wait for OpenBLAS 0.3.14 to be released?

It's fine to include the patch here.

[omus]

CI error seems to be the same as: #27055. Unfortunately I can't trigger the issue locally by changing the optimization level

[pablosanjose]

It's fine to include the patch here.

Should I do a PR onto this PR? The patch is quite simple, just this

EDIT: updated the gist above with a typo correction, as noted here, after confirming with an OpenBLAS maintainer.

[omus]

Just providing the patch is good enough. I can incorporate the patch here but I may start with it disabled until I sort out the CI issue.

[pablosanjose]

Many thanks! The issue addressed by the patch is making my life complicated, given that it produces convergence errors that are difficult to predict and are platform-dependent. I really appreciate this!

[omus]

Disabling the test that seems to cause the failure on CI just to see if that's the only issue

[omus]

The patch is quite simple, just this

The patch you provided appears to be corrupt. I've regenerated the patch. If you could make sure it still is correct that would be great

[pablosanjose]

Uh, apologies, I actually tried to adapt the full patch at Reference-LAPACK/lapack#477 into an OpenBLAS patch (the relevant files are identical), but I guess that's not how this is done (I've never done this before, sorry). Unfortunately, the OpenBLAS PR OpenMathLib/OpenBLAS#3091 you took the patch from was merged before the full fix was made upstream in Reference-LAPACK/lapack#477, so that patch you posted is actually incomplete... Is there a way to adapt Reference-LAPACK/lapack#477 into an OpenBLAS patch?

EDIT: If that is not actually possible, I would just use the patch you've generated, as I think it is enough to solve the core of the convergence problem

[pablosanjose]

My confusion above was due to the fact that the original fix in Reference-LAPACK/lapack#477 has been ported to OpenBLAS in two parts, OpenMathLib/OpenBLAS#3087 and OpenMathLib/OpenBLAS#3091. When you corrected my corrupted patch I believe you made a patch with only the latter PR, but not the former. I'm pretty sure they are both necessary. The question is how to do a single patch that applies two PRs. I'm afraid I don't know how to do that, but perhaps you do?

[pablosanjose]

Is it clear what is the cause of the CI failure here?

[The failure ([1] pipeline_error @ ./process.jl:525 [inlined]) reminds me of this issue, if that's any help]

[omus]

Patch should now include OpenMathLib/OpenBLAS#3087 and OpenMathLib/OpenBLAS#3091.

[omus]

After applying the revised patch and rebasing I am unable to reproduce the doctest failure or the issue with opaque closures locally.

[omus]

Looks like the opaque closure failures were fixed somewhere else. I've managed to reproduce the docstrings failures in a Linux environment but oddly they don't fail on macOS

[omus]

Probably should have removed the WIP from the commit message. What's our feeling on changing it?

[ViralBShah]

I was mostly looking at the diff and not the commit message. I'm not sure if things can be(?) changed on master - but there are a few other wip commit messages in the log. I'll leave it to the git specialists to decide.

I am removing the WIP from the PR title though.

[KristofferC]

Based on #40279 I will take off the backport label here. It can be put back until it is resolved.

[ViralBShah]

Fixed through new binaries in JuliaPackaging/Yggdrasil#2769. But also needs #40343 to go with it.

[KristofferC]

#40361 (removing backport label again)

[ViralBShah]

Wow I can't imagine what is causing that.

[KristofferC]

Me neither, it was discussed a bit in the #binarybuilder channel in Slack but nothing concrete came out of it.

[omus]

#40361 has been addressed (details: #40361 (comment)). Should the backport label be added again?

[ViralBShah]

Given that 1.7 is not too far away (timed releases going forward), and we've found various openblas related issues, it might be best not to backport.

[vchuravy]

We should backport the fix to the LTS #40492 (comment)