more source-build problems, with openblas now on CI

[vtjnash]

In file included from ../kernel/x86_64/sbgemm_kernel_16x16_spr.c:31:
../kernel/x86_64/sbgemm_kernel_16x16_spr_tmpl.c: In function 'sbgemm_kernel_spr_alpha_one':
../kernel/x86_64/sbgemm_kernel_16x16_spr_tmpl.c:67:2: warning: implicit declaration of function '_tile_loadconfig' [-Wimplicit-function-declaration]
   67 |  _tile_loadconfig(&cfg);
      |  ^~~~~~~~~~~~~~~~
../kernel/x86_64/sbgemm_kernel_16x16_spr_tmpl.c:201:4: note: in expansion of macro 'TCONF'
  201 |    TCONF(cfg, 16, 16, 32);
      |    ^~~~~
../kernel/x86_64/sbgemm_kernel_16x16_spr_tmpl.c:137:22: warning: implicit declaration of function '_tile_loadd' [-Wimplicit-function-declaration]
  137 | #define LOAD_C(M, N) _tile_loadd(T_C##M##N, ptr_c##M##N, ldc * 4)
      |                      ^~~~~~~~~~~
../kernel/x86_64/sbgemm_kernel_16x16_spr_tmpl.c:212:5: note: in expansion of macro 'LOAD_C'
  212 |     LOAD_C(0, 0); LOAD_C(0, 1);
      |     ^~~~~~

https://buildkite.com/julialang/julia-master-scheduled/builds/643#018d6e03-2129-477c-9c41-22a940ea5492

[inkydragon]

These two functions are Intel® AMX (Advanced Matrix Extensions) Intrinsics Functions

• _tile_loadconfig
• _tile_loadd

Looks like a legacy issue, most recent modifications were 3 years ago:
https://github.com/OpenMathLib/OpenBLAS/blame/e5d2725e5a29981cab4068233ab97b6daa25b540/kernel/x86_64/sbgemm_oncopy_16_spr.c#L63

Maybe it's not a regression?

[giordano]

Those reported above are warnings, not errors?

[inkydragon]

errors in build openblas

../kernel/x86_64/sbgemm_kernel_16x16_spr_tmpl.c:116:16: error: incompatible types when initializing type '__m256i' using type 'int'
  116 |  __m256i ymm = _mm256_loadu_epi16(ptr_b##N); \
      |                ^~~~~~~~~~~~~~~~~~
../kernel/x86_64/sbgemm_kernel_16x16_spr_tmpl.c:359:21: note: in expansion of macro 'LOAD_B_TAIL'
  359 |       LOAD_B(x, 0); LOAD_B_TAIL(x, 1);
      |                     ^~~~~~~~~~~
../kernel/x86_64/sbgemm_kernel_16x16_spr_tmpl.c:116:16: error: incompatible types when initializing type '__m256i' using type 'int'
  116 |  __m256i ymm = _mm256_loadu_epi16(ptr_b##N); \
      |                ^~~~~~~~~~~~~~~~~~
../kernel/x86_64/sbgemm_kernel_16x16_spr_tmpl.c:435:7: note: in expansion of macro 'LOAD_B_TAIL'

[inkydragon]

_mm256_loadu_epi16
Need CPU flag: AVX512BW + AVX512VL

Maybe our CPU doesn't support this?

[giordano]

We do a dynamic arch build

julia/deps/openblas.mk

Line 32 in 28db6c7

OPENBLAS_BUILD_OPTS += DYNAMIC_ARCH=1

the host CPU shouldn't matter. But it's quite possible a newer compiler is needed, in Yggdrasil we're now using gcc 11 for openblas, I don't know what's used in CI here for the source build.

[inkydragon]

Compare with old build (19th Dec 2023) which openblas build successfully.
(Search "OpenBLAS build complete" in full log)

A new flag has been added: BUILD_BFLOAT16=1.

[giordano]

But it's quite possible a newer compiler is needed, in Yggdrasil we're now using gcc 11 for openblas, I don't know what's used in CI here for the source build.

It's hard to read to raw log, I see a line containing

gcc version 10.2.1 20210110 (Debian 10.2.1-6)

if that's the compiler used for the build, according to JuliaPackaging/Yggdrasil#7202 (comment) gcc 10 should be sufficient to build the bfloat16 kernels.

[imciner2]

Well, based on the comment here https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95483#c4, many of these intrinsics were missing and then added in GCC 11, e.g. the intrinisic _mm256_loadu_epi16 is referenced in that bug report as being added for GCC 11.

I'm not entirely sure what would make it seem like it worked on Yggdrasil when I built it before and not work here, but I may have just skipped to GCC 11 and not fully dived into anything for GCC 10 when I was doing the builds.

[imciner2]

(and as it stands, building from source on Windows also needs our own patched GCC for BFloat16 to get the optimized kernels, unless the user has GCC 12.3 or 13+ and is using that), so for now, BFloat16 support is probably not something to expose in pure from-source builds due to its somewhat complex compiler support history.