5-arg mul! bug fixes

[tkf]

Another followup of #29634:

• Enforce strong-zero behavior for alpha
• Fix 5-arg mul! for Bi/Tri/Sym * Diag

[tkf]

I ran the "full test" on 9da77a4 by commenting out this line

julia/stdlib/LinearAlgebra/test/addmul.jl

Line 121 in 9da77a4

# n_samples = Inf # to try all combinations

All tests passed (after more than 6 hours...). As a side note, I find --compile=min helps a lot as the bottleneck of this test is compilation.

[andreasnoack]

LGTM except for the rending issue

[andreasnoack]

This doesn't render for me. Which character are you using here?

[tkf]

It's *_s (for short-circuiting):

Maybe *ˢ (superscript s) or *′ (prime) is better?

Or we can pick a non-standard character (maybe ⋇?) from:

* / ÷ % & ⋅ ∘ × |\\| ∩ ∧ ⊗ ⊘ ⊙ ⊚ ⊛ ⊠ ⊡ ⊓ ∗ ∙ ∤ ⅋ ≀ ⊼ ⋄ ⋆ ⋇ ⋉ ⋊ ⋋ ⋌ ⋏ ⋒ ⟑ ⦸ ⦼ ⦾ ⦿ ⧶ ⧷ ⨇ ⨰ ⨱ ⨲ ⨳ ⨴ ⨵ ⨶ ⨷ ⨸ ⨻ ⨼ ⨽ ⩀ ⩃ ⩄ ⩋ ⩍ ⩎ ⩑ ⩓ ⩕ ⩘ ⩚ ⩜ ⩞ ⩟ ⩠ ⫛ ⊍ ▷ ⨝ ⟕ ⟖ ⟗

[tkf]

We can also use normal function like mulsc. The code would look like:

mulsc.(A.diag .* in, alpha) .+ mulsc.(out, beta)

[JeffBezanson]

According to wikipedia ⊙ is sometimes used for element-wise product.

[staticfloat]

⊙ is indeed the notation that my profs used for element-wise product.

[tkf]

We need a non-standard binary operator and directly define the broadcasted behavior of it. (The usecase here is to hijack the broadcasted behavior to implement short-circuiting at the level of vector-vector multiplication.)

So using ⊙ would mean to write .⊙ which looks a bit strange as it sounds like doing "doubly" element-wise operation on nested objects.

[timholy]

TensorCore.jl

[JeffBezanson]

We can worry about the choice of operator later, to get these fixes in.

[timholy]

On one of my machines, the LinearAlgebra/addmul tests take 3 times as long as the third-slowest test and twice as long as the second-slowest test. That means if you're running Julia's tests locally you spend most of the time waiting for this one test to finish. Is this something we should consider optimizing, or given buildkite is it not worth it?

Of course, it might also be a good compiler benchmark, maybe worth investigating.

[timholy]

Part of the answer:

julia> m = only(methods(LinearAlgebra._generic_matmatmul!))
_generic_matmatmul!(C::AbstractVecOrMat{R}, tA, tB, A::AbstractVecOrMat{T}, B::AbstractVecOrMat{S}, _add::LinearAlgebra.MulAddMul) where {T, S, R} in LinearAlgebra at /home/tim/src/julia-branch2/usr/share/julia/stdlib/v1.9/LinearAlgebra/src/matmul.jl:850

julia> length(filter(mi -> mi !== nothing, collect(m.specializations)))
5272

😆