Add script to tune parameters

[thomasfaingnaert]

Add a simple script to perform an exhaustive search on the parameter space, based on #154.

For the RTX 4070, these are the results:

Optimal parameters:
NN: Any[128, 64, 32, 1, 4, GemmKernels.Kernel.matmul_singlestage]
NT: Any[128, 64, 32, 1, 4, GemmKernels.Kernel.matmul_singlestage]
TN: Any[128, 64, 32, 8, 1, GemmKernels.Kernel.matmul_singlestage]
TT: Any[128, 64, 32, 2, 4, GemmKernels.Kernel.matmul_singlestage]

plot.pdf

I've been gradually generalising the kernel by fixing any miscompilations / incorrect results that come up, but we still only explore a relatively small part of the design space (around 3.2 / (100 - 85) = 21.3% of the configurations that ought to work). At some point I'll have to look into the incorrect results and error-throwing configurations:

====================================================================================================
Overall configurations:
====================================================================================================
Total:                                     4000 configurations
----------------------------------------------------------------------------------------------------
Skipped due to invalid GemmKernels config: 3402 (85.0%)
Produced incorrect result:                 112 (2.8%)
Threw an error:                            359 (9.0%)
Successful runs:                           127 (3.2%)

@maleadt Could you have a look at this and run it on more recent hardware as well?

Some notes:

• Using Octavian to generate the groundtruth on the CPU is pretty much a requirement if you don't want to wait around forever. Probably better to generate the groundtruth on the GPU instead.
• I've noticed some configurations result in an illegal memory access. Seems to be a codegen issue, as adding @cuprintf resolves it. I've added an assert for now to mask the issue so we can test that configuration. (Under the assumption that that will not influence the run time too much...)

[maleadt]

RTX6000 Ada:

Optimal parameters:
NN: Any[64, 256, 128, 2, 8, GemmKernels.Kernel.matmul_singlestage]
NT: Any[128, 128, 128, 8, 1, GemmKernels.Kernel.matmul_singlestage]
TN: Any[128, 128, 128, 4, 4, GemmKernels.Kernel.matmul_singlestage]
TT: Any[128, 128, 128, 4, 2, GemmKernels.Kernel.matmul_singlestage]

plot.pdf

H100:

NN: Any[256, 128, 128, 2, 8, GemmKernels.Kernel.matmul_singlestage]
NT: Any[128, 128, 64, 4, 2, GemmKernels.Kernel.matmul_singlestage]
TN: Any[128, 256, 128, 4, 4, GemmKernels.Kernel.matmul_singlestage]
TT: Any[256, 64, 128, 2, 4, GemmKernels.Kernel.matmul_singlestage]

No plot, as it failed during make_plot:

N = 128
N = 256
ERROR: LoadError: ConfigError: Requested too many threads for this kernel: This kernel can be launched using at most 384 threads, while this configuration required 512
Stacktrace:
 [1] matmul(conf::GemmKernels.Config{(M = 256, N = 256, K = 256), (M = 256, N = 128, K = 128), 16, (M = 256, K = 1), (M = 8, K = 1), (K = 128, N = 2), (K = 8, N = 1), (M = 128, N = 1), (M = 4, N = 1), (M = 128, N = 16, K = 16), (M = 16, N = 16, K = 16), GemmKernels.Layout.UnsafeAlignedColMajor{Float16}, GemmKernels.Layout.UnsafeAlignedColMajor{Float16}, GemmKernels.Layout.Zero{Float32}, GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, GemmKernels.Layout.Padded{GemmKernels.Layout.UnsafeAlignedColMajor{Float16}, 8}, GemmKernels.Layout.Padded{GemmKernels.Layout.UnsafeAlignedColMajor{Float16}, 8}, GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, GemmKernels.Operator.WMMAOp{16, 16, 16, Float16, Float32}, true, true}, a::CuArray{Float16, 2, CUDA.Mem.DeviceBuffer}, b::CuArray{Float16, 2, CUDA.Mem.DeviceBuffer}, c::CuArray{Float32, 2, CUDA.Mem.DeviceBuffer}, d::CuArray{Float32, 2, CUDA.Mem.DeviceBuffer}; transform_global_to_shared_a::GemmKernels.Transform.Elementwise{typeof(identity)}, transform_global_to_shared_b::GemmKernels.Transform.Elementwise{typeof(identity)}, transform_global_to_shared_c::GemmKernels.Transform.Elementwise{typeof(identity)}, transform_shared_to_global_d::GemmKernels.Transform.Elementwise{typeof(identity)}, transform_shared_to_regs_a::GemmKernels.Transform.Elementwise{var"#3#5"{Float16}}, transform_shared_to_regs_b::GemmKernels.Transform.Elementwise{typeof(identity)}, transform_shared_to_regs_c::GemmKernels.Transform.Elementwise{var"#4#6"{Float16}}, transform_regs_to_shared_d::GemmKernels.Transform.Elementwise{typeof(identity)}, epilogue::GemmKernels.Epilogue.Default, kernel::typeof(GemmKernels.Kernel.matmul_singlestage))
   @ GemmKernels ~/Julia/pkg/GemmKernels/src/matmul.jl:40
 [2] run_gemm(cf::Configuration, a::CuArray{Float16, 2, CUDA.Mem.DeviceBuffer}, b::CuArray{Float16, 2, CUDA.Mem.DeviceBuffer}, c::CuArray{Float32, 2, CUDA.Mem.DeviceBuffer}, d::CuArray{Float32, 2, CUDA.Mem.DeviceBuffer})
   @ Main ~/Julia/pkg/GemmKernels/configs/configs.jl:78
 [3] macro expansion
   @ ~/Julia/pkg/CUDA/src/profile.jl:308 [inlined]
 [4] make_plot(BLOCK_M::Int64, BLOCK_N::Int64, BLOCK_K::Int64, WARPS_M::Int64, WARPS_N::Int64, kernel::Function, transpose_a::Bool, transpose_b::Bool)
   @ Main ~/Julia/pkg/GemmKernels/tuning/tune-wmma.jl:151

[maleadt]

Ha, cute! Maybe could be more reliable (e.g. when sudo is configured always to prompt again) to fork off a privileged process that waits for the parent though.