Added instr.sched options to tune_gemm.py

[ravil-mobile]

No description provided.

[xiaohuguo2023]

Can you add some description in the README file ?

Hi @xiaohuguo2023, I added the following line under GEMM Tuning Script v3.4:

• Appended the parameters tuning space with instruction scheduling variants for the main gemm-loop (k-loop).

[zhanglx13]

Do we know which gemm sizes can potentially benefit from setting instruction_sched_variant to iglp0 or ckv3?

[ravil-mobile]

Do we know which gemm sizes can potentially benefit from setting instruction_sched_variant to iglp0 or ckv3?

No, at this moment it is difficult to say

[sjw36]

Looks good but need to wait for features in TIP. Blocking for now.

[ravil-mobile]

Looks good but need to wait for features in TIP. Blocking for now.

Sorry. I didn't get what you mean

[zhanglx13]

Do we know which gemm sizes can potentially benefit from setting instruction_sched_variant to iglp0 or ckv3?

No, at this moment it is difficult to say

If this is the case, I'd say we don't want to add more tuning parameters.
If we know it works for "some" kernels for sure, we should use them by some simple heuristics.
If we don't know when they help, we should figure it out first.

[zhanglx13]

See my comments

[ravil-mobile]

Do we know which gemm sizes can potentially benefit from setting instruction_sched_variant to iglp0 or ckv3?

No, at this moment it is difficult to say

If this is the case, I'd say we don't want to add more tuning parameters. If we know it works for "some" kernels for sure, we should use them by some simple heuristics. If we don't know when they help, we should figure it out first.

Ok. Let me make sched_variants = ["\"default\""]. This means that only no scheduling variant is going to work. Tell me whether you are ok with it. At least we should push the adapted infrastructure to main_perf

[zhanglx13]

Do you have an example output after adding '\n'?

[ravil-mobile]

Yes, sure.

> ./tune_gemm.py --gemm_size_file ~/tuning/input.yaml --gpu_ids 3,4,5 --jobs 32 --o ~/tuning/output.yaml
Tuning 1 gemm sizes starts at: 2024-10-29 14:26:32.618604
SIZE: 4864 8192 4160 TN nConfigs: 720 
TFLOPS: 516.47; time(us): 641.89 
best_config: BM128_BN128_BK64_GM8_SK1_nW4_nS2_EU0_kP2_mfma16_schedDEFAULT 
>>> Elapsed time: 0:04:11.238153 = 0:00:20.441773 (compile) + 0:03:49.947198 (profile) + 0:00:00.681876 (post processing)
Tuning ends at: 2024-10-29 14:30:44.031012
Total tuning time (h:m:s): 0:04:11.412408

[zhanglx13]

Since you are here, can you replace the nested for loops with itertools.product?

[ravil-mobile]

Yes, sure. Done!

[zhanglx13]

Ok. Let me make sched_variants = [""default""]. This means that only no scheduling variant is going to work. Tell me whether you are ok with it. At least we should push the adapted infrastructure to main_perf

Yes, I'm ok with it