ktransformers_ext/bench

[wanwujie1]

When running all scripts under the bench directory and finding that, when executed on the CPU, PyTorch's native fp16 and int8 are faster than ggml_type's fp16, q8_0, q4_k_m, and q2_k, and the performance differences between ggml_type's fp16, q8_0, q4_k_m, and q2_k are also minimal, is this normal?

[chenht2022]

Have you ensured that the number of threads in cpuinfer does not exceed the number of physical CPU cores？You can set it in bench_xxx.py