[BUG] Bitblas Kernel not compatbile with bitblas > 0.0.1-dev13

[Qubitium]

@LeiWang1999 Happy Chinese New Years! When you have time, can you point to us or help us find the cause of bitblas v0.1.0 issue. Thanks!

For GPTQModel v1.8.0-dev release I am trying to migrate gptqmodel's locked support for bitblas from working 0.0.1-dev13 to 0.1.0 but running into multiple issues.

It appears since bitblas version >= 0.0.1-dev14 bitblas kernel is no longer compatible and throws shape mistmatch errors.

I tried to migrate the code to bitblas kernel and pack code to bitblas 0.1.0 in PR and failed #1184 (now partially reverted back to 0.0.1-dev13 state) with Segmentation fault core dumped at the self.bitblas_matmul.lib.call(). 0.1.0 no longer has call_lib but we see there is a lib.call but trying to use it just seg faults.

Please use the following code to replicate:

1. bitblas 0.0.1-dev13 <-- working
2. bitblas 0.0.1-dev14 <-- shape exceptions
3. bitblas 0.1.0 (requires checkout of Update/Refractor Bitblas/Marlin/Cuda #1184) but segfaults: https://github.com/ModelCloud/GPTQModel/pull/1184/files#diff-47042b63158eec98a67b6a85c5a28e5c0623b67fce9823ab425156f4d71e738a

from gptqmodel import GPTQModel, BACKEND

model = GPTQModel.load(
    "ModelCloud/Qwen2.5-0.5B-Instruct-ci-test-bitblas",
    backend=BACKEND.BITBLAS, # change to BACKEND.TORCH or MARLIN for normal generate
)

print(model.tokenizer.decode(model.generate("What is the capital of United States?")[0]))

p.s. GPTQModel was recently merged into HF Transformers main so the next Transformers non-patch release should have wide-support of optional bitblas support for all gptq models with backend toggle!

[LeiWang1999]

sure, I'll take a look