Skip to content

Conversation

@PatrikPerssonInceptron
Copy link
Contributor

Applying the FuseTransposeMatmul pass on a matmul with a constant weight would fail since the bind_constants defaults to true in the FuseOpsByPattern pass . This would lead the constant to be moved in to the fused function instead of being an input param, however on line 168 in /tvm/relax/transform/fuse_transpose_matmul.py

out_dtype = function.ret_struct_info.dtype
                return self.builder_.call_te(
                    te_transposed_matmul,
                    call.args[1],
                    call.args[0],
                    primfunc_name_hint="NT_matmul",
                )

, called later in the FuseTransposeMatmul pass, both input and weight are expected to be inputs to the fused function leading to a crash.

I fixed it by explicitly setting bind_constants to false and I added a unit test to verify correct behavior when the weight is a constant.

@PatrikPerssonInceptron PatrikPerssonInceptron changed the title [FIX][RELAX] fix fusion of transpose + matmul when the weight is a constant [FIX][RELAX] fix fusion of transpose + matmul when constant weight Mar 18, 2025
Copy link
Member

@Hzfengsy Hzfengsy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Hzfengsy Hzfengsy merged commit a0d0859 into apache:main Mar 19, 2025
15 checks passed
ShiboXing pushed a commit to ShiboXing/tvm that referenced this pull request Aug 10, 2025
…pache#17761)

fix fusion of transpose + matmul when the weight is a constant
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants