Skip to content

Commit

Permalink
fix incorrect output of Qwen2-7B-Instruct-GPTQ-Int4 and Qwen2-7B-Inst… (
Browse files Browse the repository at this point in the history
#2717)

fix incorrect output of Qwen2-7B-Instruct-GPTQ-Int4 and Qwen2-7B-Instruct-AWQ
ipex kernel provide func like add_bias, so no need add it outside

Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
  • Loading branch information
sywangyi authored Nov 4, 2024
1 parent 5eedb2e commit b1f9044
Show file tree
Hide file tree
Showing 2 changed files with 0 additions and 2 deletions.
1 change: 0 additions & 1 deletion server/text_generation_server/layers/awq/quantize/ipex.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,5 +44,4 @@ def __init__(
def forward(self, x):
out_shape = x.shape[:-1] + (self.out_features,)
out = self.woq_linear(x.reshape(-1, x.shape[-1]))
out = out + self.bias if self.bias is not None else out
return out.reshape(out_shape)
1 change: 0 additions & 1 deletion server/text_generation_server/layers/gptq/ipex.py
Original file line number Diff line number Diff line change
Expand Up @@ -122,5 +122,4 @@ def pack(self, linear, scales, zeros, g_idx=None):
def forward(self, x):
out_shape = x.shape[:-1] + (self.outfeatures,)
out = self.woq_linear(x.reshape(-1, x.shape[-1]))
out = out + self.bias if self.bias is not None else out
return out.reshape(out_shape)

0 comments on commit b1f9044

Please sign in to comment.