Skip to content

Commit ae9e713

Browse files
jeejeeleelk-chen
authored andcommitted
[Model] Adding torch compile annotations to chatglm (vllm-project#15624)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
1 parent 2427791 commit ae9e713

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

vllm/model_executor/models/chatglm.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@
1010
from torch.nn import LayerNorm
1111

1212
from vllm.attention import Attention
13+
from vllm.compilation.decorators import support_torch_compile
1314
from vllm.config import CacheConfig, VllmConfig
1415
from vllm.distributed import get_pp_group, get_tensor_model_parallel_world_size
1516
from vllm.model_executor.layers.activation import SiluAndMul
@@ -293,6 +294,7 @@ def forward(
293294
return hidden_states
294295

295296

297+
@support_torch_compile
296298
class ChatGLMModel(nn.Module):
297299

298300
def __init__(self, *, vllm_config: VllmConfig, prefix: str = ""):

0 commit comments

Comments
 (0)