昇腾NPU推理glm4-9b-chat出现'NoneType' object has no attribute 'do_sample'

[AlexYoung757]

Reminder

• I have read the README and searched the existing issues.

System Info

• Platform: Linux-4.19.36-vhulk1907.1.0.h619.eulerosv2r8.aarch64-aarch64-with-glibc2.28
• Python version: 3.10.14
• PyTorch version: 2.2.0 (NPU)
• Transformers version: 4.41.2
• Datasets version: 2.20.0
• Accelerate version: 0.32.1
• PEFT version: 0.11.1
• TRL version: 0.9.6
• NPU type: Ascend910PremiumA
• CANN version: 8.0.RC1.alpha003

Reproduction

ASCEND_RT_VISIBLE_DEVICES=4 llamafactory-cli chat glm4_9b_chat.yaml

Expected behavior

Traceback (most recent call last):
  File "/root/miniconda3/envs/llm/bin/llamafactory-cli", line 8, in <module>
    sys.exit(main())
  File "/root/project/LLaMA-Factory/src/llamafactory/cli.py", line 81, in main
    run_chat()
  File "/root/project/LLaMA-Factory/src/llamafactory/chat/chat_model.py", line 125, in run_chat
    chat_model = ChatModel()
  File "/root/project/LLaMA-Factory/src/llamafactory/chat/chat_model.py", line 44, in __init__
    self.engine: "BaseEngine" = HuggingfaceEngine(model_args, data_args, finetuning_args, generating_args)
  File "/root/project/LLaMA-Factory/src/llamafactory/chat/hf_engine.py", line 58, in __init__
    self.model = load_model(
  File "/root/project/LLaMA-Factory/src/llamafactory/model/loader.py", line 159, in load_model
    patch_model(model, tokenizer, model_args, is_trainable, add_valuehead)
  File "/root/project/LLaMA-Factory/src/llamafactory/model/patcher.py", line 120, in patch_model
    if not gen_config.do_sample and (
AttributeError: 'NoneType' object has no attribute 'do_sample'

Others

No response

[hiyouga]

不要修改模型文件

[AlexYoung757]

模型文件按照 #4388 修改的。