[BUG] 本地纯 CPU 推理失败 (Unable to run inference without GPU)

[jerrylususu]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

本地环境：Windows 11 x64, 32G 内存, 无独立显卡
根据 README 指南，调整脚本使用 CPU 运行，提示找不到 nvcuda.dll，随后执行中断推出。

修改后代码

tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True)
model = AutoModel.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True).bfloat16()
model = model.eval()

日志

(base) F:\Projects\chatglm_6b\ChatGLM-6B>python web_demo.py
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
Explicitly passing a `revision` is encouraged when loading a configuration with custom code to ensure no malicious code has been contributed in a newer revision.
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
Traceback (most recent call last):
  File "F:\Projects\chatglm_6b\ChatGLM-6B\web_demo.py", line 5, in <module>
    model = AutoModel.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True).bfloat16()
  File "C:\Users\USER_RETRACTED\anaconda3\lib\site-packages\transformers\models\auto\auto_factory.py", line 455, in from_pretrained
    model_class = get_class_from_dynamic_module(
  File "C:\Users\USER_RETRACTED\anaconda3\lib\site-packages\transformers\dynamic_module_utils.py", line 363, in get_class_from_dynamic_module
    final_module = get_cached_module_file(
  File "C:\Users\USER_RETRACTED\anaconda3\lib\site-packages\transformers\dynamic_module_utils.py", line 274, in get_cached_module_file
    get_cached_module_file(
  File "C:\Users\USER_RETRACTED\anaconda3\lib\site-packages\transformers\dynamic_module_utils.py", line 237, in get_cached_module_file
    modules_needed = check_imports(resolved_module_file)
  File "C:\Users\USER_RETRACTED\anaconda3\lib\site-packages\transformers\dynamic_module_utils.py", line 129, in check_imports
    importlib.import_module(imp)
  File "C:\Users\USER_RETRACTED\anaconda3\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "C:\Users\USER_RETRACTED\anaconda3\lib\site-packages\cpm_kernels\__init__.py", line 2, in <module>
    from .kernels import *
  File "C:\Users\USER_RETRACTED\anaconda3\lib\site-packages\cpm_kernels\kernels\__init__.py", line 1, in <module>
    from .embedding import embedding_forward, embedding_backward_stage1, embedding_backward_stage2, embedding_step
  File "C:\Users\USER_RETRACTED\anaconda3\lib\site-packages\cpm_kernels\kernels\embedding.py", line 1, in <module>
    from .base import Kernel, DevicePointer, CUDAStream, round_up
  File "C:\Users\USER_RETRACTED\anaconda3\lib\site-packages\cpm_kernels\kernels\base.py", line 4, in <module>
    from ..library import cuda, cudart
  File "C:\Users\USER_RETRACTED\anaconda3\lib\site-packages\cpm_kernels\library\cuda.py", line 7, in <module>
    cuda = Lib.from_lib("cuda", ctypes.WinDLL("nvcuda.dll"))
  File "C:\Users\USER_RETRACTED\anaconda3\lib\ctypes\__init__.py", line 382, in __init__
    self._handle = _dlopen(self._name, mode)
FileNotFoundError: Could not find module 'nvcuda.dll' (or one of its dependencies). Try using the full path with constructor syntax.

Expected Behavior

纯 CPU 推理依然可以正常执行，尽管很慢且占用大量内存。

Steps To Reproduce

1. git clone 项目，安装依赖
2. 打开 web_demo.py，按 README 修改

model = AutoModel.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True).bfloat16()

3. 运行 python web_demo.py，执行报错

Environment

- OS: Windows 11 x64 22H2
- Python: Python 3.9.13 (main, Aug 25 2022, 23:51:50) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
- Transformers: '4.26.1'
- PyTorch: '1.13.1+cpu'
- CUDA Support (`python -c "import torch; print(torch.cuda.is_available())"`) : False

Anything else?

No response

[duzx16]

Duplicate of #95