Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MING-7B无法运行,怀疑是 tokenizer.json 问题,有没有遇到类似问题的 #27

Closed
coldy1992 opened this issue May 16, 2024 · 3 comments

Comments

@coldy1992
Copy link

coldy1992 commented May 16, 2024

使用以下命令运行模型:
CUDA_VISIBLE_DEVICES=0 python -m fastchat.serve.cli --model-path /home/xx/ydd/Medical_LLM/MING-7B --max-new-token 512 --beam-size 3 --temperature 1.2
报错如下:
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]Traceback (most recent call last):
File "/data/miniconda3/envs/ming-main/lib/python3.9/site-packages/transformers/modeling_utils.py", line 535, in load_state_dict
return torch.load(
File "/data/miniconda3/envs/ming-main/lib/python3.9/site-packages/torch/serialization.py", line 1040, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "/data/miniconda3/envs/ming-main/lib/python3.9/site-packages/torch/serialization.py", line 1258, in _legacy_load
magic_number = pickle_module.load(f, **pickle_load_args)
ValueError: could not convert string to float: 'ound. Redirecting to https://cdn-lfs.huggingface.co/repos/9a/8b/9a8b0806855567bc42061011ed21fcfa01efc1f28452196caf2714c0469ddbc1/3a76dc8f5351fef1c98664c09992f4d4236c649411455c1a9046cf1aff47116c?respon'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/data/miniconda3/envs/ming-main/lib/python3.9/site-packages/transformers/modeling_utils.py", line 544, in load_state_dict
if f.read(7) == "version":
File "/data/miniconda3/envs/ming-main/lib/python3.9/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 1240: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/data/miniconda3/envs/ming-main/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/data/miniconda3/envs/ming-main/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/data/apps/ming-main/MING-main/fastchat/serve/cli.py", line 133, in
main(args)
File "/data/apps/ming-main/MING-main/fastchat/serve/cli.py", line 108, in main
chat_loop(args.model_path, args.device, args.num_gpus, args.max_gpu_memory,
File "/data/apps/ming-main/MING-main/fastchat/serve/inference.py", line 153, in chat_loop
model, tokenizer = load_model(model_path, device,
File "/data/apps/ming-main/MING-main/fastchat/serve/inference.py", line 83, in load_model
model = AutoModelForCausalLM.from_pretrained(model_path,
File "/data/miniconda3/envs/ming-main/lib/python3.9/site-packages/transformers/models/auto/auto_factory.py", line 563, in from_pretrained
return model_class.from_pretrained(
File "/data/miniconda3/envs/ming-main/lib/python3.9/site-packages/transformers/modeling_utils.py", line 3680, in from_pretrained
) = cls._load_pretrained_model(
File "/data/miniconda3/envs/ming-main/lib/python3.9/site-packages/transformers/modeling_utils.py", line 4087, in _load_pretrained_model
state_dict = load_state_dict(shard_file, is_quantized=is_quantized)
File "/data/miniconda3/envs/ming-main/lib/python3.9/site-packages/transformers/modeling_utils.py", line 556, in load_state_dict
raise OSError(
OSError: Unable to load weights from pytorch checkpoint file for '/home/Medical_LLM/MING-7B/pytorch_model-00001-of-00004.bin' at '/home/Medical_LLM/MING-7B/pytorch_model-00001-of-00004.bin'. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.

有没有大佬遇到这种问题,有无解决方法?谢谢!!

@coldy1992
Copy link
Author

@BlueZeros 求助

@BlueZeros
Copy link
Collaborator

@coldy1992 哈喽,我发现readme中的命令给错了,现在已经更新,可以试试这个命令行不行 : )

CUDA_VISIBLE_DEVICES=0 python -m ming/serve/cli.py \
    --model_path {path_to_checkpoint} \ # 模型路径
    --conv_template bloom \ # prompt
    --max_new_token 512 \ # 输出最大长度
    --beam_size 3 \ # beam search宽度
    --temperature 1.2 # 采样温度

@coldy1992
Copy link
Author

@coldy1992 哈喽,我发现readme中的命令给错了,现在已经更新,可以试试这个命令行不行 : )

CUDA_VISIBLE_DEVICES=0 python -m ming/serve/cli.py \
    --model_path {path_to_checkpoint} \ # 模型路径
    --conv_template bloom \ # prompt
    --max_new_token 512 \ # 输出最大长度
    --beam_size 3 \ # beam search宽度
    --temperature 1.2 # 采样温度

@BlueZeros hi,后来确认是下载的模型文件受损导致的运行失败,重新下载模型文件后,原readme中的命令也是可以正常启动的,谢谢您的解答~

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants