chatglm3 微调完成之后导出成功，但无法加载 #1307

Naozumi520 · 2023-10-29T10:25:28Z

OSError: C:\Users\Naozu\Downloads\chatglm3-6b-cantonese-stage1 does not appear to have a file named config.json. Checkout 'https://huggingface.co/C:\Users\Naozu\Downloads\chatglm3-6b-cantonese-stage1/main' for available files.
10/29/2023 18:19:09 - WARNING - llmtuner.tuner.core.loader - Checkpoint is not found at evaluation, load the original model.
Traceback (most recent call last):
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\gradio\routes.py", line 442, in run_predict
    output = await app.get_blocks().process_api(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\gradio\blocks.py", line 1389, in process_api
    result = await self.call_function(
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\gradio\blocks.py", line 1108, in call_function
    prediction = await utils.async_iteration(iterator)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\gradio\utils.py", line 346, in async_iteration
    return await iterator.__anext__()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\gradio\utils.py", line 339, in __anext__
    return await anyio.to_thread.run_sync(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\anyio\to_thread.py", line 33, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\anyio\_backends\_asyncio.py", line 2106, in run_sync_in_worker_thread
    return await future
           ^^^^^^^^^^^^
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\anyio\_backends\_asyncio.py", line 833, in run
    result = context.run(func, *args)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\gradio\utils.py", line 322, in run_sync_iterator_async
    return next(iterator)
           ^^^^^^^^^^^^^^
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\gradio\utils.py", line 691, in gen_wrapper
    yield from f(*args, **kwargs)
  File "C:\Users\Naozu\Downloads\LLaMA-Factory-main\src\llmtuner\webui\chatter.py", line 63, in load_model
    super().__init__(args)
  File "C:\Users\Naozu\Downloads\LLaMA-Factory-main\src\llmtuner\chat\stream_chat.py", line 15, in __init__
    self.model, self.tokenizer = load_model_and_tokenizer(model_args, finetuning_args)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Naozu\Downloads\LLaMA-Factory-main\src\llmtuner\tuner\core\loader.py", line 71, in load_model_and_tokenizer
    tokenizer = AutoTokenizer.from_pretrained(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\transformers\models\auto\tokenization_auto.py", line 738, in from_pretrained
    return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\transformers\tokenization_utils_base.py", line 2017, in from_pretrained
    return cls._from_pretrained(
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\transformers\tokenization_utils_base.py", line 2249, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Naozu/.cache\huggingface\modules\transformers_modules\chatglm3-6b-cantonese-stage1\tokenization_chatglm.py", line 93, in __init__
    super().__init__(padding_side=padding_side, clean_up_tokenization_spaces=clean_up_tokenization_spaces, **kwargs)
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\transformers\tokenization_utils.py", line 363, in __init__
    super().__init__(**kwargs)
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\transformers\tokenization_utils_base.py", line 1604, in __init__
    super().__init__(**kwargs)
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\transformers\tokenization_utils_base.py", line 861, in __init__
    setattr(self, key, value)
AttributeError: property 'eos_token' of 'ChatGLMTokenizer' object has no setter
Traceback (most recent call last):
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\gradio\routes.py", line 442, in run_predict
    output = await app.get_blocks().process_api(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\gradio\blocks.py", line 1389, in process_api
    result = await self.call_function(
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\gradio\blocks.py", line 1108, in call_function
    prediction = await utils.async_iteration(iterator)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\gradio\utils.py", line 346, in async_iteration
    return await iterator.__anext__()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\gradio\utils.py", line 339, in __anext__
    return await anyio.to_thread.run_sync(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\anyio\to_thread.py", line 33, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\anyio\_backends\_asyncio.py", line 2106, in run_sync_in_worker_thread
    return await future
           ^^^^^^^^^^^^
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\anyio\_backends\_asyncio.py", line 833, in run
    result = context.run(func, *args)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\gradio\utils.py", line 322, in run_sync_iterator_async
    return next(iterator)
           ^^^^^^^^^^^^^^
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\gradio\utils.py", line 691, in gen_wrapper
    yield from f(*args, **kwargs)
  File "C:\Users\Naozu\Downloads\LLaMA-Factory-main\src\llmtuner\webui\chatter.py", line 63, in load_model
    super().__init__(args)
  File "C:\Users\Naozu\Downloads\LLaMA-Factory-main\src\llmtuner\chat\stream_chat.py", line 15, in __init__
    self.model, self.tokenizer = load_model_and_tokenizer(model_args, finetuning_args)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Naozu\Downloads\LLaMA-Factory-main\src\llmtuner\tuner\core\loader.py", line 71, in load_model_and_tokenizer
    tokenizer = AutoTokenizer.from_pretrained(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\transformers\models\auto\tokenization_auto.py", line 738, in from_pretrained
    return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\transformers\tokenization_utils_base.py", line 2017, in from_pretrained
    return cls._from_pretrained(
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\transformers\tokenization_utils_base.py", line 2249, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Naozu/.cache\huggingface\modules\transformers_modules\chatglm3-6b-cantonese-stage1\tokenization_chatglm.py", line 93, in __init__
    super().__init__(padding_side=padding_side, clean_up_tokenization_spaces=clean_up_tokenization_spaces, **kwargs)
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\transformers\tokenization_utils.py", line 363, in __init__
    super().__init__(**kwargs)
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\transformers\tokenization_utils_base.py", line 1604, in __init__
    super().__init__(**kwargs)
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\transformers\tokenization_utils_base.py", line 861, in __init__
    setattr(self, key, value)
AttributeError: property 'eos_token' of 'ChatGLMTokenizer' object has no setter

The text was updated successfully, but these errors were encountered:

hiyouga · 2023-10-29T14:32:54Z

导出后的目录是什么样的

Naozumi520 · 2023-10-29T14:54:19Z

config.json
configuration_chatglm.py
generation_config.json
modeling_chatglm.py
pytorch_model-00001-of-00002.bin
pytorch_model-00002-of-00002.bin
pytorch_model.bin.index.json
quantization.py
special_tokens_map.json
tokenization_chatglm.py
tokenizer.model
tokenizer_config.json

hiyouga · 2023-10-29T14:56:13Z

看起来是 Windows 路径识别的问题

Naozumi520 · 2023-10-29T15:11:40Z

有辦法解決嗎

Naozumi520 · 2023-10-30T06:54:46Z

大佬在嗎, 我只能在windows下加載, 沒有其他辦法了

hiyouga · 2023-10-30T07:19:08Z

建议使用 WSL

Naozumi520 · 2023-10-30T10:00:26Z

Still, even installed ubuntu on my PC

Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:04<00:00,  1.50it/s]
10/30/2023 17:56:17 - INFO - llmtuner.tuner.core.adapter - Fine-tuning method: LoRA
10/30/2023 17:56:21 - INFO - llmtuner.tuner.core.adapter - Merged 1 model checkpoint(s).
10/30/2023 17:56:21 - INFO - llmtuner.tuner.core.adapter - Loaded fine-tuned model from checkpoint(s): saves/ChatGLM2-6B-Chat/lora/chatglm3_cantonese_pretraining_epoch3
10/30/2023 17:56:21 - INFO - llmtuner.tuner.core.loader - trainable params: 0 || all params: 6243584000 || trainable%: 0.0000
10/30/2023 17:56:21 - INFO - llmtuner.tuner.core.loader - This IS expected that the trainable params is 0 if you are using model for inference only.
10/30/2023 17:58:48 - WARNING - llmtuner.tuner.core.loader - Checkpoint is not found at evaluation, load the original model.
Traceback (most recent call last):
  File "/home/naozumi/.local/lib/python3.10/site-packages/gradio/routes.py", line 442, in run_predict
    output = await app.get_blocks().process_api(
  File "/home/naozumi/.local/lib/python3.10/site-packages/gradio/blocks.py", line 1389, in process_api
    result = await self.call_function(
  File "/home/naozumi/.local/lib/python3.10/site-packages/gradio/blocks.py", line 1108, in call_function
    prediction = await utils.async_iteration(iterator)
  File "/home/naozumi/.local/lib/python3.10/site-packages/gradio/utils.py", line 346, in async_iteration
    return await iterator.__anext__()
  File "/home/naozumi/.local/lib/python3.10/site-packages/gradio/utils.py", line 339, in __anext__
    return await anyio.to_thread.run_sync(
  File "/home/naozumi/.local/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
  File "/home/naozumi/.local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2106, in run_sync_in_worker_thread
    return await future
  File "/home/naozumi/.local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 833, in run
    result = context.run(func, *args)
  File "/home/naozumi/.local/lib/python3.10/site-packages/gradio/utils.py", line 322, in run_sync_iterator_async
    return next(iterator)
  File "/home/naozumi/.local/lib/python3.10/site-packages/gradio/utils.py", line 691, in gen_wrapper
    yield from f(*args, **kwargs)
  File "/home/naozumi/下載/LLaMA-Factory-main/src/llmtuner/webui/chatter.py", line 63, in load_model
    super().__init__(args)
  File "/home/naozumi/下載/LLaMA-Factory-main/src/llmtuner/chat/stream_chat.py", line 15, in __init__
    self.model, self.tokenizer = load_model_and_tokenizer(model_args, finetuning_args)
  File "/home/naozumi/下載/LLaMA-Factory-main/src/llmtuner/tuner/core/loader.py", line 71, in load_model_and_tokenizer
    tokenizer = AutoTokenizer.from_pretrained(
  File "/home/naozumi/.local/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 652, in from_pretrained
    tokenizer_config = get_tokenizer_config(pretrained_model_name_or_path, **kwargs)
  File "/home/naozumi/.local/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 496, in get_tokenizer_config
    resolved_config_file = cached_file(
  File "/home/naozumi/.local/lib/python3.10/site-packages/transformers/utils/hub.py", line 417, in cached_file
    resolved_file = hf_hub_download(
  File "/home/naozumi/.local/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 110, in _inner_fn
    validate_repo_id(arg_value)
  File "/home/naozumi/.local/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 158, in validate_repo_id
    raise HFValidationError(
huggingface_hub.utils._validators.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/home/naozumi/下載/chatglm2_cantonese'. Use `repo_type` argument if needed.

aleSheng · 2023-10-31T00:58:36Z

我用WSL也是有这个问题

ganting · 2023-10-31T06:54:21Z

+1，也遇到这个问题，导出后，在 ubuntu 22.04 无法使用模型

FlashAttention-2 is not installed, ignore this if you are not using FlashAttention.
10/31/2023 14:53:23 - WARNING - llmtuner.tuner.core.loader - Checkpoint is not found at evaluation, load the original model.
Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/doucai/.vscode-server/extensions/ms-python.python-2023.18.0/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/__main__.py", line 39, in <module>
    cli.main()
  File "/home/doucai/.vscode-server/extensions/ms-python.python-2023.18.0/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 430, in main
    run()
  File "/home/doucai/.vscode-server/extensions/ms-python.python-2023.18.0/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 284, in run_file
    runpy.run_path(target, run_name="__main__")
  File "/home/doucai/.vscode-server/extensions/ms-python.python-2023.18.0/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 321, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/home/doucai/.vscode-server/extensions/ms-python.python-2023.18.0/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 135, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/home/doucai/.vscode-server/extensions/ms-python.python-2023.18.0/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 124, in _run_code
    exec(code, run_globals)
  File "/home/doucai/LLaMA-Factory/src/api_demo.py", line 14, in <module>
    main()
  File "/home/doucai/LLaMA-Factory/src/api_demo.py", line 7, in main
    chat_model = ChatModel()
  File "/home/doucai/LLaMA-Factory/src/llmtuner/chat/stream_chat.py", line 15, in __init__
    self.model, self.tokenizer = load_model_and_tokenizer(model_args, finetuning_args)
  File "/home/doucai/LLaMA-Factory/src/llmtuner/tuner/core/loader.py", line 71, in load_model_and_tokenizer
    tokenizer = AutoTokenizer.from_pretrained(
  File "/home/doucai/LLaMA-Factory/venv/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 738, in from_pretrained
    return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
  File "/home/doucai/LLaMA-Factory/venv/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2017, in from_pretrained
    return cls._from_pretrained(
  File "/home/doucai/LLaMA-Factory/venv/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2249, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
  File "/home/doucai/.cache/huggingface/modules/transformers_modules/xhs_merge/tokenization_chatglm.py", line 93, in __init__
    super().__init__(padding_side=padding_side, clean_up_tokenization_spaces=clean_up_tokenization_spaces, **kwargs)
  File "/home/doucai/LLaMA-Factory/venv/lib/python3.10/site-packages/transformers/tokenization_utils.py", line 363, in __init__
    super().__init__(**kwargs)
  File "/home/doucai/LLaMA-Factory/venv/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1604, in __init__
    super().__init__(**kwargs)
  File "/home/doucai/LLaMA-Factory/venv/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 861, in __init__
    setattr(self, key, value)
AttributeError: can't set attribute 'eos_token'

hiyouga · 2023-10-31T06:59:38Z

把源目录除了 bin 和 pytorch_model.bin.index.json 以外的文件全部复制到导出目录中覆盖

nansanhao · 2023-12-05T12:17:21Z

把源目录除了 bin 和 pytorch_model.bin.index.json 以外的文件全部复制到导出目录中覆盖

这里会导致报这个错误：THUDM/ChatGLM3#152 (comment)
@hiyouga

ghx2757 · 2023-12-07T02:40:10Z

@ganting #1307 (comment) 请问你这个问题怎么解决的呢？

ghx2757 · 2023-12-07T02:45:48Z

@nansanhao AttributeError: can't set attribute 'eos_token' 这个错误你是如何解决的？

nansanhao · 2023-12-07T02:48:09Z

@nansanhao AttributeError: can't set attribute 'eos_token' 这个错误你是如何解决的？

@ghx2757 复制原模型的tokenizer_config.json文件到新模型下，覆盖新模型的这个文件

ghx2757 · 2023-12-07T02:50:53Z

@nansanhao AttributeError: can't set attribute 'eos_token' 这个错误你是如何解决的？

@ghx2757 复制原模型的tokenizer_config.json文件到新模型下，覆盖新模型的这个文件

我按照你所说的方式进行了tokenizer_config.json文件替换，但是模型的回复效果上和没有进行微调时一样，你有出现该问题吗？

nansanhao · 2023-12-07T02:52:03Z

@nansanhao AttributeError: can't set attribute 'eos_token' 这个错误你是如何解决的？

@ghx2757 复制原模型的tokenizer_config.json文件到新模型下，覆盖新模型的这个文件

我按照你所说的方式进行了tokenizer_config.json文件替换，但是模型的回复效果上和没有进行微调时一样，你有出现该问题吗？

我是全参数微调

ghx2757 · 2023-12-07T02:57:49Z

@nansanhao AttributeError: can't set attribute 'eos_token' 这个错误你是如何解决的？

@ghx2757 复制原模型的tokenizer_config.json文件到新模型下，覆盖新模型的这个文件

我按照你所说的方式进行了tokenizer_config.json文件替换，但是模型的回复效果上和没有进行微调时一样，你有出现该问题吗？

我是全参数微调

好的，谢谢，我在看看吧..

migrant620 · 2023-12-08T01:45:58Z

@nansanhao AttributeError: can't set attribute 'eos_token' 这个错误你是如何解决的？

@ghx2757 复制原模型的tokenizer_config.json文件到新模型下，覆盖新模型的这个文件

我按照你所说的方式进行了tokenizer_config.json文件替换，但是模型的回复效果上和没有进行微调时一样，你有出现该问题吗？

我是全参数微调

好的，谢谢，我在看看吧..

你好，请问下你这问题解决了吗？我也是照着作者的办法解决了加载的问题，但微调的结果在推理过程中失效了，这和在web demo中运行的结果不一样，本项目的web demo中加载后推理是能呈现微调效果的

ghx2757 · 2023-12-08T03:10:51Z

@migrant620 #1307 (comment) 还没...还在解决中

hiyouga · 2023-12-08T08:15:21Z

@migrant620 你需要加 system prompt

LLaMA-Factory/src/llmtuner/data/template.py

Lines 364 to 367 in 3378337

    
           system=( 
        
               "You are ChatGLM3, a large language model trained by Zhipu.AI. " 
        
               "Follow the user's instructions carefully. Respond using markdown." 
        
           ),

ItsFated · 2023-12-09T02:55:49Z

@migrant620 你需要加 system prompt

LLaMA-Factory/src/llmtuner/data/template.py

Lines 364 to 367 in 3378337

system=(

"You are ChatGLM3, a large language model trained by Zhipu.AI. "

"Follow the user's instructions carefully. Respond using markdown."

),

新手提问，这个加System Prompt是直接改这段代码，还是说要修改Dataset增加系统提示词字段

hiyouga · 2023-12-09T02:59:53Z

@ItsFated 直接用 https://github.com/THUDM/ChatGLM3/tree/main/composite_demo

migrant620 · 2023-12-11T03:02:34Z

@migrant620 你需要加 system prompt

LLaMA-Factory/src/llmtuner/data/template.py

Lines 364 to 367 in 3378337

system=(

"You are ChatGLM3, a large language model trained by Zhipu.AI. "

"Follow the user's instructions carefully. Respond using markdown."

),

@hiyouga 谢谢老师，这样的话就意味着每次提问都需要加role=system的指令了对吧，实际我想达到的效果是不经过system而通过微调方式改变模型的自我认知，这在glm二代是可以的

hiyouga · 2023-12-11T06:52:26Z

@migrant620 你改的方法还是微调，我只是说让你在推理时候带上之前的 system prompt

migrant620 · 2023-12-12T02:18:04Z

@migrant620 你改的方法还是微调，我只是说让你在推理时候带上之前的 system prompt

我改的是微调，比如我微调的语料就是为了改变他的自我认知。如果说推理时不带上system prompt的话，微调的结果现在看是没起作用

mavisyyc · 2023-12-18T06:19:13Z

@nansanhao AttributeError: can't set attribute 'eos_token' 这个错误你是如何解决的？

@ghx2757 复制原模型的tokenizer_config.json文件到新模型下，覆盖新模型的这个文件

我按照你所说的方式进行了tokenizer_config.json文件替换，但是模型的回复效果上和没有进行微调时一样，你有出现该问题吗？

我是全参数微调

好的，谢谢，我在看看吧..

你好，请问下你这问题解决了吗？我也是照着作者的办法解决了加载的问题，但微调的结果在推理过程中失效了，这和在web demo中运行的结果不一样，本项目的web demo中加载后推理是能呈现微调效果的

您好，请问您解决了吗？我也遇到了这问题

GitYohoo · 2023-12-19T03:32:35Z

@nansanhao AttributeError: can't set attribute 'eos_token' 这个错误你是如何解决的？

@ghx2757 复制原模型的tokenizer_config.json文件到新模型下，覆盖新模型的这个文件

我按照你所说的方式进行了tokenizer_config.json文件替换，但是模型的回复效果上和没有进行微调时一样，你有出现该问题吗？

我也是同样的问题

xiabo0816 · 2024-03-05T03:44:45Z

我是精调chatglm3-6b-32k，也遇到了AttributeError: can't set attribute 'eos_token'的问题，解决方式是编辑tokenizer_config.json，删除其中的eos_token、pad_token、unk_token就可以了

qianlongqf · 2024-03-15T14:14:37Z

我是精调chatglm3-6b-32k，也遇到了AttributeError: can't set attribute 'eos_token'的问题，解决方式是编辑tokenizer_config.json，删除其中的eos_token、pad_token、unk_token就可以了

出现notImplementdError

RENLINA123 · 2024-04-07T08:32:09Z

@migrant620 #1307 (comment) 还没...还在解决中

您好，请问这个问题有解决吗？

xiaowuzicode · 2024-04-08T09:02:57Z

我是精调chatglm3-6b-32k，也遇到了AttributeError: can't set attribute 'eos_token'的问题，解决方式是编辑tokenizer_config.json，删除其中的eos_token、pad_token、unk_token就可以了

不报错了，调完的没效果了，自我认知的

RENLINA123 · 2024-04-09T06:02:05Z

把源目录除了 bin 和 pytorch_model.bin.index.json 以外的文件全部复制到导出目录中覆盖

@hiyouga 你好，将其他文件覆盖后导致微调没有效果，请问应该如何解决？

snakecy · 2024-04-25T04:14:52Z

同样的问题。property 'eos_token' of 'ChatGLMTokenizer'

hiyouga added the pending This problem is yet to be addressed label Oct 29, 2023

hiyouga added wontfix This will not be worked on and removed pending This problem is yet to be addressed labels Oct 30, 2023

hiyouga closed this as not planned Won't fix, can't repro, duplicate, stale Oct 30, 2023

hiyouga added solved This problem has been already solved and removed wontfix This will not be worked on labels Oct 31, 2023

hiyouga closed this as completed Oct 31, 2023

This was referenced Nov 1, 2023

ChatGLM3 全参数微调后，加载checkpoint报错 #1340

Closed

ChatGLM3-6B，增量预训练后，导出独立模型，再使用SFT-LoRA微调报错 #1399

Closed

hiyouga mentioned this issue Nov 9, 2023

chatglm3-6b通过lora微调后导出模型，加载导出的模型报错AttributeError: can't set attribute 'eos_token' #1442

Closed

ghx2757 mentioned this issue Dec 6, 2023

ChatGLM3-6b合并lora微调模型后，通过ChatGLM3-6b官方代码载入该合并后权重后对话内容和效果与没微调一样.. #1752

Closed

Yuanye-F mentioned this issue Dec 14, 2023

使用 ADAPTER_MODEL_PATH 加载 QLoRA 微调的 ChatGLM3 模型失败 xusenlinzy/api-for-open-llm#200

Closed

2 tasks

FoolMark mentioned this issue Dec 20, 2023

请问在预训练时，如何保持书籍等数据的换行符号？如何把书籍切成block？ #1891

Closed

TankNee mentioned this issue Dec 30, 2023

AttributeError: can't set attribute 'eos_token' THUDM/ChatGLM3#152

Closed

kevindany mentioned this issue Jan 22, 2024

chatglm3-6b-base做lora sft微调后的权重和base模型merge后的效果下降近40% #2276

Closed

1 task

hiyouga mentioned this issue Jan 22, 2024

chaglm3-6b训练完，导出模型后训练内容失效，但是在训练页加载模型后，训练内容是有效的 #2286

Closed

1 task

This was referenced Jan 29, 2024

chatglm3在进行ceval 评估时找不到eostoken #2353

Closed

自我意识lora微调后，通过api在ChatGPT-Next-Web进行对话，回答混乱 #2437

Closed

hiyouga mentioned this issue Feb 6, 2024

lora微调chatglm3-6b-chat使用openai api调用没效果 #2439

Closed

hiyouga mentioned this issue Mar 28, 2024

对ChatGLM模型微调后输出成模型后加载出错AttributeError: can't set attribute 'eos_token' #3028

Closed

1 task

sweetning0809 mentioned this issue Jun 20, 2024

关于npu训练模型总结以及疑问 #4388

Open

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chatglm3 微调完成之后导出成功，但无法加载 #1307

chatglm3 微调完成之后导出成功，但无法加载 #1307

Naozumi520 commented Oct 29, 2023

hiyouga commented Oct 29, 2023

Naozumi520 commented Oct 29, 2023

hiyouga commented Oct 29, 2023

Naozumi520 commented Oct 29, 2023

Naozumi520 commented Oct 30, 2023

hiyouga commented Oct 30, 2023

Naozumi520 commented Oct 30, 2023

aleSheng commented Oct 31, 2023

ganting commented Oct 31, 2023 •

edited

Loading

hiyouga commented Oct 31, 2023

nansanhao commented Dec 5, 2023 •

edited

Loading

ghx2757 commented Dec 7, 2023

ghx2757 commented Dec 7, 2023

nansanhao commented Dec 7, 2023

ghx2757 commented Dec 7, 2023

nansanhao commented Dec 7, 2023

ghx2757 commented Dec 7, 2023

migrant620 commented Dec 8, 2023

ghx2757 commented Dec 8, 2023

hiyouga commented Dec 8, 2023

ItsFated commented Dec 9, 2023

hiyouga commented Dec 9, 2023

migrant620 commented Dec 11, 2023

hiyouga commented Dec 11, 2023

migrant620 commented Dec 12, 2023

mavisyyc commented Dec 18, 2023

GitYohoo commented Dec 19, 2023

xiabo0816 commented Mar 5, 2024

qianlongqf commented Mar 15, 2024

RENLINA123 commented Apr 7, 2024

xiaowuzicode commented Apr 8, 2024

RENLINA123 commented Apr 9, 2024

snakecy commented Apr 25, 2024

chatglm3 微调完成之后导出成功，但无法加载 #1307

chatglm3 微调完成之后导出成功，但无法加载 #1307

Comments

Naozumi520 commented Oct 29, 2023

hiyouga commented Oct 29, 2023

Naozumi520 commented Oct 29, 2023

hiyouga commented Oct 29, 2023

Naozumi520 commented Oct 29, 2023

Naozumi520 commented Oct 30, 2023

hiyouga commented Oct 30, 2023

Naozumi520 commented Oct 30, 2023

aleSheng commented Oct 31, 2023

ganting commented Oct 31, 2023 • edited Loading

hiyouga commented Oct 31, 2023

nansanhao commented Dec 5, 2023 • edited Loading

ghx2757 commented Dec 7, 2023

ghx2757 commented Dec 7, 2023

nansanhao commented Dec 7, 2023

ghx2757 commented Dec 7, 2023

nansanhao commented Dec 7, 2023

ghx2757 commented Dec 7, 2023

migrant620 commented Dec 8, 2023

ghx2757 commented Dec 8, 2023

hiyouga commented Dec 8, 2023

ItsFated commented Dec 9, 2023

hiyouga commented Dec 9, 2023

migrant620 commented Dec 11, 2023

hiyouga commented Dec 11, 2023

migrant620 commented Dec 12, 2023

mavisyyc commented Dec 18, 2023

GitYohoo commented Dec 19, 2023

xiabo0816 commented Mar 5, 2024

qianlongqf commented Mar 15, 2024

RENLINA123 commented Apr 7, 2024

xiaowuzicode commented Apr 8, 2024

RENLINA123 commented Apr 9, 2024

snakecy commented Apr 25, 2024

ganting commented Oct 31, 2023 •

edited

Loading

nansanhao commented Dec 5, 2023 •

edited

Loading