No such file or directory: '/root/.cache/huggingface/modules/transformers_modules/chatglm-6b/tokenization_chatglm.py'

[Data2Me]

No such file or directory: '/root/.cache/huggingface/modules/transformers_modules/chatglm-6b/tokenization_chatglm.py'

[lich99]

site-packages/transformers/dynamic_module_utils.py Line 177
This appears to be a problem with transformers. A temp file is created, but conflicts may occur when running multiple threads.

[Data2Me]

site-packages/transformers/dynamic_module_utils.py Line 177 This appears to be a problem with transformers. A temp file is created, but conflicts may occur when running multiple threads.

怎么解决呢，我是在docker镜像里面跑的，我把container都删了还是会报错，有时候要过好久可能又可以run起来

[lich99]

while True:
    try:
        tokenizer = AutoTokenizer.from_pretrained(checkpoint, trust_remote_code=True, )
        model = AutoModel.from_pretrained(checkpoint, trust_remote_code=True, )
        break
    except:
        continue

最简单方法套一个try, except

[tu2022]

这个有彻底解决的方法么？我这试了好多次，一直在报这个错，这个也看运气的吗

[EthenZhang]

同样的问题，尝试删除/home/admin/.cache/huggingface/ 文件夹也没用，多试几次成功一次

[zhaochs1995]

同样遇到了，有什么好的解决方式嘛

[jack12356]

修改transformer为4.28.1

[my-yy]

transformers库的一个bug，多进程\多线程同时加载模型时有可能会冲突，升级版本到4.28.1就好了