ChatGLM-6B模型微调报错

[dean-lhb]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

[WARNING|modeling_utils.py:3034] 2023-05-08 16:09:35,083 >> Some weights of ChatGLMForConditionalGeneration were not initialized from the model checkpoint at D:\project\python\chatglm\ChatGLM-6B\model2 and are newly initialized: ['transformer.prefix_encoder.embedding.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
[INFO|modeling_utils.py:2690] 2023-05-08 16:09:35,127 >> Generation config file not found, using a generation config created from the model config.
D:/app/gcc/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/10.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot open output file C:\Users\Administrator.cache\huggingface\modules\transformers_modules\model2\quantization_kernels.so: Permission denied
collect2.exe: error: ld returned 1 exit status
No compiled kernel found.
Compiling kernels : C:\Users\Administrator.cache\huggingface\modules\transformers_modules\model2\quantization_kernels_parallel.c
Compiling gcc -O3 -fPIC -pthread -fopenmp -std=c99 C:\Users\Administrator.cache\huggingface\modules\transformers_modules\model2\quantization_kernels_parallel.c -shared -o C:\Users\Administrator.cache\huggingface\modules\transformers_modules\model2\quantization_kernels_parallel.so
Load parallel cpu kernel failed, using default cpu kernel code:
Traceback (most recent call last):
File "C:\Users\Administrator/.cache\huggingface\modules\transformers_modules\model2\quantization.py", line 145, in init
kernels = ctypes.cdll.LoadLibrary(kernel_file)
File "D:\app\python\lib\ctypes_init_.py", line 452, in LoadLibrary
return self.dlltype(name)
File "D:\app\python\lib\ctypes_init.py", line 374, in init
self._handle = _dlopen(self._name, mode)
FileNotFoundError: Could not find module 'C:\Users\Administrator.cache\huggingface\modules\transformers_modules\model2\quantization_kernels_parallel.so' (or one of its dependencies). Try using the full path with constructor syntax.

Compiling gcc -O3 -fPIC -std=c99 C:\Users\Administrator.cache\huggingface\modules\transformers_modules\model2\quantization_kernels.c -shared -o C:\Users\Administrator.cache\huggingface\modules\transformers_modules\model2\quantization_kernels.so
Load kernel : C:\Users\Administrator.cache\huggingface\modules\transformers_modules\model2\quantization_kernels.so
Using quantization cache
Applying quantization to glm layers
Quantized to 4 bit
No compiled kernel found.
Compiling kernels : C:\Users\Administrator.cache\huggingface\modules\transformers_modules\model2\quantization_kernels_parallel.c
Compiling gcc -O3 -fPIC -pthread -fopenmp -std=c99 C:\Users\Administrator.cache\huggingface\modules\transformers_modules\model2\quantization_kernels_parallel.c -shared -o C:\Users\Administrator.cache\huggingface\modules\transformers_modules\model2\quantization_kernels_parallel.so
Load parallel cpu kernel failed, using default cpu kernel code:
Traceback (most recent call last):
File "C:\Users\Administrator/.cache\huggingface\modules\transformers_modules\model2\quantization.py", line 145, in init
kernels = ctypes.cdll.LoadLibrary(kernel_file)
File "D:\app\python\lib\ctypes_init_.py", line 452, in LoadLibrary
return self.dlltype(name)
File "D:\app\python\lib\ctypes_init.py", line 374, in init
self._handle = _dlopen(self._name, mode)
FileNotFoundError: Could not find module 'C:\Users\Administrator.cache\huggingface\modules\transformers_modules\model2\quantization_kernels_parallel.so' (or one of its dependencies). Try using the full path with constructor syntax.

Compiling gcc -O3 -fPIC -std=c99 C:\Users\Administrator.cache\huggingface\modules\transformers_modules\model2\quantization_kernels.c -shared -o C:\Users\Administrator.cache\huggingface\modules\transformers_modules\model2\quantization_kernels.so
Compile default cpu kernel failed.
Failed to load kernel.
Traceback (most recent call last):
File "D:\project\python\chatglm\ChatGLM-6B\ptuning\main.py", line 431, in
main()
File "D:\project\python\chatglm\ChatGLM-6B\ptuning\main.py", line 129, in main
model = model.quantize(model_args.quantization_bit)
File "C:\Users\Administrator/.cache\huggingface\modules\transformers_modules\model2\modeling_chatglm.py", line 1430, in quantize
load_cpu_kernel(**kwargs)
File "C:\Users\Administrator/.cache\huggingface\modules\transformers_modules\model2\quantization.py", line 430, in load_cpu_kernel
assert cpu_kernels.load
AssertionError

Expected Behavior

No response

Steps To Reproduce

sh ChatGLM-6B\ptuning\train.sh

Environment

- OS: windows10
- Python: 3.10.10
- Transformers: 4.27.1
- PyTorch: 1.13.1+cu116
- CUDA Support (`python -c "import torch; print(torch.cuda.is_available())"`) : true

Anything else?

No response

[nanqiangyipo]

首先确认一下运行代码后，你的这个文件是否存在C:\Users\Administrator.cache\huggingface\modules\transformers_modules\model2\quantization_kernels_parallel.so。如果不存在需要先解决能够自动编译出这个文件的问题。

排除so文件不存在的情况，如果还存在FileNotFoundError: Could not find module 'C:\Users\Administrator.cache\huggingface\modules\transformers_modules\model2\quantization_kernels_parallel.so'的错误，需要对源码做一点改动，这是因为ctypes在Windows环境下的bug还没修复，反正python3.10目前还有问题，修改参考下图，具体原因参考https://stackoverflow.com/questions/59330863/cant-import-dll-module-in-python/64472088#64472088。

在你下载的源码chatglm-6b-int4/quantization.py的文件中，搜索找到这样两行kernels = ctypes.cdll.LoadLibrary(kernel_file)，都把它们都改成kernels = ctypes.CDLL(kernel_file,winmode=0)。

[dean-lhb]

感谢回复，采用你的修改方案问题已经解决

[xiedongmingming]

卡在这个问题好几天了，终于解决了！感谢感谢

[Qednn]

我按你的方法改了代码，但报错这个
No compiled kernel found.
Compiling kernels : C:\Users\73488.cache\huggingface\modules\transformers_modules\model\quantization_kernels_parallel.c
Compiling gcc -O3 -pthread -fopenmp -std=c99 C:\Users\73488.cache\huggingface\modules\transformers_modules\model\quantization_kernels_parallel.c -shared -o C:\Users\73488.cache\huggingface\modules\transformers_modules\model\quantization_kernels_parallel.so
Kernels compiled : C:\Users\73488.cache\huggingface\modules\transformers_modules\model\quantization_kernels_parallel.so
Traceback (most recent call last):
File "D:\ChatGLM-6B-main\web_demo.py", line 6, in
model = AutoModel.from_pretrained("model", trust_remote_code=True).half().quantize(4).cuda()
File "C:\Users\73488/.cache\huggingface\modules\transformers_modules\model\modeling_chatglm.py", line 1267, in quantize
load_cpu_kernel(**kwargs)
File "C:\Users\73488/.cache\huggingface\modules\transformers_modules\model\quantization.py", line 387, in load_cpu_kernel
cpu_kernels = CPUKernel(**kwargs)
File "C:\Users\73488/.cache\huggingface\modules\transformers_modules\model\quantization.py", line 138, in init
kernels = ctypes.CDLL(kernel_file, winmode=0)
File "D:\anaconda\envs\ChatGLM-6B\lib\ctypes_init_.py", line 374, in init
self._handle = _dlopen(self._name, mode)
OSError: [WinError 1114] 动态链接库(DLL)初始化例程失败。

[yjy8749]

chatglm2-6b-int4
感谢，我的就是windows下这个问题，参考这个改法成功运行了，

[Unmeltable-Ice]

File "D:\ChatGLM-6B-main\web_demo.py", line 6, in model = AutoModel.from_pretrained("model", trust_remote_code=True).half().quantize(4).cuda()

@Qednn 看起来像是你在直接用int4模型的时候还继续用.half().quantize(4)

web_demo.py里
model = AutoModel.from_pretrained("model", trust_remote_code=True).half().quantize(4).cuda()
这行直接改成
model = AutoModel.from_pretrained("model", trust_remote_code=True).cuda()
试一下

[fouyhk]

你好，可以给我分享一下源代码嘛

[JasonDDHZ]

感谢，终于解决了

[rorychen]

首先确认一下运行代码后，你的这个文件是否存在C:\Users\Administrator.cache\huggingface\modules\transformers_modules\model2\quantization_kernels_parallel.so。如果不存在需要先解决能够自动编译出这个文件的问题。

排除so文件不存在的情况，如果还存在FileNotFoundError: Could not find module 'C:\Users\Administrator.cache\huggingface\modules\transformers_modules\model2\quantization_kernels_parallel.so'的错误，需要对源码做一点改动，这是因为ctypes在Windows环境下的bug还没修复，反正python3.10目前还有问题，修改参考下图，具体原因参考https://stackoverflow.com/questions/59330863/cant-import-dll-module-in-python/64472088#64472088。

在你下载的源码chatglm-6b-int4/quantization.py的文件中，搜索找到这样两行kernels = ctypes.cdll.LoadLibrary(kernel_file)，都把它们都改成kernels = ctypes.CDLL(kernel_file,winmode=0)。

很感谢，卡了一天，才搜到这个答案。 正好是python3.10环境:(