The prompt is not converted to tokens

[zhoujian1028]

./main -m ./models/7B/ggml-model-q4_0.bin -p "Building a website can be done in 10 simple steps:" -t 8 -n 512

llama.cpp Is the latest version
Can anyone help me? Thanks!

[zhoujian1028]

@ggerganov Please，Thanks！

[MillionthOdin16]

I'm seeing the same issue. Model works fine in f16 version, but doesn't work after quantization

[caseylai]

Same issue here. Everything is fine on my mac laptop but after I deployed it on Linux (Centos 7), I found this issue.
Have tried to change the version of gcc, python, torch, and tried to modify the source code of 'llama_tokenize' to make the tokenizer working as expected. But none of these works. And also checked md5 sum for all files, all of the md5 sum are right.

Environment:
Mac (works fine): gcc 9.4.0, Python 3.9.6, Torch 1.9.0
Centos 7 (issue): gcc 9.3.1, Python 3.9.16, Torch 1.9.1

[MillionthOdin16]

I saw in some other issues that some people tried making sure their project was up to date and then rebuilt it. I rebuilt mine, requantized, then ran it, and it started working. I have no idea why 😬

…

On Fri, Mar 17, 2023, 04:28 Casey Lai ***@***.***> wrote: Same issue here. Everything is fine on my mac laptop but after I deployed it on Linux (Centos 7), I found this issue. Have tried to change the version of gcc, python, torch, and tried to modify the source code of 'llama_tokenize' to make the tokenizer working as expected. But none of these works. And also checked md5 sum for all files, all of the md5 sum are right. Environment: Mac (works fine): gcc 9.4.0, Python 3.9.6, Torch 1.9.0 Centos 7 (issue): gcc 9.3.1, Python 3.9.16, Torch 1.9.1 — Reply to this email directly, view it on GitHub <#113 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AYMC3ACXD3YF5F2FAYIQWMTW4QOC7ANCNFSM6AAAAAAVZ5AVUY> . You are receiving this because you commented.Message ID: <ggerganov/llama. ***@***.***>

[caseylai]

./main -m ./models/7B/ggml-model-q4_0.bin -p "Building a website can be done in 10 simple steps:" -t 8 -n 512

llama.cpp Is the latest version Can anyone help me? Thanks!

@zhoujian1028 I tried to deploy on another machine and it worked smoothly, for your reference: Ubuntu 22.04 + gcc11.3.0 + python 3.10.9 + torch 1.13.1

[xlichao]

./main -m ./models/7B/ggml-model-q4_0.bin -p "Building a website can be done in 10 simple steps:" -t 8 -n 512

llama.cpp Is the latest version Can anyone help me? Thanks!

Centos 7 even with sudo yum install devtoolset-11-gcc* can't work and output nothing meaningful, but just using an docker conatiner base on ubuntu jemmy with sudo apt-get install build-essential everything works fine in fp16.

[ar-jan]

Give it another try now that #252 landed.

[sw]

No complaints after three weeks, let's assume this is fixed, possibly by #252.