error:python3 quantize.py 7B

[sun-rabbit]

When I tried the llama model and run :python3 quantize.py 7B for operation, ```
the "quantize" script was not found in the current location appeared
If you want to use it from another location, set the -- quantify script path argument from the command line

It's still this error. I have also made other attempts: `python3/Users/sunxiaotong/Desktop/llama/llama.cpp/quantize.py - q/ quantize.py 7B`

usage: python3 quantize.py [-h] [-r] [-m MODELS_PATH]
[-q QUANTIZE_SCRIPT_PATH]
{7B,13B,30B,65B} [{7B,13B,30B,65B} ...]
python3 quantize.py: error: argument models: invalid choice: '/Users/sunxiaotong/Desktop/llama/llama.cpp/models/7B/ggml-model-f16.bin' (choose from '7B', '13B', '30B', '65B')

May I ask where I was wrong? Can you give me some suggestions

[besnardjb]

Initially had the same issue on Linux, you need to point using the "-q" argument to the "quantize" binary generated when compiling llama (quantize.py doing a subprocess call to it and it generates this error when calling itself).

python3 quantize.py -q ./BUILD/bin/quantize  7B

[prusnak]

The quantize.py script is not needed anymore. Just fetch the latest code and do this as a quantization step:

./quantize ./models/7B/ggml-model-f16.bin ./models/7B/ggml-model-q4_0.bin 2

[huangzhimin4read]

The quantize.py script is not needed anymore. Just fetch the latest code and do this as a quantization step:

./quantize ./models/7B/ggml-model-f16.bin ./models/7B/ggml-model-q4_0.bin 2

I tried this method in Colab, but it still reports an error:
/bin/bash: ./quantize: No such file or directory

[archiwed]

@besnardjb, i was having the same error and this worked for me.

OS; Linux

[sun-rabbit]

The quantize.py script is not needed anymore. Just fetch the latest code and do this as a quantization step:

./quantize ./models/7B/ggml-model-f16.bin ./models/7B/ggml-model-q4_0.bin 2

Thank you very much， this worked for me.

OS： macBookPro

[sania96]

The quantize.py script is not needed anymore. Just fetch the latest code and do this as a quantization step:

./quantize ./models/7B/ggml-model-f16.bin ./models/7B/ggml-model-q4_0.bin 2

I tried this method in Colab, but it still reports an error: /bin/bash: ./quantize: No such file or directory

Hi, Did you find any solution for colab?
I am having the same issue.
Regards.

[rickb-lb]

zsh: no such file or directory: ./quantize

There is no file called quantize in the llama.cpp folder structure