Quantize python script fails.

[SavageShrimp]

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• I am running the latest code. Development is very rapid so there are no tagged versions as of now.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new bug or useful enhancement to share.

Expected Behavior

I have my llama models stored in models/llama/{7B,13B,30B,65B}.

I expect that when I run the following command that the model will be converted

$ python3 quantize.py --models-path models/llama 30B

Current Behavior

When attempting to quantize the model by running

$ python3 quantize.py --models-path models/llama 30B

I get the following error:

The f16 model ggml-model-f16.bin was not found in models/llama/30B. If you want to use it from another location, set the --models-path argument from the command line.

modifying lines 76-79

        f16_model_parts_paths = map(
            lambda filename: os.path.join(f16_model_path_base, filename),
            glob.glob(f"{f16_model_path_base}*")
        )

To

       f16_model_parts_paths = [ filename for filename in glob.glob(f"{f16_model_path_base}*")]

Makes it work.

$ python3 --version   --> Python 3.8.10

Failure Information (for bugs)

[mqy]

I'm wondering what's the value of f16_model_parts_paths:

f16_model_parts_paths = map(
            lambda filename: os.path.join(f16_model_path_base, filename),
            glob.glob(f"{f16_model_path_base}*")
)
print(f16_model_parts_paths)  # add this line

[SavageShrimp]

It was a list with two items, each item was the paths concatenated into the same string. I will post.

['models/llama/13B/ggml-model-f16.bin/models/llama/13B/ggml-model-f16.bin.1', 'models/llama/13B/ggml-model-f16.bin/models/llama/13B/ggml-model-f16.bin']

[mqy]

I can confirm this bug.

Add to quantize.py, line 81:

        for v in f16_model_parts_paths:
             print(v)

Run

python3 quantize.py --models-path models 7B

Output:

models/7B/ggml-model-f16.bin/models/7B/ggml-model-f16.bin

Succesfully quantized all models.

[j-f1]

Can you tru again with the version of the script in #428? That should fix the issue.

[mqy]

python3 quantize.py --models-path models 7B

Run as expected, great!
Actually, the default model path was created with os.path.join(os.getcwd(), "models"), that's an absolute path.