The problem with the conversion with the new convert.py

[SrVill]

Hello! Help me figure out:

F:\Models\digitous-Alpacino13b>convert.py --dump-single F:\Models\digitous-Alpacino13b\4bit.safetensors
Traceback (most recent call last):
File "F:\Models\digitous-Alpacino13b\convert.py", line 1145, in
main()
File "F:\Models\digitous-Alpacino13b\convert.py", line 1116, in main
model_plus = lazy_load_file(args.model)
File "F:\Models\digitous-Alpacino13b\convert.py", line 853, in lazy_load_file
return lazy_load_safetensors_file(fp, path)
File "F:\Models\digitous-Alpacino13b\convert.py", line 753, in lazy_load_safetensors_file
model = {name: convert(info) for (name, info) in header.items()}
File "F:\Models\digitous-Alpacino13b\convert.py", line 753, in
model = {name: convert(info) for (name, info) in header.items()}
File "F:\Models\digitous-Alpacino13b\convert.py", line 745, in convert
assert 0 <= begin <= end <= len(byte_buf)
AssertionError

What is the error here - in the script or maybe there is a problem in the model? The model is from here: https://huggingface.co/digitous/Alpacino13b/tree/main

[kickturn]

Same error here going on with me, i'm using a different model however but get the exact same error produced on latest commit

Traceback (most recent call last):
  File "D:\llama\llama.cpp\convert.py", line 1146, in <module>
    main()
  File "D:\llama\llama.cpp\convert.py", line 1126, in main
    model_plus = load_some_model(args.model)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\llama\llama.cpp\convert.py", line 1053, in load_some_model
    models_plus.append(lazy_load_file(path))
                       ^^^^^^^^^^^^^^^^^^^^
  File "D:\llama\llama.cpp\convert.py", line 854, in lazy_load_file
    return lazy_load_safetensors_file(fp, path)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\llama\llama.cpp\convert.py", line 754, in lazy_load_safetensors_file
    model = {name: convert(info) for (name, info) in header.items()}
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\llama\llama.cpp\convert.py", line 754, in <dictcomp>
    model = {name: convert(info) for (name, info) in header.items()}
                   ^^^^^^^^^^^^^
  File "D:\llama\llama.cpp\convert.py", line 746, in convert
    assert 0 <= begin <= end <= len(byte_buf)
AssertionError

[kickturn]

Apparently I fixed it by using Linux? Use WSL or an alternative and run the script again. I'm not versed in python that much to figure out the error but it doesn't seem to fall into an assertionerror on linux with no alterations whatsoever.

[comex]

Let's see...

[kickturn]

just checked and the commit fixed it!

[SrVill]

The same mistake. Maybe I'm doing something wrong?

C:\llama.cpp>convert.py 4bit.safetensors --outtype q4_1 --outfile 4ggml.bin
Loading model file 4bit.safetensors
Traceback (most recent call last):
File "C:\llama.cpp\convert.py", line 1145, in
main()
File "C:\llama.cpp\convert.py", line 1125, in main
model_plus = load_some_model(args.model)
File "C:\llama.cpp\convert.py", line 1052, in load_some_model
models_plus.append(lazy_load_file(path))
File "C:\llama.cpp\convert.py", line 853, in lazy_load_file
return lazy_load_safetensors_file(fp, path)
File "C:\llama.cpp\convert.py", line 753, in lazy_load_safetensors_file
model = {name: convert(info) for (name, info) in header.items()}
File "C:\llama.cpp\convert.py", line 753, in
model = {name: convert(info) for (name, info) in header.items()}
File "C:\llama.cpp\convert.py", line 745, in convert
assert 0 <= begin <= end <= len(byte_buf)
AssertionError

[NextGA-OSS]

Seeing similar issues still. Latest repo trying to set up an apple m2 implementation using mixtral:

% python3 convert.py ./models/mixtral-instruct-8x7b/ \
         --outfile ./models/mixtral-instruct-8x7b/ggml-model-f16.gguf \
         --outtype f16
Loading model file models/mixtral-instruct-8x7b/model-00001-of-00019.safetensors
Traceback (most recent call last):
  File "/Users/user/Documents/Workspace/llama/convert.py", line 1279, in <module>
    main()
  File "/Users/user/Documents/Workspace/llama/convert.py", line 1207, in main
    model_plus = load_some_model(args.model)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/Documents/Workspace/llama/convert.py", line 1140, in load_some_model
    models_plus.append(lazy_load_file(path))
                       ^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/Documents/Workspace/llama/convert.py", line 797, in lazy_load_file
    raise ValueError(f"unknown format: {path}")
ValueError: unknown format: models/mixtral-instruct-8x7b/model-00001-of-00019.safetensors

[prusnak]

Seeing similar issues still.

It's a very different issue. Open a new issue.

[shrijayan]

SrVill

Facing the same issue in Google Colab. Any Solution?