Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a missing step to the gpt4all instructions #690

Merged
merged 1 commit into from
Apr 2, 2023

Conversation

ThatcherC
Copy link
Contributor

migrate-ggml-2023-03-30-pr613.py is needed to get gpt4all running.

When following the README instructions for gpt4all, I encounted the following error:

data/llama.cpp$ python3 convert-gpt4all-to-ggml.py models/gpt4all-models/gpt4all-lora-quantized.bin models/tokenizer.model

/data/llama.cpp$ ./main -m models/gpt4all-models/gpt4all-lora-unfiltered-quantized.bin -n 128
main: seed = 1680382134
llama_model_load: loading model from 'models/gpt4all-models/gpt4all-lora-unfiltered-quantized.bin' - please wait ...
models/gpt4all-models/gpt4all-lora-unfiltered-quantized.bin: invalid model file (bad magic [got 0x67676d66 want 0x67676a74])
    you most likely need to regenerate your ggml files
    the benefit is you'll get 10-100x faster load times
    see https://github.com/ggerganov/llama.cpp/issues/91
    use convert-pth-to-ggml.py to regenerate from original pth
    use migrate-ggml-2023-03-30-pr613.py if you deleted originals
llama_init_from_file: failed to load model
main: error: failed to load model 'models/gpt4all-models/gpt4all-lora-unfiltered-quantized.bin'

I think the gpt4all conversion script is a bit out of date and produces files that need to be converted with migrate-ggml-2023-03-30-pr613.py.

I was able to run python3 migrate-ggml-2023-03-30-pr613.py models/gpt4all-models/gpt4all-lora-unfiltered-quantized.bin models/gpt4all-models/gpt4all-lora-unfiltered-quantized-pr613.bin which seemed to do the trick. After that, ./main -m models/gpt4all-models/gpt4all-lora-unfiltered-quantized-pr613.bin ran successfully!

This PR updates the README to add that migrate-ggml-2023-03-30-pr613.py step.

`migrate-ggml-2023-03-30-pr613.py` is needed to get gpt4all running.
@sw
Copy link
Contributor

sw commented Apr 2, 2023

Yes, the *-to-ggml.py scripts need to be updated, I've opened an issue for this: #704

In the meantime, I think it makes sense to update the README.

Edit: looks like #545 might make this irrelevant, let's see if that one can be merged soon.

@prusnak prusnak merged commit d8d4e86 into ggml-org:master Apr 2, 2023
Deadsg pushed a commit to Deadsg/llama.cpp that referenced this pull request Dec 19, 2023
…terial-9.3.1

Bump mkdocs-material from 9.2.8 to 9.3.1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants