This repository has been archived by the owner on Jun 24, 2024. It is now read-only.
Support latest (breaking) changes to GGML file format #61
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
RoyVorster
commented
Mar 23, 2023
|
Hi! Thanks a lot for the PR 😄
That's really strange. IIRC the only changes to the format are that now vocab scores are embedded in the model. Weights should be exactly the same. Can you make a test with |
setzer22
approved these changes
Mar 23, 2023
|
Yup, will doublecheck output vs. main branch on an older model to ensure all is okay. |
Breaking changes in ggml: - Scores added in vocabulary - Format version added - Magic updated
- Generalize u32, i32 and f32 reading without a boilerplate-y trait
Merged
|
Yeah looks good to me. @setzer22 can you approve the workflow? |
|
Merged! Took care of some minor clippy lints myself. Thanks again :) |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Latest ggml models have:
This PR maintains support for older ('legacy') models
A model downloaded and converted on latest master in
llama.cppnow works withllama-rsbut the output seems qualitatively worse. Haven't played around much withllama-rsthough so it's hard to say whether this is a regression.