-
Notifications
You must be signed in to change notification settings - Fork 424
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support importing GGUF files #1187
Comments
If gguf contains the model graph information, then we can use what burn-import ONNX facility. In our burn-import, we convert ONNX graph to IR (intermediate representation) (see this doc). So, it would possible to convert the model graph to IR and generate source code + weights. If gguf contains only weights, we can go burn-import pytorch route, where we only download weights. |
From my brief research, GGUF format contains metadata + tensor weights. This aligns with burn-import pytorch route and not burn-import/ONNX. This will mean model needs to be constructed in Burn first and use the weights to load. Here is one Rust lib to parse GGUF file: https://github.com/Jimexist/gguf |
GGUF spec: ggerganov/ggml#302 |
Parser in Rust: https://github.com/Jimexist/gguf |
I apologize if this seems too far fetched, but it seemed in line with how ONNX generation works.
The text was updated successfully, but these errors were encountered: