-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bug: Dropping GGUF metadata when creating model.yaml
for Model Downloaded via URL
#3558
Labels
category: model hub
Built in models, latest HF models page
P0: critical
Mission critical
type: bug
Something isn't working
Milestone
Comments
Van-QA
changed the title
bug: [DESCRIPTION]
bug: Incorrect Settings for HF GGUF Model Downloaded via URL Import
Sep 5, 2024
@Van-QA what do you mean by "correct settings" There are a few settings:
|
hi @0xSage, currently, via importing from HF URL, Jan is only applying No.1 (default settings) without considering No.2 (author settings) |
0xSage
changed the title
bug: Incorrect Settings for HF GGUF Model Downloaded via URL Import
bug: Dropping GGUF metadata when creating Sep 6, 2024
model.yaml
for Model Downloaded via URL
4 tasks
imtuyethan
added
P0: critical
Mission critical
category: model running
category: model hub
Built in models, latest HF models page
and removed
P1: important
Important feature / fix
labels
Sep 18, 2024
This was referenced Oct 3, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
category: model hub
Built in models, latest HF models page
P0: critical
Mission critical
type: bug
Something isn't working
Describe the bug
When downloading the HF GGUF model via URL import from the model hub, the model is using default settings instead of the correct settings. The fixed context length is set to 2048, the prompt template is incorrect, and there is an issue with the stop word. However, importing GGUF directly seems to resolve these issues.
Steps to reproduce
Expected behavior
The HF GGUF model downloaded via URL import should have the correct settings for context length, prompt template, and stop word, matching the settings when GGUF is imported directly.
Additional context
The issue seems to be specific to downloading the HF GGUF model via URL import from the model hub. Importing GGUF directly does not exhibit the same issue with default settings.
Comparing model.json, the GGUF direct import in on the right
The text was updated successfully, but these errors were encountered: