Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add gguf support for StableLM #33793

Merged
merged 3 commits into from
Oct 9, 2024
Merged

Add gguf support for StableLM #33793

merged 3 commits into from
Oct 9, 2024

Conversation

VladOS95-cyber
Copy link
Contributor

@VladOS95-cyber VladOS95-cyber commented Sep 29, 2024 •
edited
Loading

What does this PR do?

Add GGUF support for StableLM

Before submitting

Who can review?

Regarding the task @SunMarc @LysandreJik @ArthurZucker .

@VladOS95-cyber VladOS95-cyber changed the title add stablelm gguf architecture support Add gguf support for StableLM Sep 29, 2024
@VladOS95-cyber
Copy link
Contributor Author

Hi @SunMarc! This PR is ready for review, please, take a look.

SunMarc
SunMarc reviewed Sep 30, 2024
View reviewed changes
Copy link
Member

@SunMarc SunMarc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding @VladOS95-cyber as always ! Left a few comments

src/transformers/models/gpt_neox/tokenization_gpt_neox_fast.py Show resolved Hide resolved
tests/quantization/ggml/test_ggml.py Show resolved Hide resolved
@VladOS95-cyber VladOS95-cyber force-pushed the add-GGUF-support-for-StableLM branch from c7b1540 to 9c5e33e Compare October 1, 2024 16:29
@HuggingFaceDocBuilderDev
Copy link

Hey! 🤗 Thanks for your contribution to the transformers library!

Before merging this pull request, slow tests CI should be triggered. To enable this:

  • Add the run-slow label to the PR
  • When your PR is ready for merge and all reviewers' comments have been addressed, push an empty commit with the command [run-slow] followed by a comma separated list of all the models to be tested, i.e. [run_slow] model_to_test_1, model_to_test_2
    • If the pull request affects a lot of models, put at most 10 models in the commit message
  • A transformers maintainer will then approve the workflow to start the tests

(For maintainers) The documentation for slow tests CI on PRs is here.

@VladOS95-cyber
Copy link
Contributor Author

Hey @SunMarc! I resolved all comments, you could take a look again

SunMarc
SunMarc reviewed Oct 2, 2024
View reviewed changes
Copy link
Member

@SunMarc SunMarc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for iterating ! Could you also add a test to check the weights of each layer for the fp16 model, to see if we get the same model ? We recently merged the gguf falcon model and he added a nice test for that. Also there are a few merge conflits, thanks !

@SunMarc SunMarc mentioned this pull request Oct 2, 2024
Open
15 tasks
@VladOS95-cyber VladOS95-cyber force-pushed the add-GGUF-support-for-StableLM branch from 9c5e33e to 1d13301 Compare October 2, 2024 12:52
@VladOS95-cyber
Copy link
Contributor Author

Thanks for iterating ! Could you also add a test to check the weights of each layer for the fp16 model, to see if we get the same model ? We recently merged the gguf falcon model and he added a nice test for that. Also there are a few merge conflits, thanks !

Hey @SunMarc! Sure, it is done

SunMarc
SunMarc approved these changes Oct 2, 2024
View reviewed changes
Copy link
Member

@SunMarc SunMarc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM ! Left a question to better understand

tests/quantization/ggml/test_ggml.py Show resolved Hide resolved
@SunMarc SunMarc requested a review from LysandreJik October 2, 2024 14:36
@HuggingFaceDocBuilderDev
Copy link

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@VladOS95-cyber
Copy link
Contributor Author

VladOS95-cyber commented Oct 9, 2024 •
edited
Loading

Hi @LysandreJik! Just a kind reminder to take a look on this PR!

LysandreJik
LysandreJik approved these changes Oct 9, 2024
View reviewed changes
Copy link
Member

@LysandreJik LysandreJik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @VladOS95-cyber, this seems to have passed through the net. Looks great to me!

@LysandreJik LysandreJik merged commit faa0f63 into huggingface:main Oct 9, 2024
24 checks passed
NielsRogge pushed a commit to NielsRogge/transformers that referenced this pull request Oct 21, 2024
@VladOS95-cyber @NielsRogge
Add gguf support for StableLM (huggingface#33793)
6417301 
* add stablelm gguf architecture support

* add additional quantization tests

* resolve merge conflict, add weight conversion tests for fp16
BernardZach pushed a commit to BernardZach/transformers that referenced this pull request Dec 5, 2024
@VladOS95-cyber @BernardZach
Add gguf support for StableLM (huggingface#33793)
a753808 
* add stablelm gguf architecture support

* add additional quantization tests

* resolve merge conflict, add weight conversion tests for fp16
BernardZach pushed a commit to innovationcore/transformers that referenced this pull request Dec 6, 2024
@VladOS95-cyber @BernardZach
Add gguf support for StableLM (huggingface#33793)
6830c90 
* add stablelm gguf architecture support

* add additional quantization tests

* resolve merge conflict, add weight conversion tests for fp16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@SunMarc SunMarc SunMarc approved these changes

@LysandreJik LysandreJik LysandreJik approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@VladOS95-cyber @HuggingFaceDocBuilderDev @LysandreJik @SunMarc