-
Couldn't load subscription status.
- Fork 31k
[torchao] Add regex support for ModuleFqnToConfig #41242
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
33b49f2 to
225d5f3
Compare
|
cc @MekkCyber for quantization |
3fc0850 to
c82e4fa
Compare
Summary: Similar to pytorch/ao#3084 we added regex support in transformers so people can use regex to quantize the models. See pytorch/ao#3084 for docs and precedence of different configurations Uploaded model: https://huggingface.co/torchao-testing/opt-125m-ModuleFqnToConfig-v1-regex-0.14.0.dev Test Plan: pytest tests/quantization/torchao_integration/test_torchao.py -k test_module_fqn_to_config_regex Reviewers: Subscribers: Tasks: Tags:
c82e4fa to
dbc07a0
Compare
|
also cc @SunMarc please take a look |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Amazing feature ! Would be nice to add it to the docs for visibility
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice ! Please add some docs around that also. As we are moving to v5 in the next release, maybe we can bump torchao version to clean a but the code @jerryzh168 ?
yeah we can bump the torchao version, we are preparing for a 0.14 release, we can do it after that I think |
|
@bot /style |
|
Style bot fixed some files and pushed the changes. |
|
@bot /style |
da20f14 to
97598f6
Compare
97598f6 to
8111e5b
Compare
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
|
[For maintainers] Suggested jobs to run (before merge) run-slow: torchao_integration |
* Add regex support for ModuleFqnToConfig Summary: Similar to pytorch/ao#3084 we added regex support in transformers so people can use regex to quantize the models. See pytorch/ao#3084 for docs and precedence of different configurations Uploaded model: https://huggingface.co/torchao-testing/opt-125m-ModuleFqnToConfig-v1-regex-0.14.0.dev Test Plan: pytest tests/quantization/torchao_integration/test_torchao.py -k test_module_fqn_to_config_regex Reviewers: Subscribers: Tasks: Tags: * Apply style fixes * add assert for --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Mohamed Mekkouri <93391238+MekkCyber@users.noreply.github.com> Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com>
Summary:
Similar to pytorch/ao#3084 we added regex support in transformers so people can use regex to quantize the models.
See pytorch/ao#3084 for docs and precedence of different configurations
Uploaded model: https://huggingface.co/torchao-testing/opt-125m-ModuleFqnToConfig-v1-regex-0.14.0.dev
Test Plan:
pytest tests/quantization/torchao_integration/test_torchao.py -k test_module_fqn_to_config_regex
Reviewers:
Subscribers:
Tasks:
Tags: