Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DO NOT MERGE] Hf quantizer refactor #28703

Closed
wants to merge 78 commits into from
Closed

[DO NOT MERGE] Hf quantizer refactor #28703

wants to merge 78 commits into from

Conversation

younesbelkada
Copy link
Contributor

What does this PR do?

Built on top of #26610 - this PR is just to see if I don't any surprising diff similar as in poedator#4

poedator added 30 commits January 16, 2024 13:10
@poedator
squashed earlier commits for easier rebase
e0650b2
@poedator
rm rebase leftovers
42adf9d
@poedator
4bit save enabled @quantizers
7f57f26
@poedator
TMP gptq test use exllama
f1f5da0
@poedator
fix AwqConfigTest::test_wrong_backend for A100
a94d3a7
@poedator
quantizers AWQ fixes
0b30de4
@poedator
_load_pretrained_model low_cpu_mem_usage branch
4cdaf0d
@poedator
quantizers style
0db1107
@poedator
remove require_low_cpu_mem_usage attr
89d1177
@poedator
rm dtype arg from process_model_before_weight_loading
0c71b00
@poedator
rm config_origin from Q-config
2b4122a
@poedator
rm inspect from q_config
02ad562
@poedator
fixed docstrings in QuantizationConfigParser
3e51d51
@poedator
logger.warning fix
2569367
@poedator
mv is_loaded_in_4(8)bit to BnbHFQuantizer
3259243
@poedator
is_accelerate_available error msg fix in quantizer
ab61417
@poedator
split is_model_trainable in bnb quantizer class
95e44cd
@poedator
rm llm_int8_skip_modules as separate var in Q
b936cfb
@poedator
Q rm todo
0b40d21
@poedator
fwd ref to HFQuantizer in type hint
c53a3fb
@poedator
rm note re optimum.gptq.GPTQQuantizer
dbd93f2
@poedator
quantization_config in __init__ simplified
e34bd58
@poedator
replaced NonImplemented with create_quantized_param
fcd5a7a
@poedator
rm load_in_4/8_bit deprecation warning
954c5e6
@poedator
QuantizationConfigParser refactoring
49e163f
@poedator
awq-related minor changes
f8b9e07
@poedator
awq-related changes
5eaf9ac
@poedator
awq config.modules_to_not_convert
d678d99
@poedator
raise error if no q-method in q-config in args
7c9c49b
@poedator
minor cleanup
0d739d3
@HuggingFaceDocBuilderDev
Copy link

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@younesbelkada
Copy link
Contributor Author

Thanks @poedator for your comments and ideas - I think the way forward would be to extend the xxxQuantizer by adding new arguments in the corresponding quantization config object - e.g. quantize_mlp_only . I feel mixing different quantization approaches (your second point) might be a bit too much of an edge case but contributors can always create a new quantizer for it MixedQuantizer with a MixedQuantizationConfig.

younesbelkada and others added 5 commits January 26, 2024 15:10
@younesbelkada @ArthurZucker
Update src/transformers/modeling_utils.py
48c5761 
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
@younesbelkada @ArthurZucker
Update src/transformers/modeling_utils.py
3744fb1 
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
@younesbelkada
address final comment
c4995ab
@younesbelkada
Merge branch 'hf-quantizer-work' of https://github.com/younesbelkada/…
17f95bf 
…transformers into hf-quantizer-work
@younesbelkada
update
abb4db3
ArthurZucker
ArthurZucker approved these changes Jan 26, 2024
View reviewed changes
Copy link
Collaborator

@ArthurZucker ArthurZucker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔥 Great work.
Modeling utils refactor looks really great. My last comments are more around how we handle kwargs and getting as simple as possible with the way we init the quantizer, whether or not we actually need a AutoQuantizationConfig vs just a AutoHfQuantizer but otherwise good to go! 🤗

src/transformers/modeling_utils.py Show resolved Hide resolved
src/transformers/modeling_utils.py Show resolved Hide resolved
src/transformers/quantizers/auto.py Outdated Show resolved Hide resolved
src/transformers/quantizers/auto.py Outdated Show resolved Hide resolved
src/transformers/quantizers/base.py Show resolved Hide resolved
src/transformers/quantizers/quantizer_awq.py Show resolved Hide resolved
src/transformers/quantizers/quantizer_bnb_4bit.py Show resolved Hide resolved
younesbelkada and others added 3 commits January 26, 2024 19:19
@younesbelkada @ArthurZucker
Update src/transformers/quantizers/base.py
7e5a5b8 
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
@younesbelkada @ArthurZucker
Update src/transformers/quantizers/auto.py
122b494 
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
@younesbelkada
fix
901ace5
@younesbelkada younesbelkada mentioned this pull request Jan 29, 2024
Closed
@younesbelkada
Copy link
Contributor Author

I will merge the commits of this PR directly in #26610 to properly credit @poedator from his great work ! closing this - thanks @ArthurZucker for the review and offline discussions!

@younesbelkada younesbelkada mentioned this pull request Jan 30, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@stevhliu stevhliu stevhliu left review comments

@poedator poedator poedator left review comments

@ArthurZucker ArthurZucker ArthurZucker approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@younesbelkada @poedator @HuggingFaceDocBuilderDev @ArthurZucker @stevhliu