Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove quantization related config from dequantized model #34856

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Remove quantization related config from dequantized model #34856

wants to merge 2 commits into from
+3 −0

Conversation

konradkalita
Copy link

What does this PR do?

Fixes #34847

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline,
    Pull Request section?
  • Was this discussed/approved via a Github issue or the forum? Please add a link
    to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the
    documentation guidelines, and
    here are tips on formatting docstrings.
  • Did you write any new necessary tests?

Who can review?

@SunMarc

konradkalita
konradkalita commented Nov 21, 2024
View reviewed changes
src/transformers/quantizers/base.py
Comment on lines +218 to +220
del model.config.quantization_config
del model.config._pre_quantization_dtype
model.is_quantized = False
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quantizer seems better place for it than PreTrainedModel.

Not sure if that's 100% exhaustive, but it was enough to solve my issue.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, sounds good to me !

SunMarc
SunMarc approved these changes Nov 21, 2024
View reviewed changes
Copy link
Member

@SunMarc SunMarc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice ! Could you add a test in the bnb tests to check if it works as expected ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@SunMarc SunMarc SunMarc approved these changes

@ArthurZucker ArthurZucker Awaiting requested review from ArthurZucker

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@konradkalita @SunMarc