We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Currently the quantization info is hacked into the weights in many ways, result in not easily adding more features with quantization:
_set/get_gptq_params
So that we could simply binding the config info with weights, and inside weights getting the desired info directly.
A refactor for supporting quantization fetures.
Yes, I am prepared a PR.
The text was updated successfully, but these errors were encountered:
Successfully merging a pull request may close this issue.
Feature request
Currently the quantization info is hacked into the weights in many ways, result in not easily adding more features with quantization:
_set/get_gptq_params
, but which is already applied for awq and eetq, this is very confusing for maintaining the codeSo that we could simply binding the config info with weights, and inside weights getting the desired info directly.
Motivation
A refactor for supporting quantization fetures.
Your contribution
Yes, I am prepared a PR.
The text was updated successfully, but these errors were encountered: