Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] OPT model and fc1/fc2 modules #117

Closed
Qubitium opened this issue Jun 29, 2024 · 0 comments · Fixed by #118
Closed

[BUG] OPT model and fc1/fc2 modules #117

Qubitium opened this issue Jun 29, 2024 · 0 comments · Fixed by #118
Assignees
@Qubitium
Labels
bug Something isn't working

Comments

@Qubitium
Copy link
Contributor

From our testing, OPT model has suboptimal model config where fc1/fc2 modules should not be quantized as they are not a good candidate for quantization due to massive losses during calibration.

Investigate and confirm.
@Qubitium Qubitium added the bug Something isn't working label Jun 29, 2024
@Qubitium Qubitium self-assigned this Jun 29, 2024
@Qubitium Qubitium mentioned this issue Jun 29, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Qubitium Qubitium

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[FIX] opt fc1/fc2 layer modules should not be quantized ModelCloud/GPTQModel
1 participant
@Qubitium