Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ UX ] Skip Running First Cycle Through Dataset for Weight-Only Quantization #29

Closed
robertgshaw2-neuralmagic opened this issue Jul 19, 2024 · 1 comment
Closed

[ UX ] Skip Running First Cycle Through Dataset for Weight-Only Quantization #29

robertgshaw2-neuralmagic opened this issue Jul 19, 2024 · 1 comment
Labels
enhancement New feature or request

Comments

@robertgshaw2-neuralmagic
Copy link
Collaborator

Currently, we run one cycle through the dataset for QuantizationModifier for weight-only quantization.

This ~doubles the runtime of a GPTQ run
@robertgshaw2-neuralmagic robertgshaw2-neuralmagic added the enhancement New feature or request label Jul 19, 2024
@Satrat
Copy link
Contributor

Satrat commented Aug 6, 2024

This was fixed as part of #34, closing this issue!

@Satrat Satrat closed this as completed Aug 6, 2024
markmc pushed a commit to markmc/llm-compressor that referenced this issue Nov 13, 2024
@dbogunowicz
Fix failing GHA (vllm-project#29)
06200fc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Satrat @robertgshaw2-neuralmagic