Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Fix validation errors for smoothquant modifier + update examples #19

Merged
merged 3 commits into from
Jul 8, 2024

Conversation

rahul-tuli
Copy link
Collaborator

This PR fixes a bug, where SmoothQuant modifier wasn't being passed in correctly due to validation errors, this PR fixes that

Why?
The typing of one of the attributes from SmoothQuant Modifier was misleading

Scope?
This only affected cases where smoothquant modifer was passed in as an instantiated object, (cases where it is defined in a string or as a yaml recipe remain unaffected)

Test plan?
Ran the w8a8_int8 example successfully on Xenova/llama2.c-stories15M

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
@robertgshaw2-neuralmagic robertgshaw2-neuralmagic merged commit ac118ca into main Jul 8, 2024
5 of 6 checks passed
@rahul-tuli rahul-tuli deleted the smoothquant-validation branch July 18, 2024 19:09
markmc pushed a commit to markmc/llm-compressor that referenced this pull request Nov 13, 2024

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
* test forward (vllm-project#16)

* test frozen (vllm-project#17)

* test frozen

* rename

* lifecycle conftest (vllm-project#21)

* test initalize (vllm-project#18)

* test initalize

* newline

* parametrize weights and inp_act

* remove dup

* test lifecycle (vllm-project#19)

* test lifecycle

* comments

* comments

* add quantization test

* Lifecycle/min max obs (vllm-project#20)

* min max test

* add minmax obs

* test scale range and min_max update

* rebase

* rebase

* fix

* fix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants