Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add quantize #256

Merged
merged 3 commits into from
May 24, 2024
Merged

Add quantize #256

merged 3 commits into from
May 24, 2024

Commits on May 24, 2024

  1. Add WeightQuantizer and DynamicActQuantizer 

    Summary:
This exposes the AffineQuantizedTensor and LinearActQuantizedTensor
subclass as a model level API that will replace the weights of linear layers
This is in preparation to replace existing tensor subclass APIs such as `change_linear_weights_to_int4_woqtensors`
but currently we can't combine the two quantizers due to some problem with parametrization/nn.Parameter
the error is:

raise KeyError(f"attribute '{name}' already exists")
KeyError: "attribute 'weight' already exists"

happens in
```
lin.weight = torch.nn.Parameter(constructor(lin.weight, **copied_kwargs), requires_grad=False)
```

Test Plan:
regression tests:
```
python test/quantization/test_quant_api.py
```

Reviewers:

Subscribers:

Tasks:

Tags:
    @jerryzh168
    jerryzh168 committed May 24, 2024
    Configuration menu
    Copy the full SHA
    ac57a0c View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into add-api

    @jerryzh168
    jerryzh168 authored May 24, 2024
    Configuration menu
    Copy the full SHA
    899ed62 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into add-api

    @jerryzh168
    jerryzh168 authored May 24, 2024
    Configuration menu
    Copy the full SHA
    82ec155 View commit details
    Browse the repository at this point in the history