Skip to content

Commit f58f8b5

Browse files
authored
Update AutoAWQ docs (#14042)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
1 parent b3f7aac commit f58f8b5

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

docs/source/features/quantization/auto_awq.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,13 +6,13 @@ To create a new 4-bit quantized model, you can leverage [AutoAWQ](https://github
66
Quantizing reduces the model's precision from FP16 to INT4 which effectively reduces the file size by ~70%.
77
The main benefits are lower latency and memory usage.
88

9-
You can quantize your own models by installing AutoAWQ or picking one of the [400+ models on Huggingface](https://huggingface.co/models?sort=trending&search=awq).
9+
You can quantize your own models by installing AutoAWQ or picking one of the [6500+ models on Huggingface](https://huggingface.co/models?sort=trending&search=awq).
1010

1111
```console
1212
pip install autoawq
1313
```
1414

15-
After installing AutoAWQ, you are ready to quantize a model. Here is an example of how to quantize `mistralai/Mistral-7B-Instruct-v0.2`:
15+
After installing AutoAWQ, you are ready to quantize a model. Please refer to the `AutoAWQ documentation <https://casper-hansen.github.io/AutoAWQ/examples/#basic-quantization>`_ for further details. Here is an example of how to quantize `mistralai/Mistral-7B-Instruct-v0.2`:
1616

1717
```python
1818
from awq import AutoAWQForCausalLM

0 commit comments

Comments
 (0)