Skip to content

Commit

Permalink
DOC: add pre-processing stage for quantization (#68)
Browse files Browse the repository at this point in the history
  • Loading branch information
hagenw authored Oct 11, 2023
1 parent e796811 commit d1dfbf0
Show file tree
Hide file tree
Showing 2 changed files with 10 additions and 3 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,5 +54,5 @@ jobs:

- name: Test building documentation
run: |
python -m sphinx docs/ docs/_build/ -b html
python -m sphinx docs/ docs/_build/ -b html -W
if: matrix.os == 'ubuntu-20.04'
11 changes: 9 additions & 2 deletions docs/usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -251,7 +251,8 @@ Quantize weights
----------------

To reduce the memory print of a model,
we can quantize it.
we can quantize it,
compare the `MobilenetV2 example`_.
For instance, we can store model weights as 8 bit integers.
For quantization make sure
you have installed
Expand All @@ -264,9 +265,14 @@ as well as
import onnxruntime.quantization


onnx_infer_path = os.path.join(onnx_root, 'model_infer.onnx')
onnxruntime.quantization.quant_pre_process(
onnx_model_path,
onnx_infer_path,
)
onnx_quant_path = os.path.join(onnx_root, 'model_quant.onnx')
onnxruntime.quantization.quantize_dynamic(
onnx_model_path,
onnx_infer_path,
onnx_quant_path,
weight_type=onnxruntime.quantization.QuantType.QUInt8,
)
Expand Down Expand Up @@ -508,6 +514,7 @@ In that case do:
.. _audinterface: http://tools.pp.audeering.com/audinterface/
.. _audobject: http://tools.pp.audeering.com/audobject/
.. _librosa: https://librosa.org/doc/main/index.html
.. _MobilenetV2 example: https://github.com/microsoft/onnxruntime-inference-examples/blob/main/quantization/image_classification/cpu/ReadMe.md#onnx-runtime-quantization-example
.. _ONNX: https://onnx.ai/
.. _OpenSMILE: https://github.com/audeering/opensmile-python
.. _table: https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements
Expand Down

0 comments on commit d1dfbf0

Please sign in to comment.