[Relay][Quantize] Integrate data-aware calibration into quantization #4295
Conversation
vinx13
force-pushed
the
feature/quanti_eval
branch
4 times, most recently
from
d8c7679 to
f60b25d
Compare
vinx13
force-pushed
the
feature/quanti_eval
branch
from
f60b25d to
4050a78
Compare
vinx13
force-pushed
the
feature/quanti_eval
branch
from
1729eb1 to
6d3bd3b
Compare
|
It's difficult to do unit tests. The refactor is covered by nightly tests. I plan to add more nightly tests for the new calibration mode later because there are some more work to do |
|
@yzhliu comments addressed |
| @@ -54,6 +54,8 @@ def kl_divergence_scale(arr, quantized_dtype='int8', num_bins=8001, num_quantize | |||
| http://on-demand.gputechconf.com/gtc/2017/presentation/s7310-8-bit-inference-with-tensorrt.pdf | |||
| """ | |||
| assert isinstance(arr, np.ndarray) | |||
| assert stats is not None, "scipy need to be installed for \ | |||
| @@ -143,9 +141,20 @@ def qconfig(**kwargs): | |||
| nbit_dict: dict of QAnnotateKind -> int | |||
| Number of bit for every kind of annotate field. | |||
|
|
|||
| calibrate_mode: str | |||
| The calibration mode. 'global_scale' or 'kl'. | |||
There was a problem hiding this comment.
can we spell it out so it's more self explanatory (e.g. kullback_leibler)?
|
@vinx13 I realize that this PR might have broken the E2E VTA tutorial flow (that includes quantization for VTA hardware pipeline). I'll need to investigate, but what puzzles me more is why that wasn't caught by the CI which should be building the sphinx galleries successfully, which should include the e2e VTA tutorial. |
|
The script in question is |
…pache#4295) * [Relay][Quantize] Integrate data-aware calibration into quantization * Update _calibrate.py * trigger ci * Address comments * address comments
…pache#4295) * [Relay][Quantize] Integrate data-aware calibration into quantization * Update _calibrate.py * trigger ci * Address comments * address comments
|
Fix for the VTA bug introduced is in #4433. |
This PR did some refactor work for the calibration part: integrate evaluation script for KL, the collect stats part has been moved into internal collect_stats. New config options
calibration_mode,weight_scalehave been added.Removed
opt_level=3forprerequisite_optimize, as it caused some accuracy issue whenFoldScaleAxisis invoked before calibration.Part of this PR is based on #3828.
@ZihengJiang @anijain2305 @tmoreau89