Don't constant fold Quantize/DequantizeLinear nodes by default #2713
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I added support for exporting
QuantizeLinear/DequantizeLinearnodes (fromfake_quantize_per_*_affinetorch operators) in a previous PR.Unfortunately, the current default onnxscript optimizer settings tend to automatically remove any weight quantization. This is because the
Weight -> QDQ -> ...pattern looks like it can be just constant folded toQDQ(Weight) -> ....I believe that this behavior is not desirable, since the presence of
QDQnodes in the graph is what allows inference engines to run the supported computations using quantized data types. So the purpose ofQDQnodes is to hold the relevant quantization "metadata". As such, they normally shouldn't be constant folded.I have extended the existing logic in
FoldConstantsPassthat was used to excludeConstantOfShapefrom constant folding.I haven't found any tests verifying this behavior for
ConstantOfShapeand I'm not sure, how to set up such a unit test, so I have left this code untested for now. If adding tests is mandatory, please give me a hint on where should I add such a test and what would be the best way to check/assert that the optimized graph matches the expectations (hopefully without reinventing the wheel or manually introspecting their.Modelobject).