-
Notifications
You must be signed in to change notification settings - Fork 530
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added QARepVGGBlock #562
Added QARepVGGBlock #562
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This PR breaks the logic we have built for quantization.
instead of having a block and registering a quantized block, we define here a block that knows how to be quantized. I don't like this and I am not sure exactly how that will work with the rest of the quantization code.
Instead, lets define a block without the quantization part and (no if qat_mode
) and override some logics in a second block that is registered as his Quantized version
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
waiting for fixes
…/super-gradients into feature/implement-qarepvgg-block
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
only one minor comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
QARepVGG (S4) block from 'Make RepVGG Greater Again: A Quantization-aware Approach' (https://arxiv.org/pdf/2212.01593.pdf)
It consists of three branches:
3x3: a branch of a 3x3 Convolution + BatchNorm
1x1: a branch of a 1x1 Convolution with bias
identity: a Residual branch
BatchNorm is applied after summation of all three branches.
In contrast to our implementation of RepVGGBlock, SE is applied AFTER NONLINEARITY in order to fuse Conv+Act in inference frameworks.
This module converts to Conv+Act in a PTQ-friendly way by calling QARepVGGBlock.fuse_block_residual_branches().
Has the same API as RepVGGBlock and is designed to be a plug-and-play replacement but is not compatible parameter-wise.
Has less trainable parameters than RepVGGBlock because it has only 2 BatchNorms instead of 3.