Add QuantizeFusionLSTM pass and collect lstm scales #33797
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks for your contribution! |
lidanqing-intel
changed the title
lidanqing-intel
force-pushed
the
develop-lstm-quant
branch
from
bd0ba63
to
62308b3
Compare
wozna
reviewed
Jun 29, 2021
| @@ -2251,7 +2251,8 @@ PDNode *patterns::QuantizePlacement::operator()( | |||
| std::unordered_set<std::string> supported_op_types = | |||
| std::unordered_set<std::string>( | |||
| {"concat", "conv2d", "elementwise_add", "fc", "matmul", "pool2d", | |||
| "prior_box", "relu", "reshape2", "transpose2", "fusion_gru"}); | |||
| "prior_box", "relu", "reshape2", "transpose2", "fusion_gru", | |||
| "multi_gru", "fusion_lstm"}); | |||
There was a problem hiding this comment.
For now, multi_gru is supported in QAT, but it is not in PTQ (it'll be added #33749). As well as, here you add support for fusion_lstm for PTQ but not for QAT. I think that we should separate these lists because it is very illegible and I think it might cause bugs.
Maybe in quant2_int8_mkldnn_pass.py we should define default quant2 list by setting quantize_enabled_op_types to cpu_quantize_placement_pass.
There was a problem hiding this comment.
Hi
- now I only add fusion_lstm now, I removed multi_gru
- fusion_lstm here is used for QAT, I m not sure if it is for PTQ. But QAT also need this pattern detection too. I need to detect the pattern and then substitue with fusion_lstm mkldnn kernel.
|
Sorry to inform you that 2a6feca's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually. |
lidanqing-intel
changed the title
lidanqing-intel
force-pushed
the
develop-lstm-quant
branch
from
2a6feca
to
4ded008
Compare
lidanqing-intel
force-pushed
the
develop-lstm-quant
branch
from
3bef8d5
to
2464c72
Compare
lidanqing-intel
force-pushed
the
develop-lstm-quant
branch
from
2464c72
to
def2632
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
New features
PR changes
Others
Describe
Add QuantizeFusionLSTM pass and add LSTM scales