Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dequantize matmul and matmul_v2 Y weights in quant2_int8 #37618

Merged
merged 8 commits into from
Dec 1, 2021
Merged

dequantize matmul and matmul_v2 Y weights in quant2_int8 #37618

Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
e9b4bb2
dequantize matmul and matmul_v2 Y weights in qat2_int8
sfraczek Nov 26, 2021
c603778
review fix
sfraczek Nov 29, 2021
1d077f5
split conv and mul tests, add matmul test
sfraczek Nov 29, 2021
a11eb6a
fixup
sfraczek Nov 29, 2021
beaa1dc
fix ci build
sfraczek Nov 29, 2021
cc40577
remove unused variables
sfraczek Nov 30, 2021
41d1a6d
formatting fix
sfraczek Nov 30, 2021
02c054e
remove extra newline at end of file
sfraczek Dec 1, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion python/paddle/fluid/contrib/slim/quantization/quant2_int8_mkldnn_pass.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ def __init__(self,
self._mul_ops = ['mul']
self._fc_ops = ['fc']
self._relu_ops = ['relu', 'relu6']
self._matmul_ops = ['matmul']
self._matmul_ops = ['matmul', 'matmul_v2']
self._gru_ops = ['fusion_gru', 'multi_gru']
self._lstm_ops = ['fusion_lstm']
self._weight_thresholds = {}
Expand Down Expand Up @@ -328,6 +328,8 @@ def _swap_inputs(self, op, old_input, new_input):
def _dequantize_weights(self, graph):
def _is_int8_weights(op_node, weight_name):
weight_var_name = op_node.input(weight_name)[0]
if self._scope.find_var(weight_var_name) is None:
return False
weight = self._load_param(self._scope, weight_var_name)
return np.all(np.mod(weight, 1) == 0)

Expand All @@ -336,6 +338,9 @@ def _is_int8_weights(op_node, weight_name):
self._dequantize_op_weights(graph, op, "Filter", "Output")
elif op.name() in self._mul_ops and _is_int8_weights(op, "Y"):
self._dequantize_op_weights(graph, op, "Y", "Out")
elif op.name() in self._matmul_ops and _is_int8_weights(op, "Y"):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like you can combine this part into one and just check op.name() in [self._mul_ops, self._matmul_ops]. What do you think?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can apply your idea, but I will concatenate those lists before for loop

Copy link
Contributor Author

@sfraczek sfraczek Nov 29, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we even need matmul_ops variable separate from mul_ops

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, it looks like we are doing exactly the same things for them, so for me you can combine it.

self._dequantize_op_weights(graph, op, "Y", "Out")

return graph

def _dequantize_op_weights(self, graph, op_node, weight_name, output_name):
Expand Down