[Custom operator] Fix custom operator backward=None bug #48656
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
Bug fixes
PR changes
OPs
Describe
Bug describes: when custom backward op outputs gradient with a discrete order, the output of some gradient=None.
For example: backward op's input = {a, b, c, d, e}, output = {a@GRAD, d@GRAD, e@GRAD}, the output of d@GRAD and e@GRAD will be None.
Reason: The order of
RunCustomOpNode::operator(): line220
incustom_operator_node.cc
is {a@GRAD, null, null, d@GRAD, e@GRAD}. However, the order ofeager_api_run_custom_op: line560
ineager_functions.cc
is {a@GRAD, d@GRAD, e@GRAD, null, null}.Change: Change the order in
RunCustomOpNode::operator()
to {a@GRAD, d@GRAD, e@GRAD, null, null}本 PR 修复了自定义算子反向 op 的输出值可能为 None 的 bug,当用户的输出梯度不是连续输出时(例如输出第0、3、4个前向输入的梯度时),第3、4个梯度会被置为 None,这是由于对反向 op 的输出梯度进行处理时,
RunCustomOpNode::operator(): line220
和eager_api_run_custom_op: line560
的动作不一致。更改为按照RunCustomOpNode::operator()
对齐,在反向 op 的输出梯度后面,填充不需要梯度的输出,修改后的返回值为 {第 0 个梯度,第 3 个梯度,第 4 个梯度,null, null}