move dgc_momentum kernel to phi #56158
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
huangjiyi
commented
Aug 10, 2023
Comment on lines
-174
to
-175
| } else if (param_var->IsType<phi::SelectedRows>() && | ||
| grad_var->IsType<phi::SelectedRows>() && |
There was a problem hiding this comment.
@GhostScreaming，帮忙看一下这种情况怎么处理，输入参数 param 和 grad 可能是 DenseTensor 也可能是 SelectedRows，但是改造成函数式 Kernel 后需要在参数列表中明确类型，我这个 PR 里只保留了 DenseTensor 的情况，本地跑单测也能通过，如果还需要保留 SelectedRows 的情况的话，需要像 sgd_kernel.cc 一样实现并注册多个 Kernel，但是我看 sgd_kernel.cc 每一个 Kernel 都有对应的 op，而 dgc_momentum 只有一个 op，所以不太清楚要怎么处理
There was a problem hiding this comment.
SelectedRows可以不用再适配。python侧的DGCMomentumOptimizer在param和grad类型是SelectedRows，会调用momentum而不是dgc_momentum
GhostScreaming
approved these changes
Aug 16, 2023
Comment on lines
-174
to
-175
| } else if (param_var->IsType<phi::SelectedRows>() && | ||
| grad_var->IsType<phi::SelectedRows>() && |
There was a problem hiding this comment.
SelectedRows可以不用再适配。python侧的DGCMomentumOptimizer在param和grad类型是SelectedRows，会调用momentum而不是dgc_momentum
Get. |
|
@luotao1 , 帮忙豁免一下 CI ~ |
luotao1
approved these changes
Aug 16, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
Others
PR changes
Others
Description