Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for forward and reverse high-order automatic differentiation mechanism #41919

Merged
merged 277 commits into from
May 18, 2022
Merged

Add support for forward and reverse high-order automatic differentiation mechanism #41919

merged 277 commits into from
May 18, 2022

Conversation

0x45f
Copy link
Contributor

@0x45f 0x45f commented Apr 18, 2022 •
edited by levi131
Loading

PR types

New features

PR changes

Others

Describe

本pr配合pr #41201 一起构成一套新的自动微分机制
该自动微分机制目前只在静态图中生效，在一部分算子上，支持前向和反向的高阶自动微分。

本pr中包含的内容包括如下几个部分：
（1）在C++中注册的自动微分基础算子的python wrapper

（2）在自动微分基础算子之上定义的orig2prim，prim2orig，jvp，transpose规则

（3）在4种规则之上形成的orig2prim，prim2orig，linearize，transpose四种变换

       a、orig2prim paddle原生算子转化为自动微分基础算子
       b、prim2orig 自动微分基础算子转化为paddle原生算子
       c、linearize 根据前向计算计算图生成前向微分计算图
       d、transpose 根据前向微分计算图生成反向微分计算图

（4）提供出的用户接口

       a、沿用静态图中已经存在的gradients和minimize接口
       b、提供基于自动微分基础算子的自动微分机制切换的全局控制接口，enable_prim, disable_prim, prim_enabled
       c、将基础算子体系转化到原生算子体系以支撑执行的接口prim2orig

veyron95 and others added 30 commits September 28, 2021 08:54
@veyron95
Updated triple_grad_check func
04eab89
@JiabinYang
Merge pull request PaddlePaddle#9 from veyron95/ops_derivative
38ca20a 
Updated triple grad teses func
@JiabinYang
merge develop
886d9fb
@JiabinYang
add todo for gradient checker and refine some comments
e9f643d
@JiabinYang
remove additional code
2d6370b
@JiabinYang
add test for infer_var dtype warning
a3b8e4e
@JiabinYang
Merge branch 'support_derivative' of https://github.com/JiabinYang/Pa…
13af3ed 
…ddle into support_derivative
@JiabinYang
add test for warnging in backward.py
20ca8e7
@JiabinYang
format python code
a961e3c
@JiabinYang
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…
ee5489d 
… support_derivative
@JiabinYang
support multi input in triple gradient checker
a495960
@veyron95
Add matmul triple grad kernel
ebe8559
@veyron95
Merge branch 'support_derivative' of https://github.com/JiabinYang/Pa…
4f31159 
…ddle into ops_derivative
@veyron95
Updated comments of TODO
4d56a30
@veyron95
Merge develop branch and all conflicts fixed
15f2a32
@veyron95
Supported some special tests
07d1490
@JiabinYang
merge develop
d5fdd20
@veyron95
merge jiabin/support_derivative branch
0e44f39
@veyron95
Change code-format to follow CI std
b52794e
@veyron95
Updated gradient_checker.py
4202d96
@veyron95
Fix conflicts
91149a7
@veyron95
Merge develop and fix conflicts
e20ef17
@veyron95
Removed unnecessary printing log
d0741f4
@veyron95
Change code style to follow CI std
46dbd64
Merge remote-tracking branch '3rd_order/ops_derivative' into develop
e32e10e
Merge remote-tracking branch 'upstream/develop' into develop
46607df
Merge branch 'develop' of github.com:PaddlePaddle/Paddle into develop
9da53dd
Merge remote-tracking branch 'upstream/develop' into develop
528ef73
Merge remote-tracking branch 'upstream/develop' into develop
34116c1
Merge remote-tracking branch 'upstream/develop' into develop
affdf9f
TCChenlong
TCChenlong reviewed May 16, 2022
View reviewed changes
python/paddle/incubate/autograd/utils.py Show resolved Hide resolved
python/paddle/incubate/autograd/primx.py Show resolved Hide resolved
python/paddle/incubate/autograd/primx.py Outdated Show resolved Hide resolved
python/paddle/incubate/autograd/utils.py Outdated Show resolved Hide resolved
python/paddle/incubate/autograd/utils.py Outdated Show resolved Hide resolved
python/paddle/incubate/autograd/utils.py Outdated Show resolved Hide resolved
python/paddle/incubate/autograd/utils.py Outdated Show resolved Hide resolved
python/paddle/incubate/autograd/utils.py Show resolved Hide resolved
python/paddle/incubate/autograd/utils.py Outdated Show resolved Hide resolved
TCChenlong
TCChenlong reviewed May 16, 2022
View reviewed changes
python/paddle/incubate/autograd/primx.py Outdated Show resolved Hide resolved
python/paddle/incubate/autograd/utils.py Outdated Show resolved Hide resolved
@levi131
Copy link
Contributor

levi131 commented May 17, 2022

LGTM

TCChenlong
TCChenlong approved these changes May 17, 2022
View reviewed changes
Copy link
Contributor

@TCChenlong TCChenlong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

XieYunshen
XieYunshen approved these changes May 17, 2022
View reviewed changes
Copy link
Contributor

@XieYunshen XieYunshen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for set_tests_properties(test_gradients_and_minimize PROPERTIES TIMEOUT 60)

XiaoguangHu01
XiaoguangHu01 approved these changes May 18, 2022
View reviewed changes
Copy link
Contributor

@XiaoguangHu01 XiaoguangHu01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@levi131 levi131 merged commit f6ee202 into PaddlePaddle:develop May 18, 2022
@0x45f 0x45f deleted the science-add-rules-ut branch May 18, 2022 08:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cxxly cxxly cxxly left review comments

@levi131 levi131 levi131 approved these changes

@XiaoguangHu01 XiaoguangHu01 XiaoguangHu01 approved these changes

@XieYunshen XieYunshen XieYunshen approved these changes

@TCChenlong TCChenlong TCChenlong approved these changes

Assignees

@cxxly cxxly

@0x45f 0x45f

@levi131 levi131

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
9 participants
@0x45f @levi131 @cxxly @TCChenlong @XieYunshen @XiaoguangHu01 @veyron95 @JiabinYang @tongxin