【NewIR】add add_n_grad and split_with_num_grad (Split op ) #56873

xiaoguoguo626807 · 2023-09-01T06:15:40Z

PR types

Bug fixes

PR changes

others

Description

pcard-67164
基于#57218 pr
1.手写add-n反向相关代码，add_n 反向没有op,在api层直接处理；add_ninfermeta 没有设置输出类型；add_n 手写增加stop_gradient 推导。
2.split_with_num反向没有op,复用split_grad, api层手写直接调用split_grad op；
3.开启test_sum_op add_n 全部optest;
add_n _optest 修复：

splitbf16修复：

… develop

… split_op_genbug

paddle-bot · 2023-09-01T06:15:51Z

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

… split_op_genbug

…ub.com/PaddlePaddle/Paddle into split_op_genbug

… dev/refine_mutable_attr_split

…ub.com/PaddlePaddle/Paddle into split_op_genbug

… split_op_genbug

… develop

YuanRisheng · 2023-09-18T03:21:59Z

paddle/fluid/pir/dialect/operator/ir/manual_api.cc

@@ -28,6 +28,15 @@ pir::OpResult builtin_combine(const std::vector<pir::Value>& x) {
  return combine_op.out();
 }

+std::vector<pir::OpResult> add_n_grad(std::vector<pir::OpResult> inputs,
+                                      pir::OpResult out_grad) {


pir::OpResult -> pir::Value

0x45f · 2023-09-18T03:45:29Z

paddle/fluid/pir/dialect/operator/ir/manual_api.cc

@@ -76,5 +85,23 @@ pir::OpResult embedding_grad(pir::Value x,
  }
 }

+pir::OpResult split_with_num_grad(std::vector<pir::Value> out_grad, int axis) {


这里为什么需要手写一个split_with_num_grad的api呢？另外yaml里split_with_num_grad invoke的是concat，为什么不去调用concat op呢

对齐split_grad, vjp 需要调用该api, invoke的复用在build函数中手写，由于有split_grad op 不需要加split_with_num

lanxianghit

LGTM for 'check_new_ir' in test

… split_op_genbug

Aurelius84

LGTM, need fix commnet in next PR

Aurelius84 · 2023-09-19T09:33:35Z

paddle/fluid/pir/dialect/operator/ir/manual_api.cc

+  return split_grad_op.result(0);
+}
+
+pir::OpResult split_with_num_grad(std::vector<pir::Value> out_grad,


Suggested change

pir::OpResult split_with_num_grad(std::vector<pir::Value> out_grad,

pir::OpResult split_with_num_grad(std::vector<pir::Value>& out_grad,

虽然 pir::Value 的构造成本很低，但这里还是建议传入vector<>&，如有必要，也要加上const 限定符

这个在vjp处处理直接调用了invoke的api, 如果没有其他直接调用api 的情况此处后续会删除

Aurelius84 · 2023-09-19T09:35:11Z

paddle/fluid/pir/dialect/operator/ir/manual_api.h

@@ -25,6 +25,9 @@ namespace dialect {

 pir::OpResult builtin_combine(const std::vector<pir::Value>& x);

+std::vector<pir::OpResult> add_n_grad(std::vector<pir::Value> inputs,


结合代码生成同时修改

Aurelius84 · 2023-09-19T09:37:12Z

paddle/fluid/pir/dialect/operator/ir/manual_op_vjp.cc

+  pir::CombineOp combine_op_obj = op_obj.inputs()
+                                      .dyn_cast<pir::OpResult>()
+                                      .owner()
+                                      ->dyn_cast<pir::CombineOp>();


对于dyn_cast的使用，建议要check下，因为可能会有空指针导致段错误，导致问题排查比较难

下个pr修复

Aurelius84 · 2023-09-19T09:38:14Z

paddle/fluid/primitive/backend/manual/manual_backend.h

@@ -18,6 +18,7 @@
 #include <vector>

 #include "paddle/phi/api/include/tensor.h"
+#include "paddle/utils/optional.h"


这里为什么要include optional头文件，新增代码里并没有用到？

下个pr删除

xiaoguoguo626807

comment 修复 #57520

…le#56873) * add reference of lbfgs * add reference of lbfgs * tmp * split gen modify * fix conflict * add split * fix bug * fix bug * test split * add meta tensor * refine code * fix bug * fix bug * fix comflict * Call _C_ops.sum in new ir * modify concat kernel choose * modify ci * modify sum zero_dim optest * modify split_with_num api * modify split -1 * modify split test * fix bug * xxx * delete extra modify * add add_n * tmp * add split_with_num_grad * modify split grad num bug * modify ci * modify ci * clear code * modify * recover * add add_n stop_gradient infer * modify opreslut to value * fix conflict * recover to aviod conflict * recover to aviod conflict * modify opreslut to value * recover complex tanh * modify add_n optest * skip bfp16 * modify split bf16 * fix conflict * delete print --------- Co-authored-by: zhangbo9674 <zhangbo54@baidu.com> Co-authored-by: 0x45f <wangzhen45@baidu.com>

xiaoguoguo626807 added 11 commits August 11, 2023 07:40

add reference of lbfgs

2c0166c

add reference of lbfgs

37883b2

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

185d30b

… develop

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

92e5303

… develop

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

221b70c

… develop

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

aba6f0e

… develop

tmp

bce9b3b

fix conflict

c2341a5

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

4d30fdd

… develop

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

c8f5864

… split_op_genbug

split gen modify

805ceb2

xiaoguoguo626807 and others added 18 commits September 4, 2023 01:21

fix conflict

008a8b2

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

c619c4e

… split_op_genbug

add split

23b37ed

Merge branch 'develop', commit 'refs/pull/56924/head' of https://gith…

9640576

…ub.com/PaddlePaddle/Paddle into split_op_genbug

fix bug

000a8db

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

03ec67b

… dev/refine_mutable_attr_split

fix bug

06def9f

test split

d9f20d0

Merge branch 'develop', commit 'refs/pull/56924/head' of https://gith…

c027bbf

…ub.com/PaddlePaddle/Paddle into split_op_genbug

add meta tensor

9df5940

refine code

3924ce3

fix bug

d3805c7

fix bug

4405f3c

fix comflict

a2fa7be

Merge branch 'develop', commit 'refs/pull/56973/head' of https://gith…

fa23c5d

…ub.com/PaddlePaddle/Paddle into split_op_genbug

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

7273556

… split_op_genbug

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

24115d0

… develop

Call _C_ops.sum in new ir

b8e98c5

xiaoguoguo626807 added 7 commits September 18, 2023 01:14

recover

6612f3a

add add_n stop_gradient infer

0882578

modify opreslut to value

cf910ac

modify opreslut to value

17ad8a6

fix conflict

8f0bd46

recover to aviod conflict

825fea4

recover to aviod conflict

f5fc3a4

YuanRisheng reviewed Sep 18, 2023

View reviewed changes

modify opreslut to value

be2ffec

0x45f reviewed Sep 18, 2023

View reviewed changes

xiaoguoguo626807 added 2 commits September 18, 2023 06:45

recover complex tanh

a6e1ff2

modify add_n optest

8703219

0x45f mentioned this pull request Sep 18, 2023

[PIR]Migrate dropout into pir #57319

Merged

xiaoguoguo626807 added 2 commits September 18, 2023 11:15

skip bfp16

cf64265

modify split bf16

e4e57d4

lanxianghit previously approved these changes Sep 19, 2023

View reviewed changes

xiaoguoguo626807 added 2 commits September 19, 2023 03:12

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

05c4023

… split_op_genbug

fix conflict

df0dd36

xiaoguoguo626807 dismissed lanxianghit’s stale review via df0dd36 September 19, 2023 03:20

delete print

eaab743

lanxianghit approved these changes Sep 19, 2023

View reviewed changes

changeyoung98 approved these changes Sep 19, 2023

View reviewed changes

xiaoguoguo626807 merged commit 0029a24 into PaddlePaddle:develop Sep 19, 2023
27 checks passed

Aurelius84 approved these changes Sep 19, 2023

View reviewed changes

xiaoguoguo626807 commented Sep 20, 2023

View reviewed changes

xiaoguoguo626807 deleted the split_op_genbug branch September 20, 2023 03:19

YuanRisheng mentioned this pull request Mar 11, 2024

新IR Python API适配升级（第三期） #62618

Closed

risemeup1 mentioned this pull request May 21, 2024

【PIR】Sparse 算子 Python API 适配 #64492

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

【NewIR】add add_n_grad and split_with_num_grad (Split op ) #56873

【NewIR】add add_n_grad and split_with_num_grad (Split op ) #56873

xiaoguoguo626807 commented Sep 1, 2023 •

edited

Loading

paddle-bot bot commented Sep 1, 2023

YuanRisheng Sep 18, 2023

xiaoguoguo626807 Sep 18, 2023

0x45f Sep 18, 2023

xiaoguoguo626807 Sep 18, 2023

lanxianghit left a comment

Aurelius84 left a comment

Aurelius84 Sep 19, 2023

xiaoguoguo626807 Sep 19, 2023

Aurelius84 Sep 19, 2023

xiaoguoguo626807 Sep 19, 2023

Aurelius84 Sep 19, 2023

xiaoguoguo626807 Sep 19, 2023

Aurelius84 Sep 19, 2023

xiaoguoguo626807 Sep 19, 2023

xiaoguoguo626807 left a comment

	pir::OpResult split_with_num_grad(std::vector<pir::Value> out_grad,
	pir::OpResult split_with_num_grad(std::vector<pir::Value>& out_grad,

		@@ -25,6 +25,9 @@ namespace dialect {

		pir::OpResult builtin_combine(const std::vector<pir::Value>& x);

		std::vector<pir::OpResult> add_n_grad(std::vector<pir::Value> inputs,

【NewIR】add add_n_grad and split_with_num_grad (Split op ) #56873

【NewIR】add add_n_grad and split_with_num_grad (Split op ) #56873

Conversation

xiaoguoguo626807 commented Sep 1, 2023 • edited Loading

PR types

PR changes

Description

paddle-bot bot commented Sep 1, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lanxianghit left a comment

Choose a reason for hiding this comment

Aurelius84 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xiaoguoguo626807 left a comment

Choose a reason for hiding this comment

xiaoguoguo626807 commented Sep 1, 2023 •

edited

Loading