add cumprod_grad composite #64432

YibinLiu666 · 2024-05-19T08:18:36Z

PR Category

Others

PR Types

New features

Description

基于支持了 inclusive, reverse 参数的 cumprod 算子(#64022)，实现 cumprod_grad 的组合逻辑，避免 cumprod_grad 的组合算子因为输入含有 0 而导致输出除 0 出 nan 的问题

paddle-bot · 2024-05-19T08:18:40Z

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

HydrogenSulfate

辛苦提交PR，单测和代码可以再看一下

HydrogenSulfate · 2024-05-20T05:55:10Z

paddle/fluid/prim/api/composite_backward/composite_backward_api.h

+    auto ones_tensor = full<T>(x_dim, 1.0, x.dtype());
+    auto replace_one = (1 - zero_mask) * x + zero_mask * ones_tensor;


加法第二项zero_mask * ones_tensor是不是可以化简为zero_mask，一个张量乘以全1矩阵ones_tensor等于没乘？

如果上面结论成立，可以删除ones_tensor

HydrogenSulfate · 2024-05-20T06:01:35Z

test/prim/prim/vjp/eager/test_comp_eager_cumprod_grad.py

+@param.parameterized_class(
+    ('primal', 'dtype'),
+    [
+        (
+            np.random.rand(2, 3, 4),
+            np.float32,
+        ),
+        (
+            np.random.rand(2, 3, 3, 4),
+            np.float32,
+        ),
+    ],
+)


形状可以添加0D、1D、2D、5D

数据类型添加int64

数据本身添加包含1个0、2个0，全部都是0的case

HydrogenSulfate · 2024-05-20T06:04:38Z

test/prim/prim/vjp/static/test_comp_cumprod_grad.py

补充更多单测case，同test/prim/prim/vjp/eager/test_comp_eager_cumprod_grad.py

HydrogenSulfate · 2024-05-21T03:20:35Z

paddle/fluid/prim/api/composite_backward/composite_backward_api.h

+    auto zero_mask_cumsum1 = cumsum<T>(zero_mask, dim, false, false, reverse);
+    auto zero_mask_cumsum2 = cumsum<T>(zero_mask, dim, false, true, reverse);


变量命名尽量准确符合语义，
zero_mask_cumsum1 --> zero_mask_cumsum_left
zero_mask_cumsum2 --> zero_mask_cumsum_right

HydrogenSulfate · 2024-05-21T03:22:36Z

paddle/fluid/primitive/rule/vjp/details.h

+    auto ones_tensor = full<T>(x_dim, 1.0, x.dtype());
+    auto replace_one = (1 - zero_mask) * x + zero_mask * ones_tensor;


这里的ones_tensor应该可以删掉吧

HydrogenSulfate · 2024-05-21T03:26:41Z

paddle/fluid/primitive/rule/vjp/details.h

+    auto ones_tensor = full<T>(x_dim, 1.0, x.dtype());
+    auto replace_one = (1 - zero_mask) * x + zero_mask * ones_tensor;


Suggested change

auto ones_tensor = full<T>(x_dim, 1.0, x.dtype());

auto replace_one = (1 - zero_mask) * x + zero_mask * ones_tensor;

auto replace_one = (1 - zero_mask) * x + zero_mask;

HydrogenSulfate · 2024-05-21T03:28:36Z

paddle/fluid/primitive/rule/vjp/details.h

+    auto ones_tensor = full<T>(x_dim, 1.0, x.dtype());
+    auto replace_one = (1 - zero_mask) * x + zero_mask * ones_tensor;


逻辑连续的代码块之间可以加上一些注释，比如此处将0的位置填充成1，可以加上: # fill the positions of 0 with 1.

HydrogenSulfate · 2024-05-21T03:30:21Z

test/prim/prim/vjp/eager/test_comp_eager_cumprod_grad.py

+
+@param.parameterized_class(
+    ('primal', 'dtype'),
+    [


加上一个单元素的测试：np.array(np.rand(), dtype="float32")

HydrogenSulfate · 2024-05-21T03:33:10Z

test/prim/prim/vjp/static/test_comp_cumprod_grad.py

+        return out
+
+
+@param.parameterized_class(


同上，补充单元素的0-Dcase

HydrogenSulfate · 2024-05-21T03:35:51Z

paddle/fluid/prim/api/composite_backward/composite_backward_api.h

此处的实现跟details.h里的实现好像有一些区别？可以确认一下

… cumprod_grad

HydrogenSulfate · 2024-05-22T11:39:19Z

paddle/fluid/prim/api/composite_backward/composite_backward_api.h

+    auto zero_mask_cumsum_exclusive =
+        cumsum<T>(zero_mask, dim, false, true, reverse);
+    auto zero_mask_cumsum =
+        zero_mask_cumsum_inclusive + zero_mask_cumsum_exclusive;


inclusive + exclusive 是否相当于两倍的exclusive + x？两次cumprod应该比一次cumprod + scale + add耗时会更长，是否可以改成后者实现方式？
zero_mask_cumsum_inclusive + zero_mask_cumsum_exclusive --> scale<T>(zero_mask_cumsum_exclusive, 2) + zero_mask

cyber-pioneer

todo: 建议适配optest，当前的这批单侧属于遗留case，未来会被清理

* add cumprod_grad composite * remove cout * update test and fix some bug * update test * add comment and test * remove cout * update * update static test * update * Update details.h

add cumprod_grad composite

10b16ad

YibinLiu666 requested review from cxxly, xiaoguoguo626807, Charles-hit, cyber-pioneer and JiabinYang as code owners May 19, 2024 08:18

paddle-bot bot added the contributor External developers label May 19, 2024

remove cout

d37962e

HydrogenSulfate requested changes May 20, 2024

View reviewed changes

update test and fix some bug

db81952

HydrogenSulfate reviewed May 21, 2024

View reviewed changes

YibinLiu666 added 6 commits May 21, 2024 12:06

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

4e08872

… cumprod_grad

update test

d23d6d3

add comment and test

a0b8dc0

remove cout

3c8b7cf

update

300c63e

update static test

d200f8b

HydrogenSulfate reviewed May 22, 2024

View reviewed changes

YibinLiu666 and others added 2 commits May 22, 2024 13:59

update

f985123

Update details.h

15a1fa2

zyfncg approved these changes May 24, 2024

View reviewed changes

cyber-pioneer approved these changes May 24, 2024

View reviewed changes

XieYunshen approved these changes May 24, 2024

View reviewed changes

HydrogenSulfate approved these changes May 24, 2024

View reviewed changes

HydrogenSulfate merged commit 4b757dd into PaddlePaddle:develop May 24, 2024
32 checks passed

YibinLiu666 deleted the cumprod_grad branch May 24, 2024 05:12

YibinLiu666 mentioned this pull request Jun 11, 2024

WAVE SUMMIT+2024上半年飞桨开源之星评选-信息征集 PaddlePaddle/community#892

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add cumprod_grad composite #64432

add cumprod_grad composite #64432

YibinLiu666 commented May 19, 2024 •

edited by HydrogenSulfate

Loading

paddle-bot bot commented May 19, 2024

HydrogenSulfate left a comment

HydrogenSulfate May 20, 2024

YibinLiu666 May 20, 2024

HydrogenSulfate May 20, 2024

YibinLiu666 May 20, 2024

HydrogenSulfate May 20, 2024

YibinLiu666 May 20, 2024

HydrogenSulfate May 21, 2024

HydrogenSulfate May 21, 2024

HydrogenSulfate May 21, 2024

HydrogenSulfate May 21, 2024

YibinLiu666 May 23, 2024

HydrogenSulfate May 21, 2024

YibinLiu666 May 23, 2024

HydrogenSulfate May 21, 2024 •

edited

Loading

YibinLiu666 May 23, 2024

HydrogenSulfate May 21, 2024

YibinLiu666 May 23, 2024

HydrogenSulfate May 22, 2024 •

edited

Loading

YibinLiu666 May 23, 2024

cyber-pioneer left a comment

		auto ones_tensor = full<T>(x_dim, 1.0, x.dtype());
		auto replace_one = (1 - zero_mask) * x + zero_mask * ones_tensor;

		auto zero_mask_cumsum1 = cumsum<T>(zero_mask, dim, false, false, reverse);
		auto zero_mask_cumsum2 = cumsum<T>(zero_mask, dim, false, true, reverse);

add cumprod_grad composite #64432

add cumprod_grad composite #64432

Conversation

YibinLiu666 commented May 19, 2024 • edited by HydrogenSulfate Loading

PR Category

PR Types

Description

paddle-bot bot commented May 19, 2024

HydrogenSulfate left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

HydrogenSulfate May 21, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

HydrogenSulfate May 22, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cyber-pioneer left a comment

Choose a reason for hiding this comment

YibinLiu666 commented May 19, 2024 •

edited by HydrogenSulfate

Loading

HydrogenSulfate May 21, 2024 •

edited

Loading

HydrogenSulfate May 22, 2024 •

edited

Loading