Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

【PIR】Sparse 算子 Python API 适配 #64492

Closed
risemeup1 opened this issue May 21, 2024 · 0 comments
Closed

【PIR】Sparse 算子 Python API 适配 #64492

risemeup1 opened this issue May 21, 2024 · 0 comments
Assignees
Labels
status/close 已关闭 type/others 其他问题

Comments

@risemeup1
Copy link
Contributor

risemeup1 commented May 21, 2024

问题描述 Please describe your issue

一,背景和任务列表

任务详情和单测验证请参考之前发布的任务:#57097

  1. Sparse Python API 情况:
    目前 python 端注册了大约46个左右的 API,集中在 python/paddle/sparse 目录下,绝大部分是动态图 API,14个是动静统一的 API,算子定义都定义在 sparse_ops.yaml 及 sparse_backward.yaml 文件中,通过自动代码生成的方式生成在 paddle/fluid/operators/generated_sparse_op.cc 文件,全部符合 yaml 定义规范,多数算子都注册了 coo、csr 两种 kernel,根据 PIR 算子/Kernel 一对一的特点,PIR 下需要区分定义 coo、csr,这点与 SelectedRows 特点相近。
序号 Python API 动态图 静态图 Kernel 是否做到动静一致
1 sin @MrJs133
@risemeup1
sin_coo、sin_csr
2 tan @MrJs133
tan_coo、tan_csr
3 asin @MrJs133
@risemeup1
asin_coo、asin_csr
4 transpose transpose_coo、transpose_csr
5 sum sum_coo、sum_csr
6 atan atan_coo、atan_csr
7 sinh @epiphanyer #67442
sinh_coo、sinh_csr
8 asinh asinh_coo、asinh_csr
9 atanh atanh_coo、atanh_csr
10 tanh tanh_coo、tanh_csr
11 square square_coo、square_csr
12 sqrt @MrJs133
@risemeup1
sqrt_coo、sqrt_csr
13 log1p @MrJs133
@risemeup1
log1p_coo、log1p_csr
14 cast cast_coo、cast_csr
15 pow pow_coo、pow_csr
16 neg scale_coo、scale_csr
17 abs abs_coo、abs_csr
18 coalesce coalesce_coo
19 rad2deg cast_coo、cast_csr、scale_coo、scale_csr
20 deg2rad cast_coo、cast_csr、scale_coo、scale_csr
21 expm1 expm1_coo、expm1_csr
22 reshape reshape_coo、reshape_csr
23 isnan isnan_coo、isnan_csr
24 slice slice_coo、slice_csr
25 pca_lowrank 通过其他 API 组合
26 addmm addmm_csr_dense、addmm_csr_csr、addmm_coo_dense、addmm_coo_coo
27 sparse_coo_tensor sparse_coo_tensor
28 sparse_csr_tensor sparse_csr_tensor
29 matmul matmul_csr_dense、matmul_csr_csr、matmul_coo_dense、matmul_coo_coo
30 masked_matmul masked_matmul_csr
31 mv mv_coo、mv_csr
32 add add_coo_coo、add_csr_csr、add_coo_dense
33 subtract subtract_coo_coo、subtract_csr_csr
34 multiply multiply_coo_coo、multiply_csr_csr
35 divide divide_coo_coo、divide_csr_csr
36 is_same_shape tensor method
37 relu relu_coo、relu_csr
38 softmax softmax_coo、softmax_csr
39 relu6 relu6_coo、relu6_csr
40 leaky_relu leaky_relu_coo、leaky_relu_csr
41 conv3d conv3d_coo
42 subm_conv3d conv3d_coo
43 conv2d conv2d_coo
44 subm_conv2d conv3d_coo
45 max_pool3d maxpool_coo
46 attention fused_attention_csr
  1. 单测现状:
  • 有一些动静统一的api既有动态图下的单测也有静态图下的单测,如sum,conv等,此类单测已经修复完毕;
    image

  • 有一些动静统一的api只有动态图下的单测,没有静态图下的单测,如subm_conv3d,需要补在静态图下的单测,
    image

  • 有一些仅仅在动态图下有定义的api,如sin,需要对python API进行适配升级,然后补全在动态图和静态图下的单测,请参考
    【开源任务】新 IR API 推全升级 #57097
    image

二、Task 📚

  1. 动静统一的python api,需要补全静态图单测的任务
序号 Python API 所在文件 对应单测 认领人/PR 单测覆盖率
1 reshape Paddle/python/paddle/sparse/unary.py test/legacy_test/test_sparse_reshape_op.py
2 add Paddle/python/paddle/sparse/binary.py test/deprecated/legacy_test/test_sparse_elementwise_op.py
3 softmax python/paddle/sparse/nn/functional/activation.py test/legacy_test/test_sparse_fused_attention_op.py
4 subm_conv3d python/paddle/sparse/nn/functional/conv.py test/legacy_test/test_sparse_conv_op.py
5 subm_conv2d python/paddle/sparse/nn/functional/conv.py test/legacy_test/test_sparse_conv_op.py
6 max_pool3d python/paddle/sparse/nn/functional/pooling.py test/legacy_test/test_sparse_pooling_op.py

2 仅在动态图下有定义的api,需要对python API进行适配升级,然后补全在动态图和静态图下的单测

序号 Python API 所在文件 对应单测 认领人/PR 单测覆盖率
7 sin python/paddle/sparse/unary.py test_sparse_unary_op.py
8 tan python/paddle/sparse/unary.py test_sparse_unary_op.py
9 asin python/paddle/sparse/unary.py test_sparse_unary_op.py
10 transpose python/paddle/sparse/unary.py test_sparse_transpose_op.py
11 atan python/paddle/sparse/unary.py test_sparse_unary_op.py
12 sinh python/paddle/sparse/unary.py test_sparse_unary_op.py
13 asinh python/paddle/sparse/unary.py test_sparse_unary_op.py
14 atanh python/paddle/sparse/unary.py test_sparse_unary_op.py
15 tanh python/paddle/sparse/unary.py test_sparse_unary_op.py
16 square python/paddle/sparse/unary.py test_sparse_unary_op.py
17 sqrt python/paddle/sparse/unary.py test_sparse_unary_op.py
18 log1p python/paddle/sparse/unary.py test_sparse_unary_op.py
19 pow python/paddle/sparse/unary.py test_sparse_unary_op.py
20 neg python/paddle/sparse/unary.py test_sparse_unary_op.py
21 abs python/paddle/sparse/unary.py test_sparse_unary_op.py
22 cast python/paddle/sparse/unary.py test_sparse_unary_op.py
23 coalesce python/paddle/sparse/unary.py test_sparse_conv_op.py/test_sparse_pca_lowrank.py/test_sparse_utils_op.py
24 rad2deg python/paddle/sparse/unary.py test_sparse_unary_op.py
25 deg2rad python/paddle/sparse/unary.py test_sparse_unary_op.py
26 expm1 python/paddle/sparse/unary.py test_sparse_unary_op.py
27 addmm python/paddle/sparse/unary.py test_sparse_addmm_op.py
28 sparse_csr_tensor python/paddle/sparse/unary.py test_sparse_utils_op.py/test_sparse_matmul_op.py
29 matmul python/paddle/sparse/binary.py test_sparse_matmul_op.py
30 masked_matmul python/paddle/sparse/binary.py test_sparse_matmul_op.py
31 mv python/paddle/sparse/binary.py test_sparse_mv_op.py
32 subtract python/paddle/sparse/binary.py test_sparse_elementwise_op.py
33 multiply python/paddle/sparse/binary.py test_sparse_elementwise_op.py
34 divide python/paddle/sparse/binary.py test_sparse_elementwise_op.py
35 is_same_shape python/paddle/sparse/binary.py test_sparse_is_same_shape_op.py
36 relu6 python/paddle/sparse/nn/functional/activation.py test_sparse_unary_op.py
37 leaky_relu python/paddle/sparse/nn/functional/activation.py test_sparse_unary_op.py
38 leaky_relu python/paddle/sparse/nn/functional/transformer.py test_sparse_fused_attention_op.py

1. 认领方式

请大家以 comment 的形式认领任务,如:

【报名】:1、3、12-13

多个任务之间需要使用中文顿号分隔,报名多个连续任务可用横线表示,如 2-5

2. PR提交

  • PR名称需要加前缀 【Migrate Sparse API No.XXX】
  • PR描述中需要附上本issue
  • 评论里或者 review request @risemeup1 研发会进行审核

看板信息

任务方向 任务数量 提交作品 / 任务认领 提交率 完成 完成率
【快乐开源】PIR下Sparse算子Python API 适配 38 1 / 6 2.63% 1 2.63%

统计信息

排名不分先后 @epiphanyer (1)

@paddle-bot paddle-bot bot added status/close 已关闭 and removed status/new-issue 新建 labels May 21, 2024
@risemeup1 risemeup1 reopened this May 21, 2024
@paddle-bot paddle-bot bot added status/reopen 重新打开 and removed status/close 已关闭 labels May 21, 2024
@luotao1 luotao1 assigned luotao1 and risemeup1 and unassigned From00 May 21, 2024
@risemeup1 risemeup1 changed the title Sparse算子Python API 适配 PIR下Sparse算子Python API 适配 May 22, 2024
@luotao1 luotao1 changed the title PIR下Sparse算子Python API 适配 【PIR】Sparse 算子 Python API 适配 May 23, 2024
@PaddlePaddle PaddlePaddle deleted a comment from from-Pb2Ag Jun 26, 2024
@PaddlePaddle PaddlePaddle deleted a comment from ooooo-create Jun 26, 2024
@PaddlePaddle PaddlePaddle deleted a comment from YibinLiu666 Jun 26, 2024
@PaddlePaddle PaddlePaddle deleted a comment from risemeup1 Aug 13, 2024
@luotao1 luotao1 closed this as completed Aug 21, 2024
@paddle-bot paddle-bot bot added status/close 已关闭 and removed status/reopen 重新打开 labels Aug 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status/close 已关闭 type/others 其他问题
Projects
Development

No branches or pull requests

3 participants