Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

【Hackathon 4th No.24】为 Paddle 新增 paddle.sparse.is_nan 稀疏 API #415

Merged
merged 11 commits into from
Mar 13, 2023
98 changes: 98 additions & 0 deletions rfcs/APIs/20230305_api_design_for_paddle_sparse_isnan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
# paddle.sparse.is_nan 设计文档
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个API的名字调整成 paddle.sparse.isnan 吧,和dense的风格对齐,后面涉及到名字的都需要改下

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已修改


| API名称 | paddle.sparse.is_nan |
| ------------------------------------------------------------ | ------------------------------------------------- |
| 提交作者<input type="checkbox" class="rowselector hidden"> | thunder95 |
| 提交时间<input type="checkbox" class="rowselector hidden"> | 2023-03-05 |
| 版本号 | V1.0 |
| 依赖飞桨版本<input type="checkbox" class="rowselector hidden"> | Develop |
| 文件名 | 20230305_api_design_for_paddle_sparse_isnan.md<br> |


# 一、概述

## 1、相关背景

is_nan 是一个检查输入稀疏 Tensor 的每一个值是否为 +/-NaN 。目前在 PaddlePaddle 种,没有稀疏涨量的is_nan计算逻辑。
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这句话没写清楚

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已修改

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PaddlePaddle中

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已修改

针对 Paddle 的两种稀疏 Tensor 格式 COO 与 CSR ,都需新增 reshape 的计算逻辑,一共需要新增 2个 kernel 的前向与反向。
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reshape?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已修改


## 2、功能目标

在飞桨中增加 paddle.sparse.is_nan API。

## 3、意义

飞桨将支持 paddle.sparse.is_nan API。

# 二、飞桨现状

目前飞桨还没有稀疏张量的 is_nan 操作。


# 三、业内方案调研

Pytorch, Tensorflow, Scipy都没有直接实现对稀疏张量的 is_nan 的一元操作。

# 四、对比分析

Pytorch, Tensorflow, Scipy虽然对于稠密张量支持is_nan操作,但都没有直接实现对稀疏张量的 is_nan 的一元操作。
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

torch支持稀疏的is_nan判断

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

谢谢指导,已修改

在 PaddlePaddle 中将对这个算子操作进行支持。

# 五、设计思路与实现方案

## 命名与参数设计

sparse is_nan 这个稀疏张量上的方法的命名和参数不需要额外设计,在 paddle/phi/api/yaml 下新增注册该算子的前向和反向。
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

需要写下API命名和参数设计

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已修改


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

paddle/phi/api/yaml/sparse_ops.yaml中注册,需要描述一下yaml形式

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已修改

## 底层OP设计

新增两个 Kernel:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kernel形式写IsnanCooKernel就可以,参数不对

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已修改


``` cpp
SparseCooTensor IsnanCoo(const Context& dev_ctx,const SparseCooTensor& x) {
SparseCooTensor coo;
IsnanCooKernel<T, Context>(dev_ctx, x, shape, &coo);
return coo;
}
```

```cpp
SparseCsrTensor IsnanCsr(const Context& dev_ctx,const SparseCsrTensor& x) {
SparseCsrTensor csr;
IsnanCsrKernel<T, Context>(dev_ctx, x, shape, &csr);
return csr;
}
```

在前向推理中,一元操作的实现较简单,取出 DenseTensor 类型的 non_zero_elements() 后,逐元素进行 is_nan 操作,并创建新的稀疏张量即可。
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个API建议参考下unary_kernel.cc,看能否批量注册复用dense kernel

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

可以复用dense kernel

在反向梯度传播里, 只对+/-NaN的元素进行梯度为1.0的数值进行反向传播,对于非NaN数值的元素设置梯度为0.0.

## API实现方案

在 Paddle repo 的 python/paddle/sparse/unary.py 文件中新增api:

```cpp
@dygraph_only
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

新增的sparse API需要同时实现动态图和静态图

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已修改

def is_nan(x, name=None):
return _C_ops.sparse_is_nan(x)
```

# 六、测试和验收的考量

测试考虑的case如下:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

具体有哪些case设计呢

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已修改


- 数值正确性

# 七、可行性分析和排期规划

前两周实现代码、文档和测试。

第三周进行 Code Review 和继续迭代。

# 八、影响面

对其它模块没有影响。

# 名词解释

# 附件及参考资料