Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NPU] add dropout npu op #34081

Merged
merged 5 commits into from
Jul 12, 2021
Merged

Conversation

pangyoki
Copy link
Contributor

@pangyoki pangyoki commented Jul 11, 2021

PR types

New features

PR changes

OPs

Describe

实现的是upscale_in_train方式的dropout:
图片

使用DropOutGenMask+DropOutDoMask实现Dropout NPU OP。

因为随机性,单测中只测试了dropout_prob = 0 和 dropout_prob = 1的情况。
手动测试了下dropout_prob = 0.3的输出结果,看scale的比例确实是1.428倍 ( 1 / (1-0.3) = 1.428):
图片

注意:

  • DropOutGenMask中需要使用Tensor prob,在NPU执行时其shape必须为0,这里在runner中做了特殊处理,后面需要优化实现。
  • 添加了对uint8数据类型的支持。

单测结果:
图片

log:
图片
图片

@paddle-bot-old
Copy link

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@pangyoki pangyoki requested a review from zhiqiu July 11, 2021 12:22
Copy link
Contributor

@zhiqiu zhiqiu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@wanghuancoder wanghuancoder left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@pangyoki pangyoki merged commit c4e0498 into PaddlePaddle:develop Jul 12, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants