Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

【PIR API adaptor No.78、79、82】 Migrate api in fused_transformer.py into pir #58453

Merged

Conversation

MarioLulab
Copy link
Contributor

@MarioLulab MarioLulab commented Oct 27, 2023

PR types

Others

PR changes

Others

Description

任务 issue: #58067

PIR API 推全升级
paddle.incubate.nn.FusedMultiHeadAttention 迁移升级至 pir,并更新单测,单测覆盖率(1/2)

TODO: test/legacy_test/static_model_parallel_fused_attention.py 中的 TestModelParallel 未测试,因为当前暂不支持 pir 模式下的反向

paddle.incubate.nn.FusedBiasDropoutResidualLayerNorm 迁移升级至 pir,并更新单测,单测覆盖率(x/x)

TODO: 暂时搁置,因为无法自动生成 C++ 端的api

paddle.incubate.nn.FusedFeedForward 迁移升级至 pir,并更新单测,单测覆盖率(2/5)

主要工作:

  1. 修复原先存在于 APITestStaticFusedFFN 里的 bug
  2. APITestStaticFusedFFN 单测中原先是对两个不同的网络在同一个 Program 下组网,并对比结果。现在拆分成两个 Program,修复了 Exception: There is a fetch op in Program which will fetch variable that is not belong to fetch_targets 的问题

TODO:

  1. test/legacy_test/static_model_parallel_fused_feedforward.py 中的 TestModelParallel 未测试,因为当前暂不支持 pir 模式下的反向
  2. 跳过 pir 模式下的 test/legacy_test/test_fused_feedforward_op.py 中的 TestFusedFFNOpError 的 test_errors 输入 dtype 检查
  3. 跳过 test/legacy_test/test_static_model_parallel_fused_attention.py
  4. 跳过 test/legacy_test/test_static_model_parallel_fused_feedforward.py

@paddle-bot
Copy link

paddle-bot bot commented Oct 27, 2023

你的PR提交成功,感谢你对开源项目的贡献!
请关注后续CI自动化测试结果,详情请参考Paddle-CI手册
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@paddle-bot paddle-bot bot added the contributor External developers label Oct 27, 2023
@paddle-bot
Copy link

paddle-bot bot commented Oct 27, 2023

✅ This PR's description meets the template requirements!
Please wait for other CI results.

@MarioLulab MarioLulab marked this pull request as ready for review November 1, 2023 12:40
@0x45f
Copy link
Contributor

0x45f commented Nov 3, 2023

paddle.incubate.nn.FusedBiasDropoutResidualLayerNorm 适配需要先将kernel迁移到phi下,然后再做pir的适配,大概做法已经同步给陆琦~

Copy link
Contributor

@sunzhongkai588 sunzhongkai588 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no docs changes,LGTM

Copy link
Contributor

@XiaoguangHu01 XiaoguangHu01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
contributor External developers
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants