Skip to content

Conversation

@lshpku
Copy link
Contributor

@lshpku lshpku commented Aug 26, 2025

PR Category

Auto Parallel

PR Types

Improvements

Description

在 DualPipeV 中,当执行 backward_pass 时,不直接调用 backwad_chunk.backward,而是调用模型的 overlapped_forward_backward 方法,但只传入 backward_chunk,不传入 forward_chunk:

model.overlapped_forward_backward(
    forward_chunk=[],
    backward_chunk=backward_chunk
)

这样允许模型自主编排 backward_chunk,实现 backward 之间的自 Overlap

该 PR 为 兼容性升级,不会破坏现有模型的兼容性,模型若未实现 backward 的自 Overlap 机制,则会跑原来不 overlap 的 backward,性能不变

DSV3 实现自 Overlap 机制:PaddlePaddle/PaddleNLP#11003

Pcard-85711

@paddle-bot
Copy link

paddle-bot bot commented Aug 26, 2025

你的PR提交成功,感谢你对开源项目的贡献!
请关注后续CI自动化测试结果,详情请参考Paddle-CI手册
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@lshpku lshpku force-pushed the custom-backward-overlap branch from 75f3f68 to 2056999 Compare August 26, 2025 07:11
@codecov-commenter
Copy link

codecov-commenter commented Aug 26, 2025

Codecov Report

❌ Patch coverage is 33.33333% with 2 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (develop@b95cb68). Learn more about missing BASE report.

Files with missing lines Patch % Lines
...addle/distributed/fleet/meta_parallel/dualpipev.py 33.33% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             develop   #74891   +/-   ##
==========================================
  Coverage           ?   33.33%           
==========================================
  Files              ?        1           
  Lines              ?        3           
  Branches           ?        0           
==========================================
  Hits               ?        1           
  Misses             ?        2           
  Partials           ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@phlrain phlrain merged commit 8bb6606 into PaddlePaddle:develop Aug 27, 2025
105 of 110 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants