Link cpp test to libpaddle.so/libpaddle.dll #56691
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
你的PR提交成功，感谢你对开源项目的贡献! |
winter-wang
force-pushed
the
fix_compile_kill_9_bug
branch
29 times, most recently
from
50b5b08 to
50faf51
Compare
winter-wang
force-pushed
the
fix_compile_kill_9_bug
branch
5 times, most recently
from
9d01e92 to
dbe9f55
Compare
winter-wang
force-pushed
the
fix_compile_kill_9_bug
branch
from
dbe9f55 to
bea6d0a
Compare
risemeup1
changed the title
YuanRisheng
approved these changes
Sep 26, 2023
YuanRisheng
reviewed
Sep 26, 2023
JiabinYang
approved these changes
Sep 26, 2023
There was a problem hiding this comment.
This file can be deleted
There was a problem hiding this comment.
全局搜索了一下egr::scale符号，这个函数在很多单测文件中进行了大量使用(benchmark_utils.cc、forward_autograd_test.cc、fwd_bwd_joint_test.cc)。直接删除该文件的话，相应的单测也需要修改，不然会编译失败。因此这个文件可能需要动态图相关的同学来帮忙进行删除。
zhangbo9674
approved these changes
Sep 26, 2023
Aurelius84
approved these changes
Sep 26, 2023
chenwhql
approved these changes
Sep 26, 2023
This was referenced Sep 26, 2023
Frida-a
pushed a commit
to Frida-a/Paddle
that referenced
this pull request
Oct 14, 2023
jiahy0825
pushed a commit
to jiahy0825/Paddle
that referenced
this pull request
Oct 16, 2023
danleifeng
pushed a commit
to danleifeng/Paddle
that referenced
this pull request
Nov 14, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
Bug fixes
PR changes
Others
Description
修复CI流水线中，单测体积过大问题。
目前的很多单测，对Paddle框架通过静态库(.a、.lib)的形式进行链接，导致单个单测的体积过大（超过1G）。在CI流水线编译中，当并发线程过多，就容易因为内存空间不足而使编译失败。
本pr内容如下：
将部分单测修改为对Paddle框架通过动态库(.so、.dll)的形式进行链接。多个单测共享同一份动态库，有效减少内存的占用。
提供了TEST_API宏定义， 在定义了WITH_TESTING的情况下，TEST_API修饰的对象会被编译为动态库的导出符号，方便单测进行使用。
提供了 paddle_test函数， 该函数会自动链接libpaddle.so或者libpaddle.dll动态库。当单个单测体积过大时，建议通过paddle_test链接动态库进行单元测试。
本pr合入前， coverage单测编译后，build目录的大小为 *G, 合入后， build目录的大小为**G。
本pr主要是打通了windows下和linux下同时使用动态库进行测试的道路。后续会逐步将所有大于**M的单测，都改为通过动态库进行链接测试。
Todo:
@risemeup1
将现有的cc_test函数的使用都替换为paddle_test, 移除cc_test函数。
作为对paddle_test函数功能的补充：
当单个单测的大小超过**M的时候，通过CI进行拦截。
Other
Pcard-67164