fix LoDTensorArray crash in Debug mode build #37954
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks for your contribution! |
sneaxiy
approved these changes
Dec 9, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
Others
PR changes
Others
Describe
Paddle中 LoDTensorArray 实际为 std::vector
<LoDTensor>，Pybind11针对stl容器提供自动转换功能，std::vector<LoDTensor>自动转换为List[LoDTensor]；通过bind前添加PYBIND11_MAKE_OPAQUE(paddle::framework::LoDTensorArray)避免自动转换。现有实现中：
pybind.cc中，bind前通过添加PYBIND11_MAKE_OPAQUE，避免自动转换为Python的List，Python层LoDTensorArray相关的功能基于此(LoDTensorArray不是Python List)完成。
reader_py.cc中，bind也有使用std::vector<
framework::LoDTensor>，且在bind前没有添加PYBIND11_MAKE_OPAQUE，所以接口中会自动转换为List。这样在Python层两者一起使用会造成不一致的情况，进而导致Debug Mode下参数传递问题。
为修复这种不一致，可以在reader_py.cc中添加PYBIND11_MAKE_OPAQUE，但这会产生副作用，即reader_py.cc中也有bind接口是期望自动转换的：
MultiDeviceFeedReader::ResultList()，所以随着PYBIND11_MAKE_OPAQUE的添加，同时需要在Python层对期望得到List的调用进行LoDTensorArray对象的手动_move_to_list()调用，这样能够保持LoDTensorArray使用的一致性，同时也可以显式转换避免问题。相关：#37387 #32252 #24863