Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix core so name mismatch error #43977

Merged
merged 2 commits into from
Jul 1, 2022

Conversation

chenwhql
Copy link
Contributor

@chenwhql chenwhql commented Jun 30, 2022

PR types

Bug fixes

PR changes

Others

Describe

Fix core so name mismatch error

paddle编译完成后,将libpaddle_pybind.so copy为core_(no)avx.so,但是并没有更新其soname,其soname仍然是libpaddle_pybind.so,这导致外部链接的时候会找不到的libpaddle_pybind.so,示例如下

  • core_avx.so信息
    image
  • 编译完自定义算子so的链接信息
    image

因此本PR修复该问题,修复之后,core_avx.so信息如下:
image

该问题修复后,可以基于安装后的paddle phi c++ API在外部编译可执行文件,示例如下:

  • CMakeLists.txt
cmake_minimum_required(VERSION 3.14)
project(tensortest LANGUAGES CXX)

find_package(Threads REQUIRED)

find_package (PythonLibs REQUIRED PATH /work/dev3/paddle/build_venv/bin/python/lib)
message(STATUS "PYTHON_LIBRARIES = ${PYTHON_LIBRARIES}")

find_package(pybind11 CONFIG)
message(STATUS "Pybind11_INCLUDES = ${pybind11_INCLUDE_DIRS}, pybind11_LIBRARIES=${pybind11_LIBRARIES}, pybind11_DEFINITIONS=${pybind11_DEFINITIONS}")

set(link_libraries
/usr/local/python3.7.0/lib/libpython3.7m.so.1.0
-l:core_avx.so)

set(include_dirs
${pybind11_INCLUDE_DIRS}
/work/dev3/paddle/build_venv/lib/python3.7/site-packages/paddle/include/)

set(link_libraries_path
${CMAKE_SOURCE_DIR}
/work/dev3/paddle/build_venv/lib/python3.7/site-packages/paddle/libs/
/work/dev3/paddle/build_venv/lib/python3.7/site-packages/paddle/fluid/)

include_directories(${include_dirs})
link_directories(${link_libraries_path})

set(name main)
set(source main.cc)
add_executable(${name} ${source})
target_link_libraries(${name} ${link_libraries})
  • main.cc
#include "paddle/extension.h"

int main (){
    std::cout << "Run Start" << std::endl;

    auto a = paddle::full({3, 4}, 2.0);
    auto b = paddle::full({4, 5}, 3.0);
    auto out = paddle::matmul(a, b);

    std::cout << "Run End" << std::endl;
}
  • 执行结果:
Run Start
Run End

patchelf 命令

patchelf 是一个用来修改elf格式的动态库和可执行程序的小工具,可以修改动态链接库的库名字,以及链接库的RPATH。

打印出动态库的soname

patchelf --print-soname xxx.so

修改动态库的soname

patchelf --set-soname oldxxx.so newxxx.so

查看并修改第三方依赖库

patchelf --print-needed xxx.so

patchelf --replace-needed oldxxx.so newxxx.so this.so

修改rpath

patchelf --set-rpath '$ORIGIN/' main

@paddle-bot-old
Copy link

你的PR提交成功,感谢你对开源项目的贡献!
请关注后续CI自动化测试结果,详情请参考Paddle-CI手册
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@@ -578,7 +578,8 @@ if '${CMAKE_BUILD_TYPE}' == 'Release':
commands = ["install_name_tool -id '@loader_path/../libs/' ${PADDLE_BINARY_DIR}/python/paddle/fluid/${FLUID_CORE_NAME}" + '.so']
commands.append("install_name_tool -add_rpath '@loader_path/../libs/' ${PADDLE_BINARY_DIR}/python/paddle/fluid/${FLUID_CORE_NAME}" + '.so')
else:
commands = ["patchelf --set-rpath '$ORIGIN/../libs/' ${PADDLE_BINARY_DIR}/python/paddle/fluid/${FLUID_CORE_NAME}" + '.so']
commands = ["patchelf --set-soname '${FLUID_CORE_NAME}.so' ${PADDLE_BINARY_DIR}/python/paddle/fluid/${FLUID_CORE_NAME}" + '.so']
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个会再创建一个软链接吗 还是直接改名字

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

不会,直接改名字

Copy link
Contributor

@Aurelius84 Aurelius84 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@zh794390558 zh794390558 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@chenwhql chenwhql merged commit ccb333c into PaddlePaddle:develop Jul 1, 2022
chenwhql added a commit to chenwhql/Paddle that referenced this pull request Aug 8, 2022
* fix core avx soname error

* remove print info
phlrain pushed a commit that referenced this pull request Aug 9, 2022
* fix device context init error (#43910)

* Fix core so name mismatch error (#43977)

* fix core avx soname error

* remove print info

* add clip_extra (#44008)

* fix tensor stream error in custom op (#44500)

* fix custom op attr names size error (#44938)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants