Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot build Debug build with WITH_GPU=ON due to file truncation #14775

Closed
Sand3r- opened this issue Dec 6, 2018 · 8 comments
Closed

Cannot build Debug build with WITH_GPU=ON due to file truncation #14775

Sand3r- opened this issue Dec 6, 2018 · 8 comments
Assignees
Labels

Comments

@Sand3r-
Copy link
Contributor

Sand3r- commented Dec 6, 2018

System information
-PaddlePaddle version 29d9fb5
-CPU: including MKLDNN version 0.17
-GPU: including CUDA 8.0/CUDNN 7 version
-OS Platform Ubuntu 16.04
-Python version 2.7.12

  • Install method: build from source(without docker)

To Reproduce
Run cmake:

cmake -DCMAKE_BUILD_TYPE=Debug -DWITH_GPU=ON -DWITH_MKLDNN=ON -DWITH_TESTING=ON -DWITH_PROFILER=ON -DWITH_STYLE_CHECK=OFF -DWITH_INFERENCE=ON -DWITH_INFERENCE_API_TEST=ON .. 

and build with make -j
Describe your current behavior
When GPU build is ran with Debug build type, a linking error occurs:

/usr/bin/ar: /home/mgallus/src/Sander/Paddle/build_gpu_deb/paddle/fluid/inference/libpaddle_fluid.a: File truncated
paddle/fluid/inference/CMakeFiles/paddle_fluid.dir/build.make:4981: recipe for target 'paddle/fluid/inference/libpaddle_fluid.a' failed
make[2]: *** [paddle/fluid/inference/libpaddle_fluid.a] Error 1
make[2]: *** Deleting file 'paddle/fluid/inference/libpaddle_fluid.a'
CMakeFiles/Makefile2:58681: recipe for target 'paddle/fluid/inference/CMakeFiles/paddle_fluid.dir/all' failed
make[1]: *** [paddle/fluid/inference/CMakeFiles/paddle_fluid.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
/usr/bin/ar: /home/mgallus/src/Sander/Paddle/build_gpu_deb/paddle/fluid/inference/libpaddle_fluid_origin.a: File truncated
paddle/fluid/inference/CMakeFiles/paddle_fluid_origin.dir/build.make:4909: recipe for target 'paddle/fluid/inference/libpaddle_fluid_origin.a' failed
make[2]: *** [paddle/fluid/inference/libpaddle_fluid_origin.a] Error 1
make[2]: *** Deleting file 'paddle/fluid/inference/libpaddle_fluid_origin.a'
CMakeFiles/Makefile2:59135: recipe for target 'paddle/fluid/inference/CMakeFiles/paddle_fluid_origin.dir/all' failed
make[1]: *** [paddle/fluid/inference/CMakeFiles/paddle_fluid_origin.dir/all] Error 2

@luotao1
Copy link
Contributor

luotao1 commented Dec 7, 2018

There need some times to fix this compile bug.
And if you only want to fix the test_concat_mkldnn_op (which fails the TC build), you can remove following lines, which will not compile inference code:

add_subdirectory(inference)

@Sand3r-
Copy link
Contributor Author

Sand3r- commented Dec 10, 2018

@luotao1 Thank you for the tip, I'll definitely try that 👍

@luotao1
Copy link
Contributor

luotao1 commented Jan 16, 2019

solved by #15109

@luotao1 luotao1 closed this as completed Jan 16, 2019
@Sand3r- Sand3r- reopened this Mar 15, 2019
@kbinias
Copy link
Contributor

kbinias commented Mar 15, 2019

System information:

  • PaddlePaddle version c49e604
  • CPU: SKX 8180, MKLDNN version 0.18
  • GPU: including CUDA 8.0, cuDNN v7
  • OS Platform Ubuntu 16.04
  • Python version 2.7.12

Reproduce:

  • cmake .. -DWITH_GPU=ON -DWITH_PROFILER=ON -DWITH_STYLE_CHECK=OFF -DWITH_MKLDNN=ON -DWITH_TESTING=ON
  • make -j

Result

[ 74%] Linking CXX static library libpaddle_fluid_origin.a
/usr/bin/ar: /home/kbinias/dev/paddle/build/paddle/fluid/inference/libpaddle_fluid_origin.a: File truncated
paddle/fluid/inference/CMakeFiles/paddle_fluid_origin.dir/build.make:5701: recipe for target 'paddle/fluid/inference/libpaddle_fluid_origin.a' failed
make[2]: *** [paddle/fluid/inference/libpaddle_fluid_origin.a] Error 1
make[2]: *** Deleting file 'paddle/fluid/inference/libpaddle_fluid_origin.a'
CMakeFiles/Makefile2:64213: recipe for target 'paddle/fluid/inference/CMakeFiles/paddle_fluid_origin.dir/all' failed
make[1]: *** [paddle/fluid/inference/CMakeFiles/paddle_fluid_origin.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

@luotao1
Copy link
Contributor

luotao1 commented Mar 15, 2019

@kbinias Please use the latest image paddlepaddle/paddle:latest-dev, or you can solve it by #15109.

@lidanqing-intel
Copy link
Contributor

Building with the same command still reports the same error: file truncation.
Removing the inference and training doesn't work.
PaddlePaddle commit: 81b4fad8b94d85a473875ae0efe634fe46697314

image

Related to #15615, we can not reproduce the same results as yours. Thanks

@luotao1
Copy link
Contributor

luotao1 commented Mar 16, 2019

@lidanqing-intel please see #14775 (comment)

@lidanqing-intel
Copy link
Contributor

If this would be searched again, consider update binutils to 2.28.0 or higher version.
To check: ar --version

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants