Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test_PyDataProvider2 failed due to segfault #430

Closed
yu239-zz opened this issue Nov 11, 2016 · 2 comments
Closed

test_PyDataProvider2 failed due to segfault #430

yu239-zz opened this issue Nov 11, 2016 · 2 comments
Assignees
@reyoung
Labels
Bug

Comments

@yu239-zz
Copy link

[idl57-Alienware-Area-51-R2]haonanyu:~/paddle/build(ret:8)$ PYTHONPATH= ctest -R DataProvider2 -V
UpdateCTestConfiguration from :/home/haonanyu/paddle/build/DartConfiguration.tcl
UpdateCTestConfiguration from :/home/haonanyu/paddle/build/DartConfiguration.tcl
Test project /home/haonanyu/paddle/build
Constructing a list of tests
Done constructing a list of tests
Checking test dependency graph...
Checking test dependency graph end
test 28
Start 28: test_PyDataProvider2

28: Test command: /home/haonanyu/paddle/paddle/.set_python_path.sh "-d" "/home/haonanyu/paddle/paddle/gserver/tests:/home/haonanyu/paddle/python" "/home/haonanyu/paddle/build/paddle/gserver/tests/test_PyDataProvider2"
28: Test timeout computed to be: 9.99988e+06
28: + getopts d: opt
28: + case $opt in
28: + PYPATH=/home/haonanyu/paddle/paddle/gserver/tests:/home/haonanyu/paddle/python
28: + getopts d: opt
28: + shift 2
28: + export PYTHONPATH=/home/haonanyu/paddle/paddle/gserver/tests:/home/haonanyu/paddle/python:
28: + PYTHONPATH=/home/haonanyu/paddle/paddle/gserver/tests:/home/haonanyu/paddle/python:
28: + /home/haonanyu/paddle/build/paddle/gserver/tests/test_PyDataProvider2
28: WARNING: Logging before InitGoogleLogging() is written to STDERR
28: I1110 17:27:47.987854 27631 hl_dso_loader.cc:51] Try to find cuda library: libcudart.so from default system path.
28: I1110 17:27:47.988507 27631 Util.cpp:155] commandline: /home/haonanyu/paddle/build/paddle/gserver/tests/test_PyDataProvider2
28: I1110 17:27:48.128258 27631 hl_dso_loader.cc:51] Try to find cuda library: libcublas.so from default system path.
28: I1110 17:27:48.208549 27631 hl_dso_loader.cc:51] Try to find cuda library: libcurand.so from default system path.
28: I1110 17:27:48.217546 27631 hl_dso_loader.cc:51] Try to find cuda library: libcudnn.so from default system path.
28: I1110 17:27:50.104863 27631 Util.cpp:130] Calling runInitFunctions
28: I1110 17:27:50.105031 27631 Util.cpp:143] Call runInitFunctions done.
28: File "", line 11
28: paddle version:
28: ^
28: SyntaxError: invalid syntax
28: [==========] Running 11 tests from 1 test case.
28: [----------] Global test environment set-up.
28: [----------] 11 tests from PyDataProvider2
28: [ RUN ] PyDataProvider2.dense_no_seq
28: I1110 17:27:50.160810 27631 PyDataProvider2.cpp:257] loading dataprovider test_PyDataProvider2::test_dense_no_seq
28: [ OK ] PyDataProvider2.dense_no_seq (104 ms)
28: [ RUN ] PyDataProvider2.index_no_seq
28: I1110 17:27:50.216799 27631 PyDataProvider2.cpp:257] loading dataprovider test_PyDataProvider2::test_index_no_seq
28: [ OK ] PyDataProvider2.index_no_seq (1 ms)
28: [ RUN ] PyDataProvider2.init_hook
28: I1110 17:27:50.218468 27631 PyDataProvider2.cpp:257] loading dataprovider test_PyDataProvider2::test_init_hook
28: [ OK ] PyDataProvider2.init_hook (2 ms)
28: [ RUN ] PyDataProvider2.sparse_no_value_no_seq
28: I1110 17:27:50.219805 27631 PyDataProvider2.cpp:257] loading dataprovider test_PyDataProvider2::test_sparse_non_value_no_seq
28: [ OK ] PyDataProvider2.sparse_no_value_no_seq (1 ms)
28: [ RUN ] PyDataProvider2.sparse_value_no_seq
28: I1110 17:27:50.220360 27631 PyDataProvider2.cpp:257] loading dataprovider test_PyDataProvider2::test_sparse_value_no_seq
28: *** Aborted at 1478827670 (unix time) try "date -d @1478827670" if you are using GNU date ***
28: PC: @ 0x7f22cf507bf8 (unknown)
28: *** SIGSEGV (@0xa9) received by PID 27631 (TID 0x7f2266550700) from PID 169; stack trace: ***
28: @ 0x7f22cff34340 (unknown)
28: @ 0x7f22cf507bf8 (unknown)
28: @ 0x7f22cf5174be (unknown)
28: @ 0x7f22cf4a18b7 (unknown)
28: @ 0x7f22cf4a24eb (unknown)
28: @ 0x7f22cf4a2556 (unknown)
28: @ 0x7f22cf4e943f (unknown)
28: @ 0x7f22cf4caf7e (unknown)
28: @ 0x7f22cf504153 (unknown)
28: @ 0x7f22cf4cbe20 (unknown)
28: @ 0x7f22cf504153 (unknown)
28: @ 0x7f22cf41347b (unknown)
28: @ 0x65c6cb paddle::PyDataProvider2::loadThread()
28: @ 0x7f22cec96a60 (unknown)
28: @ 0x7f22cff2c182 start_thread
28: @ 0x7f22ce3fe47d (unknown)
28: @ 0x0 (unknown)
28: /home/haonanyu/paddle/paddle/.set_python_path.sh: line 42: 27631 Segmentation fault (core dumped) $@
1/1 Test #28: test_PyDataProvider2 .............***Failed 2.58 sec

0% tests passed, 1 tests failed out of 1

Total Test time (real) = 2.58 sec

The following tests FAILED:
28 - test_PyDataProvider2 (Failed)
Errors while running CTest
@backyes backyes assigned emailweixu and reyoung and unassigned emailweixu Nov 11, 2016
@reyoung
Copy link
Collaborator

reyoung commented Nov 11, 2016

Could you use debug mode to rerun this test for a better callstack?

To build a debug version, you need set CMAKE_BUILD_TYPE=Debug when build paddle by cmake

@yu239-zz
Copy link
Author

30: I1110 19:30:14.924366 17254 Util.cpp:155] commandline: /home/haonanyu/paddle/build/paddle/gserver/tests/test_PyDataProvider2
30: I1110 19:30:17.291312 17254 Util.cpp:130] Calling runInitFunctions
30: I1110 19:30:17.291770 17254 Util.cpp:143] Call runInitFunctions done.
30: [==========] Running 12 tests from 1 test case.
30: [----------] Global test environment set-up.
30: [----------] 12 tests from PyDataProvider2
30: [ RUN ] PyDataProvider2.dense_no_seq
30: I1110 19:30:17.353341 17254 PyDataProvider2.cpp:257] loading dataprovider test_PyDataProvider2::test_dense_no_seq
30: [ OK ] PyDataProvider2.dense_no_seq (117 ms)
30: [ RUN ] PyDataProvider2.index_no_seq
30: I1110 19:30:17.419425 17254 PyDataProvider2.cpp:257] loading dataprovider test_PyDataProvider2::test_index_no_seq
30: [ OK ] PyDataProvider2.index_no_seq (1 ms)
30: [ RUN ] PyDataProvider2.init_hook
30: I1110 19:30:17.422075 17254 PyDataProvider2.cpp:257] loading dataprovider test_PyDataProvider2::test_init_hook
30: [ OK ] PyDataProvider2.init_hook (4 ms)
30: [ RUN ] PyDataProvider2.sparse_no_value_no_seq
30: I1110 19:30:17.424038 17254 PyDataProvider2.cpp:257] loading dataprovider test_PyDataProvider2::test_sparse_non_value_no_seq
30: [ OK ] PyDataProvider2.sparse_no_value_no_seq (0 ms)
30: [ RUN ] PyDataProvider2.sparse_value_no_seq
30: I1110 19:30:17.424867 17254 PyDataProvider2.cpp:257] loading dataprovider test_PyDataProvider2::test_sparse_value_no_seq
30: *** Aborted at 1478835017 (unix time) try "date -d @1478835017" if you are using GNU date ***
30: PC: @ 0x7f1d85046bf8 (unknown)
30: *** SIGSEGV (@0xa9) received by PID 17254 (TID 0x7f1d1c7a9700) from PID 169; stack trace: ***
30: @ 0x7f1d85a73340 (unknown)
30: @ 0x7f1d85046bf8 (unknown)
30: @ 0x7f1d850564be (unknown)
30: @ 0x7f1d84fe08b7 (unknown)
30: @ 0x7f1d84fe14eb (unknown)
30: @ 0x7f1d84fe1556 (unknown)
30: @ 0x7f1d8502843f (unknown)
30: @ 0x7f1d85009f7e (unknown)
30: @ 0x7f1d85043153 (unknown)
30: @ 0x7f1d8500ae20 (unknown)
30: @ 0x7f1d85043153 (unknown)
30: @ 0x7f1d84f5247b (unknown)
30: @ 0x9cf62c paddle::py::iterNext()
30: @ 0x9d4143 paddle::PyDataProvider2::loadThread()
30: @ 0x9d49a1 _ZZN6paddle15PyDataProvider29resetImplEbENKUlvE_clEv
30: @ 0x9e257c _ZNSt12_Bind_simpleIFZN6paddle15PyDataProvider29resetImplEbEUlvE_vEE9_M_invokeIJEEEvSt12_Index_tupleIJXspT_EEE
30: @ 0x9e24d3 _ZNSt12_Bind_simpleIFZN6paddle15PyDataProvider29resetImplEbEUlvE_vEEclEv
30: @ 0x9e246c _ZNSt6thread5_ImplISt12_Bind_simpleIFZN6paddle15PyDataProvider29resetImplEbEUlvE_vEEE6_M_runEv
30: @ 0x7f1d847d5a60 (unknown)
30: @ 0x7f1d85a6b182 start_thread
30: @ 0x7f1d83f3d47d (unknown)
30: @ 0x0 (unknown)
30: /home/haonanyu/paddle/paddle/.set_python_path.sh: line 42: 17254 Segmentation fault (core dumped) $@
1/1 Test #30: test_PyDataProvider2 .............***Failed 2.88 sec

This is what I got after set Debug flag.

@reyoung reyoung added the Bug label Nov 11, 2016
@qingqing01 qingqing01 added this to the 0.10.0 milestone Nov 11, 2016
@reyoung reyoung removed this from the 0.10.0 milestone May 3, 2017
zhhsplendid pushed a commit to zhhsplendid/Paddle that referenced this issue Sep 25, 2019
@typhoonzero @shanyi15
Update nccl2 tips (PaddlePaddle#430)
468c82a 
* update nccl2 dist train tips

* update

* update

* update

* update

* follow comments
thisjiang pushed a commit to thisjiang/Paddle that referenced this issue Oct 28, 2021
@wenming2014
add load_params test and opt (PaddlePaddle#430)
e347c22 
Co-authored-by: wangyue50 <wangyue50@baidu.com>
gglin001 added a commit to graphcore/Paddle-fork that referenced this issue Mar 17, 2022
@gglin001
add ipu_build_use_ipumodel (PaddlePaddle#430)
2807880 
* add ipu_build_use_ipumodel

* up 0

* fix using ipumodels

* disable real ipu ci

* add use_ipumodel flag for ut

* restore ci

* up x
lizexu123 pushed a commit to lizexu123/Paddle that referenced this issue Feb 23, 2024
@yukavio
Complete prune load api (PaddlePaddle#430)
a412b6f 
* add new argument for easy load prune program

* add argument make load pruned program more easier

* change save function arg

* fix test

* fix format
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@reyoung reyoung

Labels
Bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@Superjomn @reyoung @yu239-zz @qingqing01 @emailweixu