Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

demo.py crashes scitos_node #3

Closed
bfalacerda opened this issue Nov 13, 2013 · 11 comments
Closed

demo.py crashes scitos_node #3

bfalacerda opened this issue Nov 13, 2013 · 11 comments

Comments

@bfalacerda
Copy link
Contributor

When I try running rosrun marathon_touch_gui demo.py on Bob, scitos_bringup.launch gives an error on scitos_node. Here are two different errors messages I got in 2 different tries:

erialCommS300: Checksum's dont match, thats bad (data packet size 1104)
SerialCommS300: Checksum's dont match, thats bad (data packet size 1104)
[ WARN] [1384377596.310758702]: (MIRA) Invalid CRC on MetraLabs CAN-to-Serial Adapter. Is 0xf467 should 0x2e65
[WARNING ] 2013-Nov-13 21:19:55.787731 Invalid CRC on MetraLabs CAN-to-Serial Adapter. Is 0xf467 should 0x2e65
[ WARN] [1384377596.480050293]: (MIRA) Invalid message length on MetraLabs CAN-to-Serial Adapter
[WARNING ] 2013-Nov-13 21:19:56.479592 Invalid message length on MetraLabs CAN-to-Serial Adapter
[ WARN] [1384377596.483879377]: (MIRA) Invalid CRC on MetraLabs CAN-to-Serial Adapter. Is 0xf96a should 0x4d05
[WARNING ] 2013-Nov-13 21:19:56.483843 Invalid CRC on MetraLabs CAN-to-Serial Adapter. Is 0xf96a should 0x4d05
[ WARN] [1384377596.490029568]: (MIRA) Invalid CRC on MetraLabs CAN-to-Serial Adapter. Is 0x3925 should 0x62ca
[WARNING ] 2013-Nov-13 21:19:56.489963 Invalid CRC on MetraLabs CAN-to-Serial Adapter. Is 0x3925 should 0x62ca
terminate called after throwing an instance of 'boost::exception_detail::clone_implmira::XRPC'
what(): Failed to read SDO node(0x08) index(0x2006) subindex (0x01) (/home/chmartin/release/MIRA-commercial/toolboxes/CAN/include/can/CANOpenSDOClient.h:134)
[ERROR] [1384377599.812173608]: (MIRA) Aborted!
[ERROR ] 2013-Nov-13 21:19:57.657012 Aborted!
[CALLSTACK]:
#0 0x7f56682bb425 in gsignal from /lib/x86_64-linux-gnu/libc.so.6
#1 0x7f56682beb8b in abort from /lib/x86_64-linux-gnu/libc.so.6
#2 0x7f5668c0e69d in __gnu_cxx::__verbose_terminate_handler() from libstdc++.so.6
#3 0x7f5668c0c846 in from libstdc++.so.6
#4 0x7f5668c0c873 in from libstdc++.so.6
#5 0x7f5668c0c96e in from libstdc++.so.6
#6 0x4978f2 in boost::exception_detail::clone_implmira::XRPC::rethrow() const from scitos_node
#7 0x491fc4 in from scitos_node

at ScitosModule.cpp:0

#8 0x493398 in ScitosModule::get_mira_param_(std::string) from scitos_node
#9 0x538525 in ScitosHead::publish_joint_state_actual() from scitos_node
#10 0x4a8e08 in ScitosG5::spin() from scitos_node
#11 0x487df8 in main from scitos_node
#12 0x7f56682a676d in __libc_start_main from /lib/x86_64-linux-gnu/libc.so.6
#13 0x48961d in from scitos_node

[scitos_node-2] process has died [pid 15739, exit code -6, cmd /opt/strands/strands_catkin_ws/devel/lib/scitos_mira/scitos_node __name:=scitos_node __log:=/home/bruno/.ros/log/e36107e8-4ca8-11e3-80fe-843a4b7f1c04/scitos_node-2.log].
log file: /home/bruno/.ros/log/e36107e8-4ca8-11e3-80fe-843a4b7f1c04/scitos_node-2*.log

SerialCommS300: Checksum's dont match, thats bad (data packet size 1104)
SerialCommS300: Checksum's dont match, thats bad (data packet size 1104)
[ERROR] [1384378451.304066233]: (MIRA) /robot/can/CANDriver Recovering: : Incoming message watchdog triggered. No incoming messages.
[ERROR ] 2013-Nov-13 21:34:10.611378 /robot/can/CANDriver Recovering: : Incoming message watchdog triggered. No incoming messages.
terminate called after throwing an instance of 'boost::exception_detail::clone_implmira::XRPC'
what(): Failed to read SDO node(0x08) index(0x2006) subindex (0x01) (/home/chmartin/release/MIRA-commercial/toolboxes/CAN/include/can/CANOpenSDOClient.h:134)
[ERROR] [1384378464.511328469]: (MIRA) Aborted!
[ERROR ] 2013-Nov-13 21:34:17.800850 Aborted!
[CALLSTACK]:
#0 0x7fa790a9c425 in gsignal from /lib/x86_64-linux-gnu/libc.so.6
#1 0x7fa790a9fb8b in abort from /lib/x86_64-linux-gnu/libc.so.6
#2 0x7fa7913ef69d in __gnu_cxx::__verbose_terminate_handler() from libstdc++.so.6
#3 0x7fa7913ed846 in from libstdc++.so.6
#4 0x7fa7913ed873 in from libstdc++.so.6
#5 0x7fa7913ed96e in from libstdc++.so.6
#6 0x4978f2 in boost::exception_detail::clone_implmira::XRPC::rethrow() const from scitos_node
#7 0x491fc4 in from scitos_node

at ScitosModule.cpp:0

#8 0x493398 in ScitosModule::get_mira_param_(std::string) from scitos_node
#9 0x538525 in ScitosHead::publish_joint_state_actual() from scitos_node
#10 0x4a8e08 in ScitosG5::spin() from scitos_node
#11 0x487df8 in main from scitos_node
#12 0x7fa790a8776d in __libc_start_main from /lib/x86_64-linux-gnu/libc.so.6
#13 0x48961d in from scitos_node

[scitos_node-2] process has died [pid 27618, exit code -6, cmd /opt/strands/strands_catkin_ws/devel/lib/scitos_mira/scitos_node __name:=scitos_node __log:=/home/bruno/.ros/log/f4fd320e-4caa-11e3-ab52-843a4b7f1c04/scitos_node-2.log].
log file: /home/bruno/.ros/log/f4fd320e-4caa-11e3-ab52-843a4b7f1c04/scitos_node-2*.log

The last messages given by roslaunch marathon_touch_gui marathon_gui_dependencies.launch are

[INFO] [WallTime: 1384377932.887379] Client connected. 2 clients total.
localhost - - [13/Nov/2013 21:25:32] "GET /bootstrap/docs-assets/ico/favicon.png HTTP/1.1" 200 -
[INFO] [WallTime: 1384377932.988054] [Client 1] Subscribed to /map
[INFO] [WallTime: 1384377934.557374] Client disconnected. 1 clients total.
localhost - - [13/Nov/2013 21:25:38] "GET /bootstrap/dist/css/bootstrap.css HTTP/1.1" 200 -
localhost - - [13/Nov/2013 21:25:38] "GET /js/jquery.js HTTP/1.1" 200 -
localhost - - [13/Nov/2013 21:25:38] "GET /js/display.js HTTP/1.1" 200 -
localhost - - [13/Nov/2013 21:25:38] "GET /js/jquery-ui.js HTTP/1.1" 200 -
localhost - - [13/Nov/2013 21:25:38] "GET /roslibjs/include/EventEmitter2/eventemitter2.js HTTP/1.1" 200 -
localhost - - [13/Nov/2013 21:25:38] "GET /roslibjs/build/roslib.js HTTP/1.1" 200 -
localhost - - [13/Nov/2013 21:25:38] "GET /ros2djs/include/EaselJS/easeljs.js HTTP/1.1" 200 -
localhost - - [13/Nov/2013 21:25:38] "GET /ros2djs/build/ros2d.js HTTP/1.1" 200 -
localhost - - [13/Nov/2013 21:25:38] "GET /ros3djs/include/threejs/three.js HTTP/1.1" 200 -
localhost - - [13/Nov/2013 21:25:38] "GET /ros3djs/include/ColladaAnimationCompress/ColladaLoader2.js HTTP/1.1" 200 -
localhost - - [13/Nov/2013 21:25:38] "GET /ros3djs/build/ros3d.min.js HTTP/1.1" 200 -
localhost - - [13/Nov/2013 21:25:38] "GET /nav2djs/build/nav2d.js HTTP/1.1" 200 -
localhost - - [13/Nov/2013 21:25:38] "GET /bootstrap/dist/js/bootstrap.js HTTP/1.1" 200 -
[INFO] [WallTime: 1384377938.989889] Client connected. 2 clients total.
localhost - - [13/Nov/2013 21:25:38] "GET /bootstrap/docs-assets/ico/favicon.png HTTP/1.1" 200 -
[INFO] [WallTime: 1384377939.008112] [Client 2] Subscribed to /map

It looks like the problem is trying to access the map. I also noticed that the map only appears on the browser after the scitos_node dies.
@hawesie
Copy link
Member

hawesie commented Nov 13, 2013

wtf!

Can you narrow down what call in demo.py is the problem? The map is called from the main page via rosbridge. @marc-hanheide have you run rosbridge on your robot? Does any of this look familiar?

@hawesie
Copy link
Member

hawesie commented Nov 13, 2013

Could it be related to strands-project/scitos_robot#10

@marc-hanheide
Copy link
Member

that was my suspicion as well. That somehow you changed the system time... No clue how though. I've not seem this in the context of rosbridge.

@bfalacerda
Copy link
Contributor Author

I'll do proper debugging tomorrow and tell you something then

@hawesie
Copy link
Member

hawesie commented Nov 14, 2013

I'll try to get to the lab around 3.30 and join in.

@bfalacerda
Copy link
Contributor Author

the scitos_node crashes only if:

  1. scitos_2d_nav.launch has been launched (i.e., there is a map being published), and
  2. the main page is displayed for more than some seconds. If it's displayed for 2 seconds and then another page is displayed (bumper recover page for example) it's ok, but if it's displayed for 10 seconds, scitos_node crashes

@hawesie
Copy link
Member

hawesie commented Nov 14, 2013

Do you ever see the map displayed? Is this all under the bob account for me to try shortly?

@hawesie
Copy link
Member

hawesie commented Nov 14, 2013

The problem diagnosed by @cburbridge and @bfalacerda is that the huge CPU and memory usage of displaying the map in the browser causes the bus to get confused and die. I will take the map out of the GUI for now so that it can still run.

@hawesie
Copy link
Member

hawesie commented Nov 15, 2013

Raising the priority of the scitos_node process did not prevent this crash.

@hawesie
Copy link
Member

hawesie commented Nov 15, 2013

I will crop the map (currently it's 4k x 4k) to see if that improves things (we think it will). If it does I suggest we close this and move it to scitos_robot or similar as a bug where high CPU load crashes the scitos_node.

@hawesie
Copy link
Member

hawesie commented Nov 15, 2013

A cropped map fixes this. It's not a long term solution, but it works for now.

@hawesie hawesie closed this as completed Nov 15, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@hawesie @marc-hanheide @bfalacerda