Getting RGB information from Pepper depth camera

[TuyenNguyenTanViet]

It was clear that we easily can get depth information from Pepper Xtion camera , in addtion to depth information, I also need RGB information for my further process, is it possible to get RGB inforation as well?some thing like this picture

[suryaambrose]

Hi,
AFAIK it is not possible, if I remember well the RGB is deactivated in the driver used on the robot. I don't think there is an easy way to reactivate it.

[TuyenNguyenTanViet]

@suryaambrose thank for your answer, if RGB is impossible to obtained from Pepper depth camera, is there anyway to combine RGB image from Pepper front camera with depth camera?

[suryaambrose]

What do you mean by "combine" ?

[suryaambrose]

If you mean "have a RGBD image" yes it is possible (RViz does it) but I don't really know how and it won't be as pretty as if both images were coming from the same camera.
You might want to look towards ROS's image pipeline :)

[TuyenNguyenTanViet]

thank @suryaambrose , I need RGB+D image because I need RGB image for my processing and Depth information (corresponding to each pixel in RGB image) for getting depth imformation

[nlyubova]

Hello,
you can get a colored pointcloud with pepper_bringup (http://wiki.ros.org/pepper_bringup) package that actually depends on this naoqi_driver package.

Once installed, try to get/visualize in RviZ this topic: /pepper_robot/camera/depth_registered/points

In case if you cannot visualize this topic, then uncomment lines 197 and 198 https://github.com/ros-naoqi/naoqi_driver/blob/master/src/converters/camera_info_definitions.hpp#L197 then compile the naoqi_driver and source it

Natalia

[TuyenNguyenTanViet]

Hello Natalia,
thank you for your answer, by uncommented L 197 and 198 https://github.com/ros-naoqi/naoqi_driver/blob/master/src/converters/camera_info_definitions.hpp#L197, the color point cloud can be obtained.

[TuyenNguyenTanViet]

Hello, @nlyubova , I have one more question, after getting the color point cloud, I had separate it into RGB and depth image. However, it turns out that the position of RGB pixels and Depth (x,y,z) pixels were not associated to each other

For example, the red dot now located in the center of this RGB image, however, its position in the Depth image is not at the center anymore.

I guest that because the color point cloud produced by the combination of RGB image (possibly from front camera) and Depth image (from 3D camera). However,its locations (cameras) are different, that why the its pixels are not associated to each other, is there anyway to calibrate it?

Hope to receive your feedback
Tuyen

[nlyubova]

Hello,
which topics do you use? you need to check the .../depth_registered/...

In case if "../depth_registered/.." are not ok, then you can do two things:

• intrinsic calibration -> try http://wiki.ros.org/camera_calibration -> and when done, update the parameters in https://github.com/ros-naoqi/naoqi_driver/blob/master/src/converters/camera_info_definitions.hpp
• extrinsic calibration between the color and depth cameras -> check this package: https://github.com/nlyubova/extrinsic_calibration ; you will need to record images from RGB and IR cameras and then to calibrate based on recorded images (see README)

Natalia

[nlyubova]

@TuyenNguyenTanViet did you manage to make it working? can I close the issue?

[TuyenNguyenTanViet]

Hi @suryaambrose , thank you very much, actually I still have some problems which are not fixed yet. That why I came up with tentative solution by manually calibrate color and depth camera.
Could you please still keep it open , I want to try your calibration solution and get back to you as soon as I can
Tuyen

[metalaman]

@nlyubova Hi, I tried visualizing the point cloud, but can't see anything, also followed the uncommenting steps, no luck.
Using ROS Kinetic, topic is also the same as mentioned above. I'm able to see the raw_image and depth_image. Please help.

[metalaman]

Working now. Thanks.

[nlyubova]

Ok, then Can we close this issue?

[iamrgroot]

@metalaman What did you do to get it working eventually?

[metalaman]

@Robert16296 Try changing the frame to Base Link in rviz.

[kamalpy]

@metalaman Hi Aman, I have tried changing frame in rviz but I am still not able to view the points.

[lahiruherath]

Does anyone have a concrete answer on why the point cloud topic is not publishing and how to get it fixed

[dac31415]

Anyone has been lucky to make this work? is it me or this don't work for ros melodic? This is the issue I'm having.
https://answers.ros.org/question/386755/no-image-display-from-depth-camera-pepper/

[mbusy]

Hi @dac31415, I checked your topic on ros answers, regarding that comment:

Also, if I uncomment the lines 197, 198 can't compile anymore.

I would advise you to check out that comment. Not sure that it will solve your problems but it might help

[dac31415]

Hi @mbusy, I will check that. Thank you. I tried to check the cameras individually and it appears that for some reason are named the same. I used rosrun image_view image_view and then the topic. First, I run the normal camera, which topic is /camera/image_raw and then tried to open the depth with the following topic /pepper_robot/camera/depth/camera/image_raw and the following warning appears in the first terminal:
[WARN] [1632491798.998864011]: Shutdown request received.
[ WARN] [1632491798.998936083]: Reason given for shutdown: [[/image_view] Reason: new node registered with same name]

Does this means that is just connecting one at a time in the same node? Also,m I'm working on a virtual machine VMware. Does that might be affecting to not connect properly to the robot?

[mbusy]

You're welcome. I would probably try and use rqt_image_view to visualize the images instead, switching between 2 different image topics might be more convenient. I absolutely don't know if VMware has a role to play in the obtention of the warning you mentioned. Regarding the connection to a robot, I'm guessing that it should work.

One thing I'm sure of is that creating a new node having the same exact name than an already running node will result in that kind of error. If you want to simultaneously view the RGB and depth images, RViz might be more adapted

[UdayRockzz]

Hi @dac31415 , Did you solve this issue?