Raspberry Pi 4 realsense-ros no pointcloud #1324
Comments
|
Hi @yiyinglai If the control_transfer returned warning is being generated continuously, rather than just at the start of launch, then it can indicate a communication problem with the device. If it is generated only once or twice at start-up and then does not repeat, it is safe to ignore the warning if the launch does not exit. I looked at your launch instruction: roslaunch realsense2_camera rs_camera.launch enable_pointcloud:=true initial_reset:=true The term enable_pointcloud:=true has been deprecated (not recommended) for a long time, and filters:=pointcloud is instead recommended. This is the format used by the point cloud example in the RealSense ROS wrapper's documentation: https://github.com/IntelRealSense/realsense-ros#rgbd-point-cloud Changing the point cloud term may not necessarily make a difference, but could you try it please in order to eliminate the possibility that it is an influence on the problem. Thank you. Also, could you expand open the Global Options section of the RViz options panel at the top of it to see what your Rviz has set in the Fixed Frame option, please? |
|
Hi @MartyG-RealSense , thanks for you reply. I tried using the The control_transfer returned warning appeared three times at the beginning only in this case (I left the node running for a dozen minutes; it does not exit by itselt). Publishing rate of Publishing rate of Publishing rate of I tried again with Screenshot of Rviz: Publishing rate of Publishing rate of Publishing rate of |
|
Thanks so much for the detailed report! Are you satisfied with the results please, or do you need further assistance? |
|
Hi @MartyG-RealSense , the problem persisted so I still need further assistance. |
|
It looks as though the color and depth images are being published in RViz. What is the remaining problem please - that there are still control_transfer returned warnings? Or that you are not achieving a point cloud? Thank you. |
|
The problem is that there is no point cloud data in topic |
|
Can you try using this launch term please: roslaunch realsense2_camera rs_camera.launch filters:=pointcloud and setting the Fixed Frame option in RViz to camera_link instead of base_link https://github.com/IntelRealSense/realsense-ros#rgbd-point-cloud |
|
Sure: |
|
Any luck if you Add a topic with the button at the bottom of the RViz options panel? |
|
The topic for The issue is not unable to visualize the point cloud topic in Rviz, but there is no data in point cloud topic. Notice this: |
|
I researched the subject further. In the past there have been problems related to no data in camera/depth/color/points that are apparently related to the kernel. There were a number of cases where the data would not be generated with kernel 5.0. Here are a couple of examples: Given that kernel 5.4 (which it looks as though you are using) is not yet officially supported by librealsense, I could not rule out a problem being related to it. I emphasise that I am not a ROS specialist though. |
|
Thanks for pointing the direction to look into. Can you please keep this issue open? I will try with another kernel version and provide updates afterwards. |
|
I will make a note here about keeping it open. Good luck! |
|
The links @MartyG-RealSense mentions point to a timestamp issue. |
|
Hi @dorodnic , I run two separate terminals and echoed those two topics, and here is what I got: The terminal on the left corresponds to Can you help me interpret this result? Thanks! |
|
@yiyinglai I think you meant to tag @doronhi :). |
|
@MartyG-RealSense , thanks for pointing it out. @doronhi , @dorodnic I'm so sorry about the mistake. |
|
Looking at the timestamps, I noticed that the color stream is published at 15Hz while depth stream is at 30Hz. If it was specified this way it should not have prevented the pointcloud from being created but since this is not what you specified in the launch file (according to the logs you posted previously) I wonder if the overload on your machine is not too great. Could you reduce the frame rate to 15 Hz in both streams and check again? |
|
Indeed, I looked at the publishing frequencies of both topics if specified at 30 Hz. The publishing frequency of depth remains at 30 Hz while that of color dropped from 30 Hz to 17 Hz gradually. I tried specifying both publishing frequencies to be 15 Hz, and here's the result: The publishing frequencies of both topics now remains steady: |
|
I then checked running I found out there is erroneous timestamp even in |
|
@yiyinglai I believe that there are differences between how librealsense handles the camera and how the RealSense ROS wrapper handles it. So in ROS cases, it will often be reported that the camera is working fine in librealsense programs such as the RealSense Viewer, but may be encountering problems with streaming in the ROS wrapper. Also, changes made by the user in librealsense programs such as the Viewer do not affect the camera's behavior in the ROS wrapper, and vice versa. So I would advise against comparing the Viewer with the ROS wrapper like-for-like, as they are "apples and oranges" - not directly comparable because of the different ways in which they work. Your mention of the color FPS dropping from 30 to 17 over time reminds me of a phenomenon that I have heard of before that was related to changes in the environmental lighting causing a drag to be imposed on the camera's FPS. Dorodnic the RealSense SDK Manager has said that FPS can be affected by auto-exposure of RGB if the scene is dark. Is your scene well-lit, please? |
|
@MartyG-RealSense thanks for the explanation. The scene was not dark. I have another laptop beside the Raspberry Pi, and |
|
The issue seems to be related to the timestamp. Could you check if the other timestamps shown in the realsense-viewer behave correctly? Specifically frame-timestamp but take a look at all of them. Maybe you can post 3-4 images showing the progress. We can tell how much really passed according to the progress of "frame counter". In realsense-viewer: "stereo module"->controls->Global Time enabled: change to false. Is the "Frame timestamp" now seem to advance properly? |
|
@doronhi Thanks so much for your reply, we were both answering at the same time. I had just ran some tests of the RGB mode with my own D415 in the Viewer and compared the metadata to the image kindly provided by @yiyinglai . They were almost the same, except in two respects. I was receiving a Hardware FPS and a Backend Timestamp, whilst these values were both zero in the image of @yiyinglai . This does suggest a problem with the metadata.
|
|
Thanks, @MartyG-RealSense. I had the issue you describe once and I seem to remember it was an installation issue. Probably something with kernel patches, but I don't remember. Can you describe how you managed to create that situation and solve it on your machine? |
|
@doronhi @MartyG-RealSense I started building everthing again before you commented. This time with kernel 4.19 since I saw some one manged to use I will update the result afterwards. |
|
@yiyinglai Thanks, we look forward to the results. |
|
@doronhi @MartyG-RealSense Hi, thanks for your patience. I tried several times but still wasn't able to get librealsense or realsense-ros working with kernel 4.19 because opencv3 was not working either (I believe librealsense and realsense-ros has some dependencies on opencv). After switching back to the original setup (I have a backup image, so everthing is exactly the same), I followed @doronhi 's instruciton and disabled This time, the |
|
Hi @yiyinglai Kernel 4.19 is not officially supported by librealsense. After 4.18, the list of officially supported kernels goes to 5.0 and then 5.3. A kernel not being on the list does not necessarily prevent it from working with libreaksense (5.4 works, even though it is not yet officially validated) but I thought that I should mention it. |
|
I managed to install opencv-3.4.2 successfully last night and librealsense with kernel 4.19.
Is |
|
Have you checked whether a point cloud is now visible in RViz, please? I researched your question above carefully. There was a RealSense user with timestamp problems who fixed it by adding the line below to their launch file - no harm in trying it.
Other than that I do not have an answer I would feel confident enough in to provide, so I defer back to @doronhi regarding your query about the effect of the backend timestamp on ROS point clouds. |
|
Hi @MartyG-RealSense , thanks for your comment. The effect of adding I cannot see the point cloud in Rviz, because there is still no data in topic |
|
I did further research and located another case where the backend timestamp was 0 in the Viewer. IntelRealSense/librealsense#5212 The RealSense team member who replied in that case said that the global timestamp is not affected by the state of the backend timestamp. Building with RSUSB backend = on (as you indicated that you have done at the start of this case) gives good metadata support, so it is unusual that you are experiencing this problem with a backend timestamp of 0. The RSUSB installation process is also not dependent on kernel versions or Linux versions and doesn't require patching. I have heard of problems occurring when patching has been applied after an RSUSB backend = On installation was done (because patching is not needed with this method). Can you confirm please whether or not you patched the installation after completing the install with RSUSB, please? If you did patch it, I would recommend doing another build of librealsense with RSUSB but not patching it afterward. |
|
I confirm that I build |
|
Hi
I am also getting a 0 in |
|
Hi @yiyinglai and @korroktheslavemaster The next thing you could try is downgrading your firmware from 5.12.6 to 5.12.5. You can perform this action in the RealSense Viewer. STEP ONE https://dev.intelrealsense.com/docs/firmware-releases STEP TWO
|
|
Hi @MartyG-RealSense I tried And also @korroktheslavemaster any luck on your end? |
|
Hi @yiyinglai Do you require further assistance with this case, please? Thanks! |
|
@MartyG-RealSense Yes, the problem still exists. |
|
Hi @yiyinglai I went through the case step by step from the beginning and made notes, to try to work out the key points of the case so far.
I am wondering if use of the Raspbian OS is one of the complicating factors. Regarding this error in logs posted earlier: An error has occurred during frame callback: Error occurred during execution of the processing block! @doronhi has previously said about it in another case: "From what I understand, the problem is that the filters don't keep up with the frames, due to computer load. A new image cannot be pushed to the still full queue and therefore it is missing when the filter is called to process it. Then the exception is issued". |
|
FYI I have the same issue using the realsense_ros driver (with ros2 eloquent branch) on Raspberry Pi 4 using Rasbian Buster OS as discussed in the case here intel/ros2_intel_realsense#147 (as you can see from the comments, we switched from ros2 driver to this driver's ros2 branch based on suggestion from @doronhi ). It remains unresolved, but I was able to unblock myself by writing my own realsense node based on librealsense2, which is working fine (I used the rs-pointcloud as a starting point.) I would prefer to be using the official driver and all the configuration options it exposes, so I am still interested in a resolution. |
|
@yiyinglai and @justinIRBT Can you confirm that the problem still exists for you? Thanks very much. |
|
Yes, it still exists |
|
I have been over the entire case from the beginning again but have not been able to find much that has not already been looked at or linked to. Possibly one of the existing cases that has already been looked at has a useful clue if looked at with a fresh perspective. I did find one case with similar symptoms to this one (especially in regard to 'no new messages' from the /camera/depth/color/points topic) that has not been linked to in this discussion yet. |
|
Hi @MartyG-RealSense , I am using the camera with a laptop and probably wouldn't use it with Raspberry Pi 4 afterwards. Thanks for your help. Please feel free to close this issue. |
|
Okay, thanks very much @yiyinglai |
|
I am also encountering this issue, and I believe I can contribute in the debugging of it: I think that the pointcloud filter is doing something similar to the align_depth filter. That filter runs at around 2-3 FPS on a raspberry pi, and as a result, we see the queue backing up & preventing the system from making progress as @doronhi commented above. |
|
I am also encountering this issue, that is: it works fine when opened with I use Raspberry Pi 4B 8GB, ROS Noetic, Ubuntu 20.04. Kernel 5.4.0-1038-raspi. The camera is Intel RealSense D415. My workaround was decreasing the fps in the launch file. For the argument tldr; it works when I decrease the fps to 6. |
|
Thanks very much @dispectra for sharing the details of the workaround method that worked for you! I would add that if users of the D455 camera model wish to try your method in future, they should set FPS to 5, as 6 is not supported as an FPS mode on D455. Thanks again! |
System: Linux raspberrypi 5.4.51-v7l+ #1327 SMP Thu Jul 23 11:04:39 BST 2020 armv7l GNU/Linux
OS: Raspbian Buster
ROS distro: kinetic
Camera: D415
1) Name: Intel RealSense D415, serial number: 950122061295, update serial number: 948123020797, firmware version: 05.12.06.00, USB type: 3.2USB cable: official cable
librealsense 2.36.0 built with
cmake ../ -DCMAKE_BUILD_TYPE=Release -DBUILD_EXAMPLES=true -DFORCE_RSUSB_BACKEND=ON -DBUILD_WITH_TM2=false -DIMPORT_DEPTH_CAM_FW=false(firmware is already latest)realsense-ros 2.2.15
RGB image and point cloud in realsense-viewer:
When using realsense-ros:
Verified that no pointcloud data is published.
RGB image and depth image are published and can be visualized in Rviz.
Also tried specifying
initial_resetargument.The text was updated successfully, but these errors were encountered: