No stream match for pointcloud chosen texture Process - Color

[kenloumixx]

Hello,
thank you for this marvelous work :)

I use ROS2 galactic in ubuntu 18.04 docker container.

I followed your guideline with a slight modification and I encountered the error that says
No stream match for pointcloud chosen texture Process - Color

The dockerfile lines that I used is as below:

RUN apt-get install kmod && \
    cd ~ && \
    wget https://github.com/IntelRealSense/librealsense/archive/refs/tags/v2.51.1.zip && \
    unzip v2.51.1.zip && \
    cd ~/librealsense-2.51.1 && \
    ./scripts/patch-realsense-ubuntu-L4T.sh  && \
    apt-get install git libssl-dev libusb-1.0-0-dev libudev-dev pkg-config libgtk-3-dev -y && \
    ./scripts/setup_udev_rules.sh  && \
    mkdir build && cd build  && \
    cmake .. -DCMAKE_C_FLAGS_RELEASE="${CMAKE_C_FLAGS_RELEASE} -s" -DCMAKE_CXX_FLAGS_RELEASE="${CMAKE_CXX_FLAGS_RELEASE} -s" -DCMAKE_INSTALL_PREFIX=/opt/librealsense  -DBUILD_PYTHON_BINDINGS:bool=true  -DBUILD_EXAMPLES=true -DCMAKE_BUILD_TYPE=release -DFORCE_RSUSB_BACKEND=false -DBUILD_WITH_CUDA=true && make -j$(($(nproc)-1)) && sudo make install && \
    
    cd ~ && \
    wget https://github.com/IntelRealSense/realsense-ros/archive/refs/tags/4.51.1.zip && \
    apt-get install python3-rosdep -y && \
    unzip 4.51.1.zip && \
    cd realsense-ros-4.51.1 && \
    rm /etc/ros/rosdep/sources.list.d/20-default.list && \
    rosdep init && rosdep update && rosdep install -i --from-path src --rosdistro $ROS_DISTRO --skip-keys=librealsense2 -y && \
    colcon build --symlink-install && \
    . install/local_setup.zsh

and I use the docker run command refering to yours

docker run --name jetson-tmp5 -d -it --runtime nvidia -e "ACCEPT_EULA=Y" --ipc=host --privileged --network=host \
-v $(pwd)/..:/workspace:rw \
-v /tmp/.X11-unix:/tmp/.X11-unix \
-e DISPLAY=$DISPLAY \
-v /dev:/dev \
-v /usr/src:/usr/src \
-v /etc/nv_tegra_release:/etc/nv_tegra_release \
-v /etc/udev/rules.d:/etc/udev/rules.d \
-v /lib/modules/4.9.140-tegra:/lib/modules/4.9.140-tegra \
--device-cgroup-rule='c 81:* rmw' \
--device-cgroup-rule='c 189:* rmw' \
kenloumixx/jetson-rs

In installing librealsense and realsense-ros, I also referred to the guideline described in the README.md

May I ask which part is wrong?
TIA

[2b-t]

Hi @kenloumixx
You are welcome! Glad it helps!
From what I know the warning No stream match for pointcloud chosen texture Process - Color indicates dropped frames. I have got a couple of questions:

• Does the message occur occasionally or is it fairly frequent?
• Can you still display the images (e.g. in rqt_image_view) and point clouds (in RViz)? Or similarly can you use $ ros2 topic hz <topic> to see at which rate they are published?
• Are you using the cable that the Realsense came with? Are you connecting it to a USB 3.x port? Not using a suitable cable is known to result in said error.
• What is the output of rs-enumerate-devices --compact?
• Are you able to display the pointcloud in realsense-viewer?
• Which Jetson are you using? I have a Jetson Xavier AGX that I can try to replicate this on.

[kenloumixx]

@2b-t thank you for response :)
I use librealsense-2.51.1 and realsense-ros-4.51.1 wrapper and followed the installation guideline in these links(librealsense, realsense-ros) and your README.md to build it from source.

I run the command ros2 launch realsense2_camera rs_launch.py initial_reset:=true pointcloud.enable:=true align_depth.enable:=true ordered_pc:=true

• Does the message occur occasionally or is it fairly frequent?
  -> Always. If I run the command ros2 launch realsense2_camera rs_launch.py initial_reset:=true align_depth.enable:=true, the RGB image appears in rviz2. But when I add pointcloud.enable:=true and ordered_pc:=true, RGB image does not appears in rviz2 and the error No stream match for pointcloud chosen texture Process - Color appears only once.

• Can you still display the images (e.g. in rqt_image_view) and point clouds (in RViz)? Or similarly can you use $ ros2 topic hz to see at which rate they are published?
  -> No, I can see depth images(/camera/aligned_depth_to_color/image_raw) in rviz but RGB image(/camera/depth/color/image_raw) does not appear, instead 'No Image' appears and pointcloud(/camera/depth/color/points) is not visible.

❯ ros2 topic hz /camera/depth/image_rect_raw
average rate: 9.410
	min: 0.102s max: 0.108s std dev: 0.00156s window: 11
average rate: 9.432
	min: 0.102s max: 0.109s std dev: 0.00193s window: 21


❯ ros2 topic hz /camera/color/image_raw (when available)
average rate: 3.977
	min: 0.194s max: 0.303s std dev: 0.03533s window: 5
average rate: 4.010
	min: 0.194s max: 0.303s std dev: 0.02907s window: 14

• Are you using the cable that the Realsense came with? Are you connecting it to a USB 3.x port? Not using a suitable cable is known to result in said error.
  -> Yes, it is connected to a USB 3.2 port.

• What is the output of rs-enumerate-devices --compact?
  -> The output is as below

❯ rs-enumerate-devices --compact
Device Name                   Serial Number       Firmware Version    
Intel RealSense D435I         117322074667        05.12.14.50         
Device info: 
    Name                          : 	Intel RealSense D435I
    Serial Number                 : 	117322074667
    Firmware Version              : 	05.12.14.50
    Recommended Firmware Version  : 	05.12.14.50
    Physical Port                 : 	/sys/devices/3610000.xhci/usb2/2-3/2-3.3/2-3.3:1.0/video4linux/video0
    Debug Op Code                 : 	15
    Advanced Mode                 : 	YES
    Product Id                    : 	0B3A
    Camera Locked                 : 	YES
    Usb Type Descriptor           : 	3.2
    Product Line                  : 	D400
    Asic Serial Number            : 	119623060612
    Firmware Update Id            : 	119623060612

• Are you able to display the pointcloud in realsense-viewer?
  -> I can toggle the Stereo Module and RGB Camera and can see the colored output of them.

• Which Jetson are you using? I have a Jetson Xavier AGX that I can try to replicate this on.
  -> I am using Jetson Xavier NX

In summary, the RGB image does not appears compared to the depth image.

[2b-t]

I tried this on my Nvidia Jetson Xavier AGX and ran into similar issues. The images would publish just fine while once turning on the pointcloud also the images would not be published any more. This seems to be a common issues when combining a Realsense with an Nvidia Jetson: There are several issues on the official repository discussing this problem with ROS as well as ROS 2 (such as IntelRealSense/realsense-ros#1967 or IntelRealSense/realsense-ros#2886). Some people seem to have had success with specific versions of the Realsense ROS driver/librealsense/firmware. I would recommend anyone running into the same issue having a look at the existing issues there and in doubt open a new one on their repository.

I will close this as from my understanding this is an issue with the Realsense driver itself and not the Docker in this repository. If you find out a combination that works, let me know and I can add it to the documentation here.

[2b-t]

Hi @kenloumixx,
A colleague of mine recently ran into the same issue on an Nvidia AGX Orin. Turns out there are extra instructions for running the Realsense on a Jetson that can be found here. Rather than patching the kernel the solution that he chose was using this Dockerfile which compiles the drivers from source and allowed him to run the Realsense including point clouds at 25 Hz.