This demo presents an appication of AMD Kria K260 to computations of stereo-vision based depth. It also showcases a Hardware-in-the-loop (HIL) simulation with Open 3D Engine (O3DE) and the use of Robot Operating System (ROS) interfaces.
The key computation is offloaded to FPGA (Field-programmable gate array) for efficiency and determinism.
In this simple project number of technologies are utilized:
- AMD Kria 260, a board which is well-suited for robotics.
- AMD Vitis ™️, a platform to develop solutions for FPGA.
- Kria Robotics stack, a ROS 2 set of tools, nodes, and libraries to deploy hardware-accelerated solutions to Kria SOMs.
- Host machine (X86) with Ubuntu 22.04 and plenty of free space (~300 Gb), primarily for Vitis. This machine will also run the HIL simulation.
- KR260 board with Ubuntu 22.04, which you set up following this guide.
- ROS 2 Humble installed on KR260 and x86.
Download Vivado 2022.1, preferably Xilinx_Unified_2022.1. The file is quite large, so do not forget to check MD5 checksum before installation.
Install Vitis at default location /tools/Xilinx
.
If you experience freezing, follow the support recommendation and install these:
sudo apt-get install libtinfo5 libncurses5
Setup Locale:
sudo apt update && sudo apt install locales
sudo locale-gen en_US en_US.UTF-8
sudo update-locale LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8
export LANG=en_US.UTF-8
Add Sources & Repo Key:
sudo apt update && sudo apt install software-properties-common
sudo add-apt-repository universe
sudo apt update && sudo apt install curl
curl -sSL https://raw.githubusercontent.com/ros/rosdistro/master/ros.key | sudo apt-key add -
Add ROS 2 Repository:
sudo sh -c 'echo "deb [arch=$(dpkg --print-architecture)] http://packages.ros.org/ros2/ubuntu $(lsb_release -cs) main" > /etc/apt/sources.list.d/ros2.list'
Install ROS 2 Humble (full-desktop, 1.7 GB download):**
sudo apt update
sudo apt install ros-humble-desktop
Install additional tools and rmw-cyclonedds-cpp
for ROS:
sudo apt install python3-colcon-common-extensions python3-argcomplete ros-humble-rmw-cyclonedds-cpp
sudo apt-get -y install curl build-essential libssl-dev git wget \
ocl-icd-* opencl-headers python3-vcstool \
python3-colcon-common-extensions python3-colcon-mixin \
kpartx u-boot-tools pv
###################################################
# 2. create a new ROS 2 workspace with examples and
# firmware for KR260
###################################################
mkdir -p ~/krs_ws/src; cd ~/krs_ws
###################################################
# 3. Create file with KRS 1.0 additional repos
###################################################
cat << 'EOF' > krs_humble.repos
repositories:
perception/image_pipeline:
type: git
url: https://github.com/ros-acceleration/image_pipeline
version: ros2
tracing/tracetools_acceleration:
type: git
url: https://github.com/ros-acceleration/tracetools_acceleration
version: humble
firmware/acceleration_firmware_kr260:
type: zip
url: https://github.com/ros-acceleration/acceleration_firmware_kr260/releases/download/v1.1.1/acceleration_firmware_kr260.zip
acceleration/adaptive_component:
type: git
url: https://github.com/ros-acceleration/adaptive_component
version: humble
acceleration/ament_acceleration:
type: git
url: https://github.com/ros-acceleration/ament_acceleration
version: humble
acceleration/ament_vitis:
type: git
url: https://github.com/ros-acceleration/ament_vitis
version: humble
acceleration/colcon-hardware-acceleration:
type: git
url: https://github.com/colcon/colcon-hardware-acceleration
version: main
acceleration/ros2_kria:
type: git
url: https://github.com/ros-acceleration/ros2_kria
version: main
acceleration/ros2acceleration:
type: git
url: https://github.com/ros-acceleration/ros2acceleration
version: humble
acceleration/vitis_common:
type: git
url: https://github.com/ros-acceleration/vitis_common
version: humble
acceleration/acceleration_examples:
type: git
url: https://github.com/ros-acceleration/acceleration_examples
version: main
EOF
###################################################
# 4. import repos of KRS 1.0 release
###################################################
vcs import src --recursive < krs_humble.repos
Build workspace
unset RMW_IMPLEMENTATION
source /opt/ros/humble/setup.bash # Sources system ROS 2 installation.
export PATH="/usr/bin":$PATH # FIXME: adjust path for CMake 3.5+
sudo ls -l # Hack to give sudo access to shell, else build may hang.
colcon build --merge-install --packages-select acceleration_firmware_kr260 vitis_common colcon-hardware-acceleration ros2acceleration ros2_kria colcon-hardware-acceleration ament_vitis ament_acceleration
Add to bash.rc
source /opt/ros/humble/setup.bash
source /home/mpelka/krs_ws/install/setup.bash
Enter the 'krs_ws' workspace. Setup environment variable by adding the following line to your ~.bashrc
:
export KRS_WS=/home/$USER/kria_depth_demo/krs_ws/
Install prerequisites (according to Install KRS)
sudo apt-get install ros-humble-rmw-cyclonedds-cpp ros-humble-cyclonedds*
sudo apt-get -y install curl build-essential libssl-dev git wget \
ocl-icd-* opencl-headers python3-vcstool \
python3-colcon-common-extensions python3-colcon-mixin \
kpartx u-boot-tools pv gcc-multilib
sudo apt-get -y install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu
sudo apt-get install qemu-user-static
sudo apt-get install ros-humble-gazebo-ros ros-humble-gazebo-plugins ros-humble-gazebo-msgs
Install repositories that are part of Kria Robotic Stack.
Note! Before proceed with cross-compilation, make sure that RMW_IMPLEMENTATION is not set #KRS/84:
unset RMW_IMPLEMENTATION
cd $KRS_WS
vcs import src --recursive < krs_humble.repos
Source Vitis and ROS 2 and build tools for x86. Note that the building needs superuser access.
cd $KRS_WS
unset RMW_IMPLEMENTATION
source /tools/Xilinx/Vitis/2022.1/settings64.sh # source Xilinx tools
source /opt/ros/humble/setup.bash # Sources system ROS 2 installation.
export PATH="/usr/bin":$PATH # FIXME: adjust path for CMake 3.5+
sudo ls -l # Hack to give sudo access to shell, else build may hang.
colcon build --merge-install --packages-select acceleration_firmware_kr260 vitis_common colcon-hardware-acceleration ros2acceleration ros2_kria colcon-hardware-acceleration ament_vitis ament_acceleration
sudo ln -s $KRS_WS/install/../acceleration/firmware/kr260/sysroots/aarch64-xilinx-linux/usr/lib/aarch64-linux-gnu/libpython3.10.so.1.0 /usr/lib/aarch64-linux-gnu/libpython3.10.so -f
With colcon
select the build target (kr260):
cd $KRS_WS
unset RMW_IMPLEMENTATION
source /tools/Xilinx/Vitis/2022.1/settings64.sh # source Xilinx tools
source /opt/ros/humble/setup.bash # Sources system ROS 2 installation.
source ./install/setup.bash # Source KRS
export PATH="/usr/bin":$PATH
colcon acceleration select kr260
Build only the depth node (takes about half an hour on Ryzen 7 9700x):
cd $KRS_WS
unset RMW_IMPLEMENTATION
source /tools/Xilinx/Vitis/2022.1/settings64.sh # source Xilinx tools
source /opt/ros/humble/setup.bash # Sources system ROS 2 installation.
source ./install/setup.bash # Source KRS
export PATH="/usr/bin":$PATH
rm -r build-kr260-ubuntu/stereolbm_accel/
colcon build --executor sequential --event-handlers console_direct+ --build-base=build-kr260-ubuntu --install-base=install-kr260-ubuntu --merge-install --mixin kr260 --cmake-args -DNOKERNELS=false --packages-select stereolbm_accel
Next, you need to copy to the board (assuming that Kria is configured as host in your .ssh/config
, and the user name is 'ubuntu'):
cd $KRS_WS
scp -r install-kr260-ubuntu/lib/stereolbm_accel ubuntu@kria:/home/ubuntu/
Next on the board (after ssh kria
):
sudo cp -r /home/$USER/stereolbm_accel /usr/lib/firmware/xilinx
sudo xmutil listapps #Queries on target FW resource manager daemon of pre-built app bitstreams available on the platform and provides summary to CLI.
sudo xmutil unloadapp #Removes application bitstream. (Takes slot number, default 0)
sudo xmutil loadapp stereolbm_accel #Loads requested application configuration bitstream to programmable logic if the device is available.
Next, source KRS and run accelerated node:
source /home/$USER/krs_ws/install/setup.bash
cd /home/$USER/stereolbm_accel
./stereolbm_accel_tb
We have a peer-to-peer connection with cyclone-dds
.
Please create a file with the DDS configuration on the x86 and Kria board. This configuration assumes that peer addresses are:
- 192.168.99.1
- 192.168.99.2 Do not forget to adjust the interface name, substitute 'enp46s0' for your network interface.
<?xml version="1.0" encoding="UTF-8" ?>
<CycloneDDS xmlns="https://cdds.io/config" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="https://cdds.io/config
https://raw.githubusercontent.com/eclipse-cyclonedds/cyclonedds/master/etc/cyclonedds.xsd">
<Domain id="any">
<General>
<Interfaces>
<!--- Interface name below --->
<NetworkInterface name="enp46s0"/>
</Interfaces>
<AllowMulticast>default</AllowMulticast>
<MaxMessageSize>65500B</MaxMessageSize>
</General>
<Internal>
<SocketReceiveBufferSize min="10MB" max="default" />
<Watermarks>
<WhcHigh>500kB</WhcHigh>
</Watermarks>
</Internal>
<Discovery>
<Peers>
<Peer address="192.168.99.1"/>
<Peer address="192.168.99.2"/>
</Peers>
<ParticipantIndex>auto</ParticipantIndex>
<MaxAutoParticipantIndex>1000</MaxAutoParticipantIndex>
</Discovery>
</Domain>
</CycloneDDS>
Now, add reference to created file (e.g. ~/kria_cyclone.xml
) in ~/.bashrc
:
export CYCLONEDDS_URI=file:///home/$USER/kria_cyclone.xml
export RMW_IMPLEMENTATION=rmw_cyclonedds_cpp
#export RMW_IMPLEMENTATION=rmw_fastrtps_cpp
export ROS_DOMAIN_ID=34
Finally, set the network stack on x86 and Kria:
sudo vim /etc/sysctl.d/10-cyclone-max.conf
and insert:
net.ipv4.ipfrag_time=3
net.ipv4.ipfrag_high_thresh=134217728
net.core.rmem_max=2147483647
Reboot both x86 and Kria.
The simulator is available under this repository ROSCon2023Demo.
Download prebuilt package for ROS 2 humble
Install prerequisites:
sudo apt install ros-${ROS_DISTRO}-ackermann-msgs ros-${ROS_DISTRO}-control-toolbox ros-${ROS_DISTRO}-nav-msgs ros-${ROS_DISTRO}-gazebo-msgs ros-${ROS_DISTRO}-vision-msgs ros-${ROS_DISTRO}-ur-msgs ros-${ROS_DISTRO}-moveit-servo ros-${ROS_DISTRO}-moveit-visual-tools ros-${ROS_DISTRO}-moveit ros-${ROS_DISTRO}-pilz-industrial-motion-planner ros-${ROS_DISTRO}-controller-manager ros-${ROS_DISTRO}-ur-client-library ros-${ROS_DISTRO}-nav2-common ros-${ROS_DISTRO}-navigation2 python3-rosdep2 python3-colcon-common-extensions
Initialize rosdep
rosdep update
Create SIM_PACKAGE environment variable (adjust accordingly to the chosen directory)
export SIM_PACKAGE=~/rosconpkg
After downloading the package and unzipping it under ${SIM_PACKAGE} on X86 you need to build Simulator's ROS 2 workspace:
cd ${SIM_PACKAGE}/ros2_ws
rosdep install --ignore-src --from-paths src/Universal_Robots_ROS2_Driver -y
colcon build --symlink-install
On Kria:
source /home/${USER}/krs_ws/install/setup.bash
cd /home/${USER}/stereolbm_accel
./stereolbm_accel_tb
Note: After each power cycle of the KRIA make sure to load the FPGA Bitstream before running the above by:
sudo xmutil unloadapp
sudo xmutil loadapp stereolbm_accel
On x86:
source ${SIM_PACKAGE}/ros2_ws/install/setup.bash
${SIM_PACKAGE}/ROSCon2023DemoGamePackage/ROSCon2023Demo.GameLauncher -r_fullscreen=false -bg_ConnectToAssetProcessor=0 -r_width=960 -r_height=960 +LoadLevel levels/demostereo/demostereo.spawnable +r_displayInfo 0