[Cross Compile] Is it possible to build for jetson(aarch64) images on x86_64 machine?

[harishkumarb-tw]

I would like to build a docker container on my x86_64 pc for Jetson aarch64. I tried the following this https://medium.com/@Smartcow_ai/building-arm64-based-docker-containers-for-nvidia-jetson-devices-on-an-x86-based-host-d72cfa535786 and installed QEMU, binfmt_static and then registered using,
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes.
But when I ran run_dev.sh by changing the PLATFORM to aarch64, the aarc64 base image is getting pulled, but later for the next layers it throws an error,

[+] Building 2.1s (3/3) FINISHED                                                                                                  docker:default
 => [internal] load .dockerignore                                                                                                           0.0s
 => => transferring context: 2B                                                                                                             0.0s
 => [internal] load build definition from Dockerfile.exec_depends                                                                           0.0s
 => => transferring dockerfile: 1.49kB                                                                                                      0.0s
 => ERROR [internal] load metadata for docker.io/library/aarch64-ros2_humble-image:latest                                                   2.1s
------
 > [internal] load metadata for docker.io/library/aarch64-ros2_humble-image:latest:
------
Dockerfile.exec_depends:2
--------------------
   1 |     ARG BASE_IMAGE
   2 | >>> FROM ${BASE_IMAGE}
   3 |     
   4 |     RUN apt-get update && mkdir -p ${ROS_ROOT}/src && cd ${ROS_ROOT} \
--------------------
ERROR: failed to solve: aarch64-ros2_humble-image: pull access denied, the repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
Failed to build base image: isaac_ros_dev-aarch64, aborting.

But, I already have the aarch64-ros2_humble-image:latest in my local and I was able to run the aarch64-based container on my x86_64 pc. docker images | grep ros output,

isaac_ros_dev-x86_64                                                                  latest                           fc66a048f618   43 minutes ago   29.5GB
aarch64-ros2_humble-image                                                             latest                           7a882c8d683b   2 months ago     18.8GB
x86_64-ros2_humble-image                                                              latest                           3c59bab3ee0b   2 months ago     28GB

So, I am wondering if is it possible to cross-compile just by changing the PLATFORM variable in run_dev.sh or if am I missing something.

[hemalshahNV]

Unfortunately, run_dev.sh is not setup yet to manage cross-compiling images (really build_base_image.sh underneath as run_dev.sh also launches the image as a container which you wouldn't want to do). Changing the platform image key by itself won't work by itself. For a good setup on cross-compiling Isaac ROS release images, take a look at work here from @rbonghi: https://github.com/rbonghi/isaac_ros_base. We're working on incorporating much of Raffaello's work into our mainline run_dev.sh scripts.

[3473f]

Unfortunately, run_dev.sh is not setup yet to manage cross-compiling images (really build_base_image.sh underneath as run_dev.sh also launches the image as a container which you wouldn't want to do). Changing the platform image key by itself won't work by itself. For a good setup on cross-compiling Isaac ROS release images, take a look at work here from @rbonghi: https://github.com/rbonghi/isaac_ros_base. We're working on incorporating much of Raffaello's work into our mainline run_dev.sh scripts.

Thanks for the pointer! I was recently looking into native cross compilation for Isaac ROS for our CI/CD pipelines. This should be 8 - 10x faster than emulation according to this discussion and this .. are there any plans to release a cross-compilation container with a reference targetfs similar to this for JetPack?

Edit: the way I am doing it right now is using the above JetPack container to get the files which are mounted inside the container from the host here, then using buildx to run the aarch64 base image, extract the "mounted" files to it, install all the dependencies within the emulated container, then "native" cross-compile against the extracted root of that container