NVIDIA: Add support for Jetpack 6.0 and split NVIDIA into two new platforms #4532

rene · 2025-01-21T17:45:52Z

Overview

NVIDIA Jetpack is the NVIDIA's software stack for Jetson modules and developer kits. Currently EVE supports the Jetpack 5.1.3 through the nvidia platform variant. However, newer AI applications and devices are based in the Jetpack 6.x version, which doesn't support the Jetson Xavier NX devices. This PR introduces the support to new Jetson Orin devices along with Jetpack 6.0 version. In order to keep the support for the Jetson Xavier devices, both Jetpacks are now supported:

Jetpack 5.1.3: For Jetson Xavier NX devices
Jetpack 6.0: For Jetson Orin based devices

These versions covers different devices from different Jetson platforms, as shown by the Table 1.

	Jetson AGX Orin	Jetson AGX Orin Industrial	Jetson Orin NX / Orin Nano	Jetson AGX Xavier	Jetson AGX Xavier Industrial	Jetson Xavier NX
Jetpack 6.0	✔️	✔️	✔️
Jetpack 5.1.3	✔️	✔️	✔️	✔️	✔️	✔️

Table 1: Jetpack support. Source: Jetson Linux Archive.

For each Jetpack version supported, EVE uses its own kernel derived from the original kernel from Jetpack Linux distro. All kernel versions used by EVE are available at the eve-kernel project.

Changes

The major changes provided by this PR relies on the removal of nvidia platform and the introduction of two new platforms:

nvidia-jp5: It installs and supports Jetpack 5.1.3
nvidia-jp6: It installs and supports Jetpack 6.0

The EVE image (install, live, etc) should be generated according the target device. For instance, for Jetson Xavier based devices the images should be generated using make ZARCH=arm64 PLATFORM=nvidia-jp5 <image_type>. For any Jetson Orin based device, image should be generated using make ZARCH=arm64 PLATFORM=nvidia-jp6 <image_type>.

Packages affected

So far only the following packages are affected with different builds according the NVIDIA variant platform:

pkg/fw: Firmware package
pkg/nvidia: NVIDIA package

The TAG of these packages will contain the platform the package was built for, so it can be easily differentiated and identified. For instance:

lfedge/eve-fw:15f4439110fb41b3af58a8d439878c632cc10e58-nvidia-jp5
lfedge/eve-fw:15f4439110fb41b3af58a8d439878c632cc10e58-nvidia-jp6
lfedge/eve-fw:15f4439110fb41b3af58a8d439878c632cc10e58-generic

When applied (for instance, for pkg/nvidia), the nvidia-jp5 variant is used as default for the generic platform.

Supported devices

The following devices based on Jetson Orin will be supported through Jetpack 6.0:

Aetina AIE-PX22
Aetina AIE-KN32
Jetson Orin Nano developer kit

Tests

Tests were performed on the mentioned devices by deploying AI apps and other workloads.

Known issues

LEDs of the 10G Ethernet port are not working on the Aetina AIE-PX22 (only the LEDs are not working, the ethernet connection works flawlessly).

GitHub workflows

GitHub build workflows are adjusted with the new variants in the following PR:

github/workflows: Add new NVIDIA platform variants #4534

rene · 2025-01-21T17:53:56Z

Note for Reviewers

GH Actions

~~Build for platform nvidia won't work because the PR is removing this platform. The adjustments for GH Actions are provided by PR #4534~~

arm64 builds

~~They won't work until FSF cgit servers come back online (they are down due to DDoS attacks).~~

OhmSpectator

You could add the necessary files into .spdxignore =)

rene · 2025-01-21T17:58:33Z

You could add the necessary files into .spdxignore =)

Very well observed @OhmSpectator , totally forgot about it. It's done. Thanks!

NVIDIA Jetpack 6.0 doesn't support Xavier devices, so EVE's nvidia platform must be split into two new variants in order to support both Jetpack versions: nvidia-jp5, for Xavier based devices (Jetpack 5); and nvidia-jp6, for Orin based devices (Jetpack 6). This commit introduces the two new platforms (nvidia-jp5 and nvidia-jp6) and removes the former nvidia platform without changing any of the existing packages, they will be adjusted later. Signed-off-by: Renê de Souza Pinto <rene@renesp.com.br>

This commit introduces the support in the firmware package for both nvidia platform variants: nvidia-jp5 and nvidia-jp6. Signed-off-by: Renê de Souza Pinto <rene@renesp.com.br>

rene · 2025-01-22T12:11:54Z

As suggested by @yash-zededa I've moved the changes to GH actions to a dedicated PR: #4534

deitch · 2025-01-22T14:13:50Z

LGTM.

I do have a question, though. Someone reading this might be confused. Aren't JP5 and JP6 Nvidia's packaged-up Ubuntu-based operating systems? What does it mean to have "jp6 EVE build"? Are we actually running JP6 and an EVE agent, or are we installing some parts of JP6 on EVE, and if so, is it base operating system or in a container?

I don't necessarily mean to answer it here, as much as if it is clear to someone coming to it with minimal EVE knowledge other than, "it is a special-purpose edge OS" and "I know what JP6 is".

OhmSpectator

In general, it looks good. I dropped several minor comments, though.

kernel-commits.mk

pkg/nvidia/Dockerfile

pkg/nvidia/scripts/jp6/nv-init.sh

pkg/nvidia/udev/jp6/rules.d/90-eve-nvidia.rules

OhmSpectator · 2025-01-22T14:05:58Z

pkg/grub/rootfs.cfg

   if [ "$smb_product" = "NVIDIA Orin Nano Developer Kit" ]; then
      set_to_existing_file devicetree /boot/dtb/nvidia/tegra234-p3767-0003-p3768-0000-a0.dtb
-      set_global dom0_console "console=ttyAMA0,115200 console=tty0 console=ttyTCU0,115200 earlycon=ttyTCU0,115200"
+      set_global dom0_console "console=ttyAMA0,115200 console=tty1 console=ttyTCU0,115200 earlycon=ttyTCU0,115200"


This commit adds the corresponding entry on GRUB config

and explicitly switches the console option for the jp5 variant to tty1. I kind of remember why it's necessary, but I'm a little confused that we still use tty0 and tty1 in different variants...

it's necessary because of the TUI monitor... actually we need to update all references for tty0 for any other platform, but it's outside the scope of this PR... I can open a new one to take care of other platforms....

Or we at least should have a ticket for that, not to forget)

pkg/grub/rootfs.cfg

OhmSpectator · 2025-01-22T14:21:10Z

"WARNING: We are assembling an arm64 image on amd64. Things may break."

deitch · 2025-01-22T14:22:26Z

WARNING: We are assembling an arm64 image on amd64. Things may break."

That is such an old message. Maybe we should get rid of it? Are we really still worried about cross-building?

OhmSpectator · 2025-01-22T14:23:55Z

WARNING: We are assembling an arm64 image on amd64. Things may break."

That is such an old message. Maybe we should get rid of it? Are we really still worried about cross-building?

Ah, sorry, the real error was:

/home/ubuntu/actions-runner/_work/eve/eve/kernel-version.mk:66: *** "KERNEL_COMMIT_nvidia is not defined. did you introduce new platform or ARCH?". Stop.

rene · 2025-01-22T17:01:33Z

WARNING: We are assembling an arm64 image on amd64. Things may break."

That is such an old message. Maybe we should get rid of it? Are we really still worried about cross-building?

I agree! Let's get rid of it.....

rene · 2025-01-22T17:03:04Z

WARNING: We are assembling an arm64 image on amd64. Things may break."

That is such an old message. Maybe we should get rid of it? Are we really still worried about cross-building?

Ah, sorry, the real error was:

/home/ubuntu/actions-runner/_work/eve/eve/kernel-version.mk:66: *** "KERNEL_COMMIT_nvidia is not defined. did you introduce new platform or ARCH?". Stop.

Yeah, I have added a new commit to this PR which keeps the nvidia platform, so all builds should pass. Once this PR and the PR that updates the GH actions get merged, I will revert the commit and complete the removal of the nvidia platform. In this way there will be no breaks or build failures.

This commit adds the support to Jetpack 6.0 by using the two new nvidia platform variants: nvidia-jp5: Supports Jetson Xavier NX devices, Jetpack 5.1.3 nvidia-jp6: Supports Jetson Orin Nano/AGX devices, Jetpack 6.0 nvidia-jp5 is also used for generic platforms. Signed-off-by: Renê de Souza Pinto <rene@renesp.com.br>

This commit adds the corresponding entry on GRUB config. for Jetson Orin Developer Kit for Jetpack 6. Signed-off-by: Renê de Souza Pinto <rene@renesp.com.br>

Add boot options for the Aetina AIX-PX11 device, which is based on the Jetson Orin AGX platform. Signed-off-by: Renê de Souza Pinto <rene@renesp.com.br>

This commit adds the corresponding entry on GRUB config. for Jetson Orin NX devices (Jetpack 6). Signed-off-by: Renê de Souza Pinto <rene@renesp.com.br>

rene · 2025-01-22T17:18:37Z

LGTM.

I do have a question, though. Someone reading this might be confused. Aren't JP5 and JP6 Nvidia's packaged-up Ubuntu-based operating systems? What does it mean to have "jp6 EVE build"? Are we actually running JP6 and an EVE agent, or are we installing some parts of JP6 on EVE, and if so, is it base operating system or in a container?

I don't necessarily mean to answer it here, as much as if it is clear to someone coming to it with minimal EVE knowledge other than, "it is a special-purpose edge OS" and "I know what JP6 is".

This is a very good point @deitch , I've updated the documentation to try to clarify that "support a Jetpack version" means provide all default libraries of the Jetpack Jetson Linux distro + run EVE on a custom kernel derived from the original kernel....

rene · 2025-01-22T17:25:07Z

WARNING: We are assembling an arm64 image on amd64. Things may break."

That is such an old message. Maybe we should get rid of it? Are we really still worried about cross-building?

I agree! Let's get rid of it.....

#4537
☝️ @deitch @OhmSpectator

deitch

Thanks for updating the docs.

docs/NVIDIA.md

deitch · 2025-01-22T17:32:29Z

This is a very good point @deitch

I am really good at being ignorant and asking questions like that 😆

Update documentation related to NVIDIA devices: - Create a general page for NVIDIA Jetson platforms - Update docs for Jetson Xavier NX devices - Add docs for Jetson Orin devices Signed-off-by: Renê de Souza Pinto <rene@renesp.com.br>

This commit brings back the nvidia platform so it won't break any GitHub build action. It should be reverted after the update of the workflows to support the new platforms: nvidia-jp5 and nvidia-jp6. Signed-off-by: Renê de Souza Pinto <rene@renesp.com.br>

eriknordmark

Run tests

rene requested review from deitch and OhmSpectator January 21, 2025 17:45

rene requested a review from eriknordmark as a code owner January 21, 2025 17:45

github-actions bot requested a review from rucoder January 21, 2025 17:46

OhmSpectator reviewed Jan 21, 2025

View reviewed changes

rene force-pushed the nvidia-platform-split branch 2 times, most recently from 8d51d64 to 08fb342 Compare January 21, 2025 18:05

rene requested a review from yash-zededa as a code owner January 21, 2025 18:10

github-actions bot requested a review from uncleDecart January 21, 2025 18:10

rene force-pushed the nvidia-platform-split branch 2 times, most recently from ab1c23c to 100e1dd Compare January 21, 2025 18:36

rene added 2 commits January 22, 2025 11:17

fw: Add support for both nvidia variants

47b99bd

This commit introduces the support in the firmware package for both nvidia platform variants: nvidia-jp5 and nvidia-jp6. Signed-off-by: Renê de Souza Pinto <rene@renesp.com.br>

rene mentioned this pull request Jan 22, 2025

github/workflows: Add new NVIDIA platform variants #4534

Merged

1 task

rene force-pushed the nvidia-platform-split branch from 100e1dd to 0fb5ce8 Compare January 22, 2025 12:09

deitch approved these changes Jan 22, 2025

View reviewed changes

OhmSpectator approved these changes Jan 22, 2025

View reviewed changes

rene added 4 commits January 22, 2025 18:09

grub: Setup Jetson Orin Developer Kit for Jetpack 6

ea87345

This commit adds the corresponding entry on GRUB config. for Jetson Orin Developer Kit for Jetpack 6. Signed-off-by: Renê de Souza Pinto <rene@renesp.com.br>

grub: Update configuration for Aetina AIX-PX11

ff954dd

Add boot options for the Aetina AIX-PX11 device, which is based on the Jetson Orin AGX platform. Signed-off-by: Renê de Souza Pinto <rene@renesp.com.br>

grub: Setup Jetson Orin NX for Jetpack 6

17d7895

This commit adds the corresponding entry on GRUB config. for Jetson Orin NX devices (Jetpack 6). Signed-off-by: Renê de Souza Pinto <rene@renesp.com.br>

rene force-pushed the nvidia-platform-split branch 2 times, most recently from b130eea to de9a70b Compare January 22, 2025 17:16

deitch approved these changes Jan 22, 2025

View reviewed changes

deitch reviewed Jan 22, 2025

View reviewed changes

docs/NVIDIA.md Outdated Show resolved Hide resolved

rene added 2 commits January 22, 2025 18:42

docs: Update documentation about NVIDIA devices

ef1b8e9

Update documentation related to NVIDIA devices: - Create a general page for NVIDIA Jetson platforms - Update docs for Jetson Xavier NX devices - Add docs for Jetson Orin devices Signed-off-by: Renê de Souza Pinto <rene@renesp.com.br>

rene force-pushed the nvidia-platform-split branch from de9a70b to 87a9470 Compare January 22, 2025 17:42

eriknordmark approved these changes Jan 22, 2025

View reviewed changes

eriknordmark merged commit 3c12828 into lf-edge:master Jan 23, 2025
45 checks passed

rene deleted the nvidia-platform-split branch February 6, 2025 09:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

NVIDIA: Add support for Jetpack 6.0 and split NVIDIA into two new platforms #4532

NVIDIA: Add support for Jetpack 6.0 and split NVIDIA into two new platforms #4532

rene commented Jan 21, 2025 •

edited

Loading

rene commented Jan 21, 2025 •

edited

Loading

OhmSpectator left a comment

rene commented Jan 21, 2025 •

edited

Loading

rene commented Jan 22, 2025

deitch commented Jan 22, 2025

OhmSpectator left a comment

OhmSpectator Jan 22, 2025

rene Jan 22, 2025

OhmSpectator Jan 22, 2025

OhmSpectator commented Jan 22, 2025

deitch commented Jan 22, 2025

OhmSpectator commented Jan 22, 2025

rene commented Jan 22, 2025

rene commented Jan 22, 2025

rene commented Jan 22, 2025

rene commented Jan 22, 2025

deitch left a comment

deitch commented Jan 22, 2025

eriknordmark left a comment

NVIDIA: Add support for Jetpack 6.0 and split NVIDIA into two new platforms #4532

NVIDIA: Add support for Jetpack 6.0 and split NVIDIA into two new platforms #4532

Conversation

rene commented Jan 21, 2025 • edited Loading

Overview

Changes

Packages affected

Supported devices

Tests

Known issues

GitHub workflows

rene commented Jan 21, 2025 • edited Loading

Note for Reviewers

GH Actions

arm64 builds

OhmSpectator left a comment

Choose a reason for hiding this comment

rene commented Jan 21, 2025 • edited Loading

rene commented Jan 22, 2025

deitch commented Jan 22, 2025

OhmSpectator left a comment

Choose a reason for hiding this comment

OhmSpectator Jan 22, 2025

Choose a reason for hiding this comment

rene Jan 22, 2025

Choose a reason for hiding this comment

OhmSpectator Jan 22, 2025

Choose a reason for hiding this comment

OhmSpectator commented Jan 22, 2025

deitch commented Jan 22, 2025

OhmSpectator commented Jan 22, 2025

rene commented Jan 22, 2025

rene commented Jan 22, 2025

rene commented Jan 22, 2025

rene commented Jan 22, 2025

deitch left a comment

Choose a reason for hiding this comment

deitch commented Jan 22, 2025

eriknordmark left a comment

Choose a reason for hiding this comment

rene commented Jan 21, 2025 •

edited

Loading

rene commented Jan 21, 2025 •

edited

Loading

rene commented Jan 21, 2025 •

edited

Loading