sd_image_raspberrypi4: Remove, as planned initially

[samueldr]

The replacement is the generic AArch64 image.

From there, you can customize an image that works better for your
needs, if need be.

Motivation for this change

• Various: Add support for raspberry pi 4. #68265 (comment)

Things done

• Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Ensured that relevant documentation is up to date
• Fits CONTRIBUTING.md.

I tested this on top of [master, #121450, #121720], but really only #121720 brings any changes to the SD image, and the only change relevant here is USB support in stage-1. The vc4 driver for early modesetting was already present in the sd image build.

cc @domenkozar (as it might affect your upcoming tutorial somewhat)

This allows us to close #111683 since the vendor-specific image is finally removed. (Though the actual underlying issue still exists, but is not an issue for raspberry pi end-users, or our other SD image builder)

[drewrisinger]

• Diff LGTM
• Commits LGTM
• Build: trivially tested eval on x86-64-linux system, but didn't try to build anything. Shouldn't need to do anything b/c this PR just removes.

[domenkozar]

Thanks for the ping! Was USB boot support the only reason to use RPI specific kernel? I'm a bit surprised that won't be needed anymore.

If I understand your description correctly, once #121720 is merged, this can be merged as well and generic aarch64 image will work fine on rpi4.

[samueldr]

The Raspberry Pi foundation kernel is required for interacting with "the hardware ecosystem", e.g. hats, gpio things and similar. This is because they're following their own conventions, which are not the same for mainline. There may be other specific use cases that requires the vendor kernel. In these cases, the user is free to switch to it in their configuration.

The nice thing is that we don't need to ship a board-specific image anymore. The generic image is enough to allow end-users to get a foothold on their Raspberry Pi 4.

#121720 is only needed for USB in stage-1, but at neither depends on the other. Merging this before the other just means nothing changes about that.

[mohe2015]

@samueldr Did you also test the 8 GiB version as I remember to have issues with it in the (recent) past with mainline?

[samueldr]

I don't have an 8GB variant to test with. Good question though.

[tfc]

The nice thing is that we don't need to ship a board-specific image anymore. The generic image is enough to allow end-users to get a foothold on their Raspberry Pi 4.

Does that mean that in the future i would download this image, iron it over a usb stick and boot the pi from it, then configure it with the rpi4 kernel and that's it (or alternatively build my own image which is the default aarch64 image but with rpi4 kernel)?

[domenkozar]

The nice thing is that we don't need to ship a board-specific image anymore. The generic image is enough to allow end-users to get a foothold on their Raspberry Pi 4.

Does that mean that in the future i would download this image, iron it over a usb stick and boot the pi from it, then configure it with the rpi4 kernel and that's it (or alternatively build my own image which is the default aarch64 image but with rpi4 kernel)?

My understanding is that you'd need to include a few profiles. I'll document that as I write a tutorial for https://nix.dev

[bbqsrc]

I have just booted nixos-sd-image-21.05pre287596.ae1c8ede09b-aarch64-linux on an 8GB rpi4, and it seems USB does indeed not work, with either the new kernel or ordinary variants. What is presently the expected behaviour there?

[samueldr]

Does that mean that in the future i would download this image, iron it over a usb stick and boot the pi from it, then configure it with the rpi4 kernel and that's it (or alternatively build my own image which is the default aarch64 image but with rpi4 kernel)?

My understanding is that you'd need to include a few profiles. I'll document that as I write a tutorial for https://nix.dev

This is becoming a bit off-topic for this thread, but here goes, strap yourself in, we're doing a quick fly by.

It all depends on the exact boot configuration you intend to use. The Raspberry Pi family of hardware is the more convoluted in the "supported" set of hardware.

Three main ways to boot:

• Raspberry Pi specific boot chain ("config.txt" direct kernel boot)
• U-Boot with distroboot (default)
• Tianocore UEFI

For U-Boot, with NixOS, there are two main ways:

• Unmanaged "opaque firmware" (default)
• Managed through configuration

Using the generic image (and the current Raspberry Pi 4 image configuration) assumes you're using an unmanaged U-Boot. This is important because it is the most equivalent boot chain, comparing to other ARM targets supported.

Now this is only considering the initial boot firmware and boot scheme.

The Raspberry Pi ecosystem is vibrant and full of "exciting" things. Mind the scare quotes. This is because the "exciting" hardware generally assumes you're using the vendor kernel, the kernel form from the Raspberry Pi foundation. Generally, "ecosystem things" won't work with the mainline kernel because of differences in the hardware support and device tree (a bit wrongly worded, but close enough).

So if you intend to use "ecosystem things", you have to use the Raspberry Pi foundation's kernel.

I will not be describing the software only differences, but hit up @cleverca22 on IRC for a deep dive of the different rendering stacks and how they stack up (heh) against each others. Because yes, the vendor kernel will differ here, compared to the mainline kernel.

If using "ecosystem things", using U-Boot may not work exactly as expected. That is because U-Boot in the "distro boot" boot scheme will load a fresh device tree binary file (.dtb), undoing any dtoverlay configuration you might have dont in config.txt. In theory there is a NixOS option that should be handling that, but I've not yet dealt with any of this, as I have been solely concerned about the generic mainline support, which is the only one we should strive to achieve.

What was that that about managed vs. unmanaged U-Boot? Oh, good thing you remembered to ask!

Managed U-Boot is the scheme that is usable only with the Raspberry Pi family of hardware, where the NixOS system build will replace the initial boot firmware in use by the board on system rebuilds. On a gut feeling this doesn't sound like it's an issue. After all, GRUB is managed, systemd-boot is too. Why would U-Boot differ? U-Boot differs here because U-Boot is both a firmware and a bootloader. It acts as a firmware because it serves to initialize some of the hardware and generally hands off to either a bootloader or an operating system, like a BIOS or UEFI implementation. It also acts as a bootloader, since it has intrinsic knowledge about booting a Linux system.

My opinion is that you should ask yourself "should the declarative generations-based operating system manage the BIOS of my computer?" My opinion is no. This is because you cannot easily go back to the previous working BIOS if it fails to update it. Just like on basically any other computer, let an out of band system manage it. In an ideal scenario, U-Boot would be managed using something like fwupd or any other standard "firmware update process".

I hope this answers some questions. I hope I didn't overwhelm you all too much!

Anyway, thank you for coming to my ted talk.

[samueldr]

I have just booted nixos-sd-image-21.05pre287596.ae1c8ede09b-aarch64-linux on an 8GB rpi4, and it seems USB does indeed not work, with either the new kernel or ordinary variants. What is presently the expected behaviour there?

What does "USB" and "not work" mean here? Does it mean "booting the image from a USB drive" or does it mean "booting from an SD card and using a USB keyboard on the running system"?

"Booting from USB" is unknown to me. There are too many variables to account for since the EEPROM firmware (yes, another firmware layer) will change the behaviour at boot for USB.

"USB keyboard on an otherwise running system" I expect to work. I burned an image using an instersection of [this PR, master, #121450, #121720] to an SD card, booted it to the shell, and was able to use a USB keyboard. Though for this basic support, I don't think any of the previously mentioned PRs are required, but I have not tested it either.

[bbqsrc]

Booting from an SD card and using a USB keyboard on the running system. On the last successfully built raspberrypi4 image, the keyboard does indeed work on that device.

[samueldr]

Interesting

The new 8GB RPi4 contains this HW design change and USB will not work without it.

• https://lists.denx.de/pipermail/u-boot/2020-June/414411.html

From almost a year ago. On U-Boot's side everything seems to be in working order. I mean, everything is merged. No work still to be done.

As for the kernel, this sounds like it should be working, and should be working since 5.8.

Our "non new kernel" build should be LTS 5.10. It's 5.10.32 on the specific image you've used. Maybe try a new_kernel variant?

[bbqsrc]

Yeah, I did try the new_kernel variant as well (which has 5.11 kernel) and got the same behaviour, unfortunately.

[samueldr]

I do not know what is the expectation here. I know there is a firmware that needs to be present for USB to work. I don't know how that differs on the 8GB variant. And I don't know how you should have it updated.

@bbqsrc can you try doing a dirty thing, and install an up-to-date Raspberry Pi OS and update your EEPROM?

https://www.raspberrypi.org/documentation/hardware/raspberrypi/booteeprom.md

But first, important figure out your current EEPROM version and take note of it!

If it fixes it, that shows that something has to be done, and kind of further supports my point about firmware updates that should be handled out of band... I'm not sure what a good robust solution for that would be.

[samueldr]

(Since edits are not obvious)

NO, you don't need to do any of that:

The VLI805 USB firmware EEPROM image - ignored on 1.4 board revision which does not have a dedicated VLI EEPROM

This explains it. And... sad trombone noises now it's transformed into a firmware file chase!

NOTE: This is a mostly independent issue from booting from USB. This is USB issues after the system is already booted.

[domenkozar]

I'm getting https://www.lenovo.com/us/en/accessories-and-monitors/keyboards-and-mice/keyboards/KBD-BO-TrackPoint-KBD-US-English/p/4Y40X49493 on Monday and will report back.

[samueldr]

@domenkozar I hear from that that you don't have an/another USB keyboard, and wanted to test USB?

Anyway, this will matter according to the board revision in use. Sadly I bought early, so I have the earliest board revisions. I don't know if the rev 1.4 is used on anything else than the 8GB variants.

I also might have been too hasty in dismissing the eeprom upgrade. I still haven't found conclusive description of where the VLSI firmware lives. U-Boot's implementation seems to imply that the videocore firmware handles passing along the firmware. It seems like U-Boot does not handle the firmware file at any point in time.

This could mean that either the main EEPROM, or other firmware files on the firmware partition, are used for this purpose.

[samueldr]

While not related to USB boot, it looks like on rev 1.4 (e.g. the 8GB variant) the EEPROM may need to be updated fully for USB to work on the booted (from SD) system.

@bbqsrc updating the EEPROM firmware I assume should fix the issue when booting from SD and wanting to use a USB keyboard. Though maybe it also requires a fresh enough image with the modules PR merged.

[bbqsrc]

@samueldr more surprisingly, no! USB keyboards simply work with image nixos-sd-image-21.05pre287941.54fa457b258-aarch64-linux. Tried with the default and latest eeproms, no difference in behaviour. 😄

[samueldr]

Well... I'll stay confused forever as to why. Good to have confirmation that USB works on the newer revisions too now.

This was the last issue to be solved, really.

[domenkozar]

I can boot the generic aarch64 image on RPI4 4GB model and attach a keyboard.

@samueldr what's blocking this PR?

[samueldr]

Nothing really was. I was waiting on your report just in case.

Even the 8GB variant issues really didn't block this issue.

I don't mind a PR being open a few days more, like it has.