chore: bump libs & fix libcamera, fix upstream nixpkgs #23

adminy · 2024-06-18T23:04:24Z

things tested and working:

boots on pi 5 b
libcamera works
nixpkgs master works

things to try out:

~~[ ] uboot booting, see if its fixed.~~ Uboot still not working
- test sd card boot

pi 4 test.

This should fix #19
Untested but maybe also #16

Serial error

DRAM:  1016 MiB (effective 8 GiB)
RPI 5 Model B (0xd04170)
Core:  22 devices, 11 uclasses, devicetree: board
MMC:   mmc@fff000: 0, mmc@1100000: 1
Loading Environment from FAT... Unable to read "uboot.env" from mmc0:1... 
In:    serial,usbkbd
Out:   serial,vidconsole
Err:   serial,vidconsole
mbox: Header response code invalid
bcm2835: Could not query MAC address
Net:   No ethernet found.

starting USB...
No working controllers found
Hit any key to stop autoboot:  0 
No EFI system partition
No EFI system partition
Failed to persist EFI variables
No EFI system partition
Failed to persist EFI variables
No EFI system partition
Failed to persist EFI variables
** Booting bootflow '<NULL>' with efi_mgr
Loading Boot0000 'mmc 0' failed
EFI boot manager: Cannot load any image
Boot failed (err=-14)
** Booting bootflow 'mmc@fff000.bootdev.part_2' with extlinux
------------------------------------------------------------
1:      NixOS - Default
Enter choice: 1 
1:      NixOS - Default
Retrieving file: /boot/extlinux/../nixos/4ljfp9dpyfv3r0pw1b8nwwxysc2c59fx-linux-rpi-6.6.34-Image
Retrieving file: /boot/extlinux/../nixos/iphdbw2np9dsacx7i37xbazqkczjsiaz-initrd-linux-rpi-6.6.34-initrd
append: init=/nix/store/4cpd34134wcaxqy4hj0dchkq020ccnb3-nixos-system-box_8-24.11/init console=serial0,115200n8 console=tty1 loglevel=7
Moving Image from 0x80000 to 0x200000, end=27a0000
ERROR: RD image overlaps OS image (OS=200000..27a0000)
Boot failed (err=-14)
No working controllers found
No ethernet found.
No ethernet found.
U-Boot>

README.md

SebTM · 2024-06-22T18:33:19Z

Thanks for the fast update (and providing binary-cache to test) - the changes itself look good to me but I encountered issues compared to building against master. When booting the pi networking is somehow broken - I haven't debugged further - just took that photo when plugging a monitor for now:

Normally the pi is configured to join by wifi after boot which works when building against master with 24.05 ✌🏻

adminy · 2024-06-22T18:48:56Z

I noticed some wifi issues also. could be that the latest wifi firmware is unstable. I switched to 5ghz and the issue still persists but I see no warnings. something's not right with the firmware.

SebTM · 2024-06-22T21:07:00Z

I see, I would suggest (not a maintainer of the project) to maybe split up the PR with the "safe" / "stable" and the wireless update in separate then maybe?

tstat

Thanks for this PR! I put this stuff on github in the first place because I want to collaborate on this work with others, so I welcome this PR.

That being said, I think there are a number of uncontroversial changes in here, but they are mixed in with other unrelated changes that require more discussion.

It would be easier to consider PRs that have a more narrow focus. So, I would recommend closing this PR and opening a number of PRs, each with a specific focus (e.g. dependency bumps, adding defconfig to kernel, etc).

flake.nix

tstat · 2024-06-22T20:47:57Z

overlays/default.nix

+          kernelPatches = [];
+          defconfig = "${board}_defconfig";
+          postFixup = "";


Adding these options sounds good to me. However, most of the other changes to this file conflict with its current design I think.

For example, master supports having multiple kernel versions under pkgs.rpi-kernels, as each key in this attrset is a kernel version, and the helper pkgs.rpi-kernels.latest exists to point to the latest. The rpi-kernel binding in this let is to help build any kernel version. The changes in this branch specialize it to the provided version, replace <VERSION> with latest_<BOARD>, and drop the latest setting. None of that seems like an improvement to me.

It would probably make more sense to replace pkgs.rpi-kernels.<VERSION>.kernel with an attrset that provides the kernels as configured per board. For example: pkgs.rpi-kernels.<VERSION>.kernels.<BOARD>.

Also maybe untangle the boot loader and wireless firmware from it. Because I don't think those are board or kernel dependent. I'll see what I can do.

updated this to pkgs.rpi-kernels.<VERSION>.<BOARD>, and untangled it from the firmware. What do you think of the new code @tstat ?

I'm on mobile right now, but it looks great! I'll give it a closer look when I'm at a computer later.

overlays/default.nix

tstat · 2024-06-22T20:50:56Z

overlays/libcamera-apps.nix

-, ninja
-}:
-
+{ libcamera-apps-src, lib, pkgs, stdenv }:


Why diverge from the well-established callPackage pattern and instead pass in pkgs?

well, those are coming from pkgs, Its mentally easier to wrap your head around the fact that callPackage just provides pkgs rather than it also does a spread, which confuses me, because we're not passing all these extra params.

overlays/libcamera.nix

tstat · 2024-06-22T20:57:05Z

overlays/libcamera.nix

+    buildInputs = old.buildInputs ++ (with final; [
+      libpisp openssl libtiff
+      (python3.withPackages (ps: with ps; [
+        python3-gnutls pybind11 pyyaml ply
+      ]))
+      libglibutil gst_all_1.gst-plugins-base
+
+    ]);
    patches = [ ];
+    mesonFlags = [
+      "--buildtype=release"
+      "-Dpipelines=rpi/vc4,rpi/pisp"
+      "-Dipas=rpi/vc4,rpi/pisp"
+      "-Dv4l2=true"
+      "-Dgstreamer=enabled"
+      "-Dtest=false"
+      "-Dlc-compliance=disabled"
+      "-Dcam=disabled"
+      "-Dqcam=disabled"
+      "-Ddocumentation=enabled"
+      "-Dpycamera=enabled"
+    ];
+


Can you describe/motivate these changes?

libcamera version 2 now requires these flags to build. it was in their readme.

rpi/config.nix

tstat · 2024-06-22T21:02:40Z

rpi/default.nix

+        # Some patches cannot be applied because they are already upstream.
+        ignoreConfigErrors = true;


Can you motivate this?

Latest kernel has patches that could not be overwritten, so rather than not using nixpkgs rpi-kernels nix configuration, I just said, ignore failed patch applies. If you have another suggestion, please let me know.

…sion

It wasn't fully applied and it isn't necessary for these overrides anyway

NixOS/nix#6688

tstat · 2024-06-24T13:29:48Z

I went to build an image with this branch yesterday, but there were some syntax errors in the kernel derivation. I pushed some fixes and will test an image later today.

adminy · 2024-06-24T21:02:36Z

So a silly mistake happened. My lock file locked a version of the local copy I had, I thought I was making testable changes but it was just running an older version of the flake :D Thanks for your changes, much appreciated!

adminy · 2024-06-24T23:15:46Z

I have tested it with 6.6.34 and it works fine, with the exception of the wifi, there seems to be an issue with it being terribly slow/choppy (but no patches required). Compiling the 6.6.31 also went smoothly! No more ignore errors!
Also on 6.6.31 the wifi has no issues! So good that its left as default.

adminy · 2024-06-28T06:58:39Z

Tested again, fixed wifi issues in the latest rc pi linux it seems.

tstat · 2024-07-08T21:09:04Z

Thanks for the contribution!

adminy added 3 commits June 18, 2024 23:58

chore: bump libs & fix libcamera, fix upstream nixpkgs

c2388c3

add my kernel build cache url

5024cc2

chore: remove unused

48bb6ed

SebTM reviewed Jun 22, 2024

View reviewed changes

README.md Outdated Show resolved Hide resolved

chore: no double caching, compression now supported

4240bff

tstat reviewed Jun 22, 2024

View reviewed changes

adminy and others added 11 commits June 22, 2024 23:49

chore: return to stable versions, undo compression, fix libcamera ver…

b9c9972

…sion

chore: downgrade u-boot to stable release

aa99f3a

chore: undo dtparams, improve kernel building code

57a5323

fix: fix targets for flake, add kernel 6.6.34

b027897

run nixpkgs-fmt

9113306

fix version lookup

50e36bf

remove overrideDerivation

f0c9639

It wasn't fully applied and it isn't necessary for these overrides anyway

fix clobbering of all but one board per version

7f3a320

use final when appropriate

4f517c8

Avoid tags with + in them due to parsing bug

16d8720

NixOS/nix#6688

remove package comment

4001188

chore: remove error silencing

caed11b

chore: build latest kernel

196c7f8

chore: undo backlight

25b8813

tstat merged commit c92bd86 into nix-community:master Jul 8, 2024

adminy deleted the upgrade-with-fixes branch July 8, 2024 21:30

		# Some patches cannot be applied because they are already upstream.
		ignoreConfigErrors = true;

Uh oh!

chore: bump libs & fix libcamera, fix upstream nixpkgs #23

chore: bump libs & fix libcamera, fix upstream nixpkgs #23

Uh oh!

Conversation

adminy commented Jun 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Serial error

Uh oh!

Uh oh!

SebTM commented Jun 22, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

adminy commented Jun 22, 2024

Uh oh!

SebTM commented Jun 22, 2024

Uh oh!

tstat left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adminy Jun 22, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tstat commented Jun 24, 2024

Uh oh!

adminy commented Jun 24, 2024

Uh oh!

adminy commented Jun 24, 2024

Uh oh!

adminy commented Jun 28, 2024

Uh oh!

tstat commented Jul 8, 2024

Uh oh!

Uh oh!

adminy commented Jun 18, 2024 •

edited

Loading

SebTM commented Jun 22, 2024 •

edited

Loading

adminy Jun 22, 2024 •

edited

Loading