[RFC 0153] Non-legacy boot NixOS

[RaitoBezarius]

Expectations : I do not expect to move forward with this proposal before a fair amount of time, this is extremely early sneak peak of something I want for NixOS in the next years.

Rendered

[7c6f434c]

Keeping the option, (switching to just tracking the snapshots,) but change the defaults for new users?

[RaitoBezarius]

Yes, definitely

[RaitoBezarius]

(but this alternative could also be considered my proposal as in: I want to introduce long term deprecation then removal of this)

[7c6f434c]

GRUB2 gives a lot of nice features beyond BIOS Boot, even features that I have had to use, though.

On the other hand, making it non-default (not that everyone knows or wants to learn using those features…) could be done on a much shorter time horizon than deprecation discussions, I think

[RaitoBezarius]

Fair @ making it non default.

(But, to be completely fair, this will turn the GRUB2 question into a "maintenance in-tree" issue as currently, no one really is interested into this problem in nixpkgs.)

[7c6f434c]

I think that's an overkill, whatever snapshot passes our tests plus maybe a stolen patch from another downstream here and there might be enough

[RaitoBezarius]

Sure, what I am saying is no one is even doing the "stolen patch" except people who actually doesn't care that much about GRUB :).

[7c6f434c]

I mean, if you don't have Secure Boot enabled anyway, GRUB just works on a ton of systems even without maintenance. Churn is a problem, not an end-goal positive, after all.

[RaitoBezarius]

This is simply not true: NixOS/nixpkgs#235222 we already had boot broken (I don't remember if we had issues, but we had people coming to us on that subject in the Matrix channels) for our users recently because as I repeat it in the RFC (and this was cited in the original RFC text and still is.)

GRUB maintenance is not done, it is the default bootloader and our users do not see any real reason to change of bootloader.

I would not have any beef with this situation if GRUB was some optional bootloader, and we had replacements, but GRUB lack of maintenance is pushing churn on other maintainers who doesn't care at all about GRUB and has to do the work for it and has to consider it, if the answer to that is: "GRUB doesn't need maintenance anyway".

I am very inclined now to ensure we don't default for GRUB anymore for UEFI boot right now.

[7c6f434c]

Oh right, copyKernels is not the default, that could lead to catches. (But I agree that switching defaults is reasonable and probably should be just done without RFC)

[SuperSandro2000]

I think this suffers the same problems as NixOS/nixpkgs#202526 and since bios implementations are even weirder and more complicated I believe this could get real hairy but I appreciate the effort and would love a focus on UEFI systems.

[ghost]

This RFC isn't about UEFI-boot vs non-UEFI boot.

It's about GRUB. Just GRUB. Nothing in the "Motivation" section applies in a non-GRUB scenario.

Could you please scope it down to just deal with GRUB specifically?

[ghost]

Suggested change

	In nixpkgs, legacy boot forces a dichotomy between `boot.loader.efi` and... two legacy bootloaders: GRUB and a family of UBoot/extlinux-compatible/etc.
	In NixOS, legacy boot forces a dichotomy between `boot.loader.efi` and... two legacy bootloaders: GRUB and a family of UBoot/extlinux-compatible/etc.

[RaitoBezarius]

This RFC isn't about UEFI-boot vs non-UEFI boot.

It's about GRUB. Just GRUB. Nothing in the "Motivation" section applies in a non-GRUB scenario.

Could you please scope it down to just deal with GRUB specifically?

Uhm, I'm not so sure, the proposal is about making platforms which can afford a UEFI environment to prefer exclusively UEFI, I am open to discussing how to relax this to make it realistic and I had special thoughts to you because I know you have interesting opinions on the whole subject (re: ownerboot, etc.).

If we made it to deal with GRUB, I would see 2 problems:

• It would make it an RFC basically "anti GRUB", something that I can already fix without doing an RFC IMHO, I can just go and implement the U-Boot proposal, then switch the default bootloader for our images to it, I believe. Which does not warrant a whole, full-blown RFC.
• It does not really address the "Nixpkgs developers can assume that end users runs UEFI as a default environment for supported architectures" motivation, does it?

[alyssais]

I don't think it would be at all good to assume that end users run UEFI, even on architectures where they could. Even on x86_64, I don't see any reason we should discourage people from running ownerboot, LinuxBoot, etc. There's nothing legacy about these bootloaders, despite what the RFC text says.

What benefit would there be to Nixpkgs developers being able to "assume that end users runs UEFI"? There aren't any mentioned, only complaints about GRUB.

[7c6f434c]

"Nixpkgs developers can assume that end users runs UEFI as a default environment for supported architectures" motivation

Uhm, can we please narrow this down? Like, for Nixpkgs development assuming anything about the specifics about how the system has been booted sounds really bad unless we are talking about UEFI management tools (those, indeed, can assume UEFI regardless of whatever NixOS defaults to!)

[RaitoBezarius]

I don't think it would be at all good to assume that end users run UEFI, even on architectures where they could. Even on x86_64, I don't see any reason we should discourage people from running ownerboot, LinuxBoot, etc. There's nothing legacy about these bootloaders, despite what the RFC text says.

I agree that defining the RFC based on non-UEFI is not great, I will rework the RFC text based on that to talk only about non-legacy boot as defined by the BIOS Boot Specification.

What benefit would there be to Nixpkgs developers being able to "assume that end users runs UEFI"? There aren't any mentioned, only complaints about GRUB.

Yes:

(1) The default ISO could not boot GRUB by default anymore: this is a win for developers, see GRUB complaints for rationale.
(2) Users could benefit in the future of NixOS/nixpkgs#84204 on a larger scale than just UEFI users running systemd-boot.
(3) Strange ESP filesystem booting can be relegated to the UEFI environment via https://efi.akeo.ie/ (which consumes GRUB drivers anyway): making it easier to deal with the "I want to boot on a ZFS ESP" problem and offering a clear way to achieve it, if this is even wanted.
(4) Boot testing can split into two parts: testing legacy boot environment boot correctly into a UEFI-enabled environment with the required features we want to support, testing that UEFI-enabled environment works with the features we are interested.

(we could also use this opportunity to upgrade our AWS AMI on x86 which are running BIOS when UEFI is available, but this is only tangent to the problem.)

Again, this RFC says in the first lines:

Expectations : I do not expect to move forward with this proposal before a fair amount of time, this is extremely early sneak peak of something I want for NixOS in the next years.

I am happy it sparked discussion and strong reactions already, as I repeated, I am totally not interested into removing the ability to boot non-UEFI things, my focus has only to do with legacy boot.

I also see that if we make this switch successfully, this will empower other distributions and other ecosystems to deem that non-legacy-boot is a valid strategy to run a Linux system.

Furthermore, I will edit the RFC text to make it clearer, but I do not expect to get out of draft mode really soon, I prefer rather linking implementation PRs here also to give context to those works and I would love to make it easier as part of this work to make it possible to use ownerboot/linuxboot easier in NixOS as the mechanism to do chain-load a 2nd stage UEFI payload can be abstracted for any 1st stage "environment" → 2nd stage "whatever", which in, ownerboot/linuxboot/any relevant (c)oreboot payload makes a lot of sense AFAIK.

[RaitoBezarius]

"Nixpkgs developers can assume that end users runs UEFI as a default environment for supported architectures" motivation

Uhm, can we please narrow this down? Like, for Nixpkgs development assuming anything about the specifics about how the system has been booted sounds really bad unless we are talking about UEFI management tools (those, indeed, can assume UEFI regardless of whatever NixOS defaults to!)

While I do understand your point, Nixpkgs development already makes a lot of assumptions about the specifics about how the system has been booted.

For example, if you run a default ISO of NixOS, you will have expectations on your firmware, e.g. supporting the BIOS Boot Specification or UEFI.

I would be interested to understand how do you imagine things because I don't know how to write boot.loader.* code that does not make assumption on how the system has been booted in general.

[7c6f434c]

(2) Users could benefit in the future of NixOS/nixpkgs#84204 on a larger scale than just UEFI users running systemd-boot.

This seems to be a logical impossibility? A change in UEFI systemd-boot will only benefit users of systemd-boot… You can say that the value provided merits defaulting more users into UEFI with specifically systemd-boot, sure.

Expectations : I do not expect to move forward with this proposal before a fair amount of time, this is extremely early sneak peak of something I want for NixOS in the next years.

I don't see the arguments raised (by anyone) as «having an expiration date», so I don't see what reiterating this is intended to imply.

I also see that if we make this switch successfully, this will empower other distributions and other ecosystems to deem that non-legacy-boot is a valid strategy to run a Linux system.

I think enough people are running UEFI boot on NixOS that whatever validation is already achieved?

For example, if you run a default ISO of NixOS, you will have expectations on your firmware, e.g. supporting the BIOS Boot Specification or UEFI.

This is not Nixpkgs development.

Nixpkgs — the package collection — should only care about UEFI in the small subset of packages, namely, UEFI bootloading/management related packages.

NixOS — the OS, runnable in VMs, containers, laptops, servers, etc. — should only care about UEFI in the bootloading part and nowhere else. Even better with boot specification where the data about a configuration can be reported and then a separate script configures the bootloader!

boot.loader.efi.* code (or whatever it is called) assumes UEFI, I hope, even today. And this can't really change. How existence of legacy boot code makes any difference to assumptions on the UEFI side is unclear to me. Changing defaults for each specific platform (they don't even need to change at once) is another question. But RFC as written is not about that.

[RaitoBezarius]

I updated the RFC text, cc @alyssais @7c6f434c @amjoseph-nixpkgs, please take another look.

[RaitoBezarius]

(2) Users could benefit in the future of NixOS/nixpkgs#84204 on a larger scale than just UEFI users running systemd-boot.

This seems to be a logical impossibility? You can say that the value provided merits defaulting more users into UEFI with specifically systemd-boot, sure.

I'm not sure that I follow how this is a logical impossibility, but I agree with your second statement.

Expectations : I do not expect to move forward with this proposal before a fair amount of time, this is extremely early sneak peak of something I want for NixOS in the next years.

I don't see the arguments raised (by anyone) as «having an expiration date», so I don't see what reiterating this is intended to imply.

I do see for legacy BIOS an expiration date though, through, potentially X86S for example.

I also see that if we make this switch successfully, this will empower other distributions and other ecosystems to deem that non-legacy-boot is a valid strategy to run a Linux system.

I think enough people are running UEFI boot on NixOS that whatever validation is already achieved?

I don't understand this remark, the question is whether adopting UEFI environments via a UBoot polyfill is a valid strategy to have more users defaulting to UEFI, not whether people are running UEFI boot on NixOS?

For example, if you run a default ISO of NixOS, you will have expectations on your firmware, e.g. supporting the BIOS Boot Specification or UEFI.

This is not Nixpkgs development.

I used Nixpkgs development as in development in https://github.com/nixos/nixpkgs not in Nixpkgs the software collection, but sure.

NixOS — the OS, runnable in VMs, containers, laptops, servers, etc. — should only care about UEFI in the bootloading part and nowhere else. Even better with boot specification where the data about a configuration can be reported and then a separate script configures the bootloader!

boot.loader.efi.* code (or whatever it is called) assumes UEFI, I hope, even today. And this can't really change. How existence of legacy boot code makes any difference to assumptions on the UEFI side is unclear to me. Changing defaults for each specific platform (they don't even need to change at once) is another question. But RFC as written is not about that.

Furthermore, I changed the RFC to make it clearer, I hope, let me know if this is still not.

[7c6f434c]

I think now RFC is kind of in a split-brain mode where it has scales of ambition in different segments?

I am not sure the change-the-defaults plan benefits from an RFC now — maybe a narrow requirement discussion «if you would consider switching to from BIOS Boot to polyfill+UEFI+systemd-boot, what features you have now that you'd want to keep» would be more focused? (I guess I would not gave much to say there, as a UEFI GRUB2 user…)

I very much doubt that an option for legacy-boot-polyfill (allowing you to enable UEFI bootloader too) would meet any opposition, but of course a poll about desirable features could inform the design. Once it is tested on many systems, I guess discussion of requirements for making it default could benefit from real-world data?

[RaitoBezarius]

I think now RFC is kind of in a split-brain mode where it has scales of ambition in different segments?

I think the only thing this RFC wants is:

(1) building consensus on 2-stage payloads infrastructure for boot
(2) building consensus on deprecation schedule for phasing out legacy boot and polyfilling with UEFI by default, for new users, then old users.

It may have been worded completely wrong from the very start and I apologize for this, I hope it is clearer now.

I am not sure the change-the-defaults plan benefits from an RFC now — maybe a narrow requirement discussion «if you would consider switching to from BIOS Boot to polyfill+UEFI+systemd-boot, what features you have now that you'd want to keep» would be more focused? (I guess I would not gave much to say there, as a UEFI GRUB2 user…)

It depends highly on how much time takes this RFC, if we will take 2 or 3 years to decide on a lot of things in this RFC, I prefer we will consider the change-the-defaults in this RFC.

On the features question, I would agree, but I feel like it comes after we agree on the core concepts first of the idea. I already put some serious drawbacks to a UEFI polyfill, so, those must be accepted before we wonder what features we would like to keep.

I very much doubt that an option for legacy-boot-polyfill (allowing you to enable UEFI bootloader too) would meet any opposition, but of course a poll about desirable features could inform the design. Once it is tested on many systems, I guess discussion of requirements for making it default could benefit from real-world data?

An option could be made, but to make interesting architecture decisions, I feel it's important to discuss an RFC to the architecture of such option, before starting large scale works in nixpkgs.

Making it default could be made conditional on a level of feedback data as part of this RFC and it would take place automatically without needing another RFC.

[7c6f434c]

On the features question, I would agree, but I feel like it comes after we agree on the core concepts first of the idea.

Wait, precommitting to an architecture as a prerequisite to collecting functional requirements, not the other way round?

[RaitoBezarius]

On the features question, I would agree, but I feel like it comes after we agree on the core concepts first of the idea.

Wait, precommitting to an architecture as a prerequisite to collecting functional requirements, not the other way round?

Functional requirements for what? I feel like this is for (2), (1) is abstract and general, I believe the problem of (1) is the "bootloader composition" problem, am I wrong?

[7c6f434c]

In what sense do you hope to define meaningful agreement on the idea of bootloader composition? If the idea is to get a roadmap to making things default before starting large scale work, isn't it more natural to discuss what desirable properties seem achievable or non-achievable as a part of a roadmap to default?

[RaitoBezarius]

In what sense do you hope to define meaningful agreement on the idea of bootloader composition? If the idea is to get a roadmap to making things default before starting large scale work, isn't it more natural to discuss what desirable properties seem achievable or non-achievable as a part of a roadmap to default?

This RFC is a draft as a place to point people potentially interested into working towards that.

My approach is to consider things from top to bottom in this scenario, expressing high level properties wanted, dive into it, expressing lower level properties, and so on.

I do hope that a proper design of (1) should make (2) quite trivial to implement, (2) considerations can be kept in mind, but I don't feel like they should leak into the design of (1).

Right now, my bootloader composition proposal is to offer composition of 2 bootloaders only in the same system, this is open for discussion, extension, rejection.

[7c6f434c]

What does it mean, only on the same system? Whether it is NixOS who installs UEFI-emulation layer, and whether NixOS uses UEFI wherever it might come from — these sound two separate concerns?

[RaitoBezarius]

What does it mean, only on the same system? Whether it is NixOS who installs UEFI-emulation layer, and whether NixOS uses UEFI wherever it might come from — these sound two separate concerns?

On a high level point of view, yes. But, in practice, those are not separate concerns as long as the only "entrypoint" for bootloader in https://switch-to-configuration.pl stays installBootLoader script.

Which means this technical limitation has to change before to consider high level constructs such as the ones you mention.

And there's no obvious ways to change this technical limitation AFAIK.

[7c6f434c]

On a high level point of view, yes. But, in practice, those are not separate concerns as long as the only "entrypoint" for bootloader in https://switch-to-configuration.pl stays installBootLoader script.

???

The script passed is a concatenation of «maybe install a BIOS Boot bootloader», «maybe install UEFI bootloader», «maybe install U-Boot», «maybe install UEFI compatibility chainloader». OK, the last part is not there yet and would take significant work to implement, but whether to include each part is a separate concern.

[RaitoBezarius]

On a high level point of view, yes. But, in practice, those are not separate concerns as long as the only "entrypoint" for bootloader in switch-to-configuration.pl stays installBootLoader script.

???

The script passed is a concatenation of «maybe install a BIOS Boot bootloader», «maybe install UEFI bootloader», «maybe install U-Boot», «maybe install UEFI compatibility chainloader». OK, the last part is not there yet and would take significant work to implement, but whether to include each part is a separate concern.

This is beyond the point: the issue with this is the architecture of NixOS modules and how they work together to assemble this script, BTW, I'm not convinced this is a good idea to just perform concatenation of Maybe especially given you need to express ordering controls towards this.

Also, for things like GRUB, this "concatenation" is already broken, installing UEFI-only systems with GRUB requires you to use nodev because the Perl script has a bug where they will force-try to install BIOS systems on UEFI-only systems if you pass an ESP device as a parameter of the NixOS module system.

Anyway, again, I am in favor of bootloader compositions because they enable you to do much more with better semantics, for example, lanzaboote is forced to disable systemd-boot because it replaces the installation script for systemd-boot, though, the systemd-boot installer script could be composed with the lanzaboote install script to achieve the same effect, the only problem is there's no easy ways to perform such compositions with information-preserving semantics that provides the rail guards and safety features required for end-users, hence the need of working a bit more on that.

[7c6f434c]

Anyway, again, I am in favor of bootloader compositions because they enable you to do much more with better semantics, for example, lanzaboote is forced to disable systemd-boot because it replaces the installation script for systemd-boot, though, the systemd-boot installer script could be composed with the lanzaboote install script to achieve the same effect, the only problem is there's no easy ways to perform such compositions with information-preserving semantics that provides the rail guards and safety features required for end-users, hence the need of working a bit more on that.

Ah, thanks for the explanation, now the scale of the ambition os much clearer!

[RaitoBezarius]

Anyway, again, I am in favor of bootloader compositions because they enable you to do much more with better semantics, for example, lanzaboote is forced to disable systemd-boot because it replaces the installation script for systemd-boot, though, the systemd-boot installer script could be composed with the lanzaboote install script to achieve the same effect, the only problem is there's no easy ways to perform such compositions with information-preserving semantics that provides the rail guards and safety features required for end-users, hence the need of working a bit more on that.

Ah, thanks for the explanation, now the scale of the ambition os much clearer!

Thank YOU for bearing with me and doing the ping-pong!

[ghost]

Nixpkgs development already makes a lot of assumptions about the specifics about how the system has been booted.

Like what? I think you're severely confusing nixpkgs with nixos.

• Nixpkgs developers can assume that end users runs UEFI as a default environment

This definitely crosses a red line for me. It probably does for the *-darwin users too since none of them use UEFI even though all of their hardware belongs to "UEFI-supported architectures".

Expectations : I do not expect to move forward with this proposal before a fair amount of time, this is extremely early sneak peak of something I want for NixOS in the next years.

I think this is a very inappropriate reason for a draft PR in the RFCs repo.

I'm going to unsubscribe from this PR now, because I really don't need to be constantly pestered by every new comment on it. You can ping me or request a review when you have something more concrete/imminent.

[ghost]

Suggested change

	and uses it as a default boot environment, for supported architectures,
	and uses it as a default boot environment, on supported machines,

UEFI does not "support" entire architectures in any meaningful sense. It "supports" MIPS and yet there is not a single UEFI MIPS machine in existence! Talking about "UEFI-supported architectures" is extremely misleading.

[ghost]

Please no. If you want these Rube Goldberg contraptions on your machine, great. Don't force them on everybody else.

[ghost]

I stopped reading at this point.

[RaitoBezarius]

Not really if coreboot/heads support comes with that RFC. Le dim. 13 août 2023 à 16:48, enc0urage ***@***.***> a écrit :

…

As a coreboot/heads <https://github.com/osresearch/heads> user this is likely to cause me headaches. You're also likely to run into vendor UEFI implementation issues. I have ran a config that created an UEFI entry per NixOS generation on a remote machine and the ASUS motherboard eventually bricked itself from having too many efibootmgr entries (required physical access for CMOS reset.) Since then I believe the secondary bootloader is a necessary abstraction from having to trust the proprietary hardware vendors to implement UEFI correctly. — Reply to this email directly, view it on GitHub <#154 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACMZRDAIL76A725IJHW7BTXVDSLNANCNFSM6AAAAAAZJMYYYM> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[RaitoBezarius]

What I meant is most coreboot setups do legacy boot with SeaBIOS and Heads uses kexec, but gets the boot order out of parsing grub.cfg. Currently it just works, but deprecating legacy boot and GRUB would break those.

There are two things to unpack here:

(a) Most coreboot setups do legacy boot with SeaBIOS → if coreboot is usable, you can just swap the payload for linuxboot and shorten the boot path
(b) Getting the boot order out of parsing grub.cfg seems orthogonal to have GRUB running or did I miss something?

As for deprecations, having thought about this RFC and the ongoing problems with GRUB (and legacy boot), I think there is nothing preventing NixOS to offer a way to plug in new bootloaders out of tree in NixOS systems.

Therefore, unless someone take upon the maintenance mantle (which I really wish), all the existing code may just get moved in an out of tree repository in a "as-is" state for everyone who have a "just works" setup with it.

Another vendor implementation "quirk" I've come across: My ThinkPad X270 loses all efibootmgr entries after UEFI capsule update via fwudp. I get around it with boot.loader.grub.efiInstallAsRemovable.

Yes, I'm aware of a lot of UEFI broken firmware out there. This is part of the design space to understand how to meaningfully take them into account, e.g. use only EFI fallback binaries, second stage bootloaders, etc.

[nyabinary]

I generally agree with this on all points except for dropping GRUB, GRUB has UEFI support, and we shouldn't just deprecate a whole bootloader people rely on because the way it's currently implemented is a mess.