[RFC 0165] Bootspec v2 #165
Conversation
|
I would like to particularly invite @samueldr for clear expertise on devicetree and non-x86 systems in Nix(OS) ecosystem (of course, you may not have time, and I am already thankful for the time you provided me online). I would like @lheckemann to join me again as a co-author if that's possible. I would like to invite the previous shepherds of bootspec: @GovanifY @06jellyjac. I would like to invite my colleagues of lanzaboote: @nikstur and @blitz to comment from their perspective as they worked a lot with Bootspec v1, maybe you can even become shepherd this time! I would like to invite the initial authors of the previous revision: @grahamc and @cole-h — thank you for your support! And I am probably forgetting a lot of folks, so everyone is welcome if you want to participate in this second revision. This work is funded by the Tech Sovereign Fund as part of https://github.com/nix-community/projects/blob/main/proposals/nixpkgs-security.md#boot-chain-security (milestone 3), if someone wants to take a chunk of the money working on that with an active role, I would be glad to distribute some in the limits of classical bureaucracy and goal definitions regarding this specification. |
|
By the way, I like this RFC and I think its going in a good direction 👍 thank you! |
This is not how I understand the RFC process to work, nor how it describes itself: Lines 47 to 49 in c8569f6 RFCs are for "substantial changes" only. Reinterpreting the RFC process as granting "write-protect unless RFC" to certain files in nixpkgs, covering even minor experimental changes, would make the RFC process even more dysfunctional than it already is. See also #165 (comment) |
rfcs/0164-bootspecv2.md
Outdated
|
|
||
| The latter indicates a potentially problematic, non-upstreamed, or in-development platform. | ||
|
|
||
| The current Bootspec v1 does not formally encode information about hardcoded device trees or the folder containing available device trees. As a result, unformalized extensions are needed to address these fundamental use cases. |
There was a problem hiding this comment.
| The current Bootspec v1 does not formally encode information about hardcoded device trees or the folder containing available device trees. As a result, unformalized extensions are needed to address these fundamental use cases. | |
| The current Bootspec v1 does not formally encode information about device trees or the folder containing available device trees. As a result, unformalized extensions are needed to address these fundamental use cases. |
There was a problem hiding this comment.
(I'm fine with applying this change once we finish the discussion on DTB/DTOs and we reach to a conclusion.)
We are not saying that the RFC process grants write protect unless RFC for minor experimental changes. The extension field offers full flexibility for anyone to do even major experimental changes, we just want to ensure that all the ecosystem is protected by strong stability properties, e.g. breaking changes are strictly forbidden, semantics change in the interpretation of values are highly ill-advised (re: the discussion on nulls). Maybe what this is all hinting at is that we need another way to track the continuous life of specifications like those, I am just afraid that if we don't go through RFC, this will just push the discussion / work in an obscure place, and we will lose the diversity of people who comes to read this or are involved in this process. Hence, also why I believe that RFC process is the right place. I would definitely be able to go around and push my Bootspec v2 by myself and try to ping everyone like I did here, but I think it would reduce the discoverability of the whole ecosystem of work that Bootspec offers to anyone stumbling on it. Now, maybe, we can have the perspective of potential other members of the RFC steering committee on that? |
|
@RaitoBezarius Out of curiosity, how would the initrd secrets interact with encryption? Does the user have to encrypt them or do they have to be around in plaintext at bootloader installation time? |
I believe they ought to be around in plaintext at bootloader installation time, but I do have this open question on: should we let systemd decrypt the initrd secrets at switch to configuration time and make them available only for this script execution? Thus, we would have decryption happening only at installation by systemd! |
|
Sharing my thoughts on XBOOTLDR support inside bootspec, as per @RaitoBezarius's comments at NixOS/nixpkgs#260241 (comment), NixOS/nixpkgs#226692 (comment) and nix-community/lanzaboote#173 (comment). In UEFI systems, the EFI system partition (ESP) might have a small size and not easily resizable (e.g., 100MB on a Windows dual-boot). This results in NixOS errors when trying to store kernels and initrd from multiple generations (e.g., here and here). The solution (apart from trying to resize or relocated the ESP) is use a separate partition for storing the boot files and use the ESP only to install the bootloader. In NixOS, GRUB already supports this scenario. @RaitoBezarius suggested that XBOOTLDR-type partition should perhaps be explicitly supported inside bootspec as an extension. However, after reading more on bootspec, I'm not sure if XBOOTLDR fits in here. As was pointed out above, bootspec applies to individual generations, while configuring the ESP and XBOOTLDR partitions is a system wide action. Individual generations will not decide which partition its files should be installed to, and enabling the XBOOTLDR partition would mean files from all generations will be moved to that partition. I might be missing something and perhaps @RaitoBezarius can chime in with how bootspec and XBOOTLDR can interact? Windows unfortunately still suggests the ESP size to be 100MB (even Windows updates fails sometimes because of that), so I think NixOS should generally support the XBOOTLDR partition in it's bootloaders, whether individually in each bootloader or as a policy in something like this bootspec. |
rfcs/0164-bootspecv2.md
Outdated
| integrity. | ||
|
|
||
| In practice, initrd secrets are often employed to establish stable fingerprints | ||
| for SSH servers within the initrd, aiding in remote disk decryption on servers. |
There was a problem hiding this comment.
| for SSH servers within the initrd, aiding in remote disk decryption on servers. | |
| for SSH servers within the initrd, or aiding in remote disk decryption on servers. |
There was a problem hiding this comment.
it's not or, those fingerprints aid the remote disk decryption afaik?
There was a problem hiding this comment.
Ah remote decryption yes.
There's also a case for automatically decrypting a server's encrypted disk on boot, without having anything to do with ssh. Atleast that's how I use it.
RaitoBezarius
force-pushed
the
bootspecv2
branch
from
32072b4 to
91adbe5
Compare
tomberek
added
the
status: open for nominations
|
I may be misunderstanding here, but from what I can see, the bootloader backend's responsibility is totally unspecified when it comes to initrd secrets. Is this proposal assuming that the nix code creating a generation and the bootloader backend are just magically in agreement about this? Does no one ever switch to a different bootloader backend on an existing system with existing generations? What, precisely, is the bootloader backend required to ensure about the stage1 environment when boot.json contains a non-empty |
We can specify this but I surmise this will be relying a lot on systemd stage 1 semantics anyway. |
There was a problem hiding this comment.
We have a bunch of proposed nominations from @RaitoBezarius. I would appreciate if the nominees could accept or deny.
- @lheckemann
- @GovanifY
- @06kellyjac (assumed typo, 👍 on the comment, please explicitly accept)
- @hesiod
- @amjoseph-nixpkgs
Just a reminder that there is no special qualification required to be a shepherd. The role just involves ensuring the RFC is moving forward by collecting feedback, ensuring the RFC text evolves to incorporate any raised concerns and eventually moving towards FCP. (The details of the process are documented in the README).
There was a problem hiding this comment.
AFAIU I was invited to be co-author, not a shepherd; that said, I'd be willing to shepherd as well.
There was a problem hiding this comment.
@lheckemann do you confirm to be co-author my dear linus? :p -- OK for shepherding, thank you!
There was a problem hiding this comment.
Speaking as a member of the RFC steering committee, after people confirmed their nominations, we approve of the shepherd team as is in the RFC:
- Leader: @JulienMalka
- Shepherds: @lheckemann (or co-author) @GovanifY @06kellyjac @hesiod
|
If there is a need for more shepherds, I'd like to self nominate for that. I've been following this RFC carefully, worked with bootspec v1 implementation in the |
RaitoBezarius
force-pushed
the
bootspecv2
branch
from
91adbe5 to
8508b1c
Compare
|
@infinisil @amjoseph-nixpkgs I addressed the concern over the RFC being not interesting enough for this repo. I will ask you that we will at least mandate in this RFC the move to another repository. @grahamc @cole-h Do you have any proposal for moving the RFC in another repository? I think either nixpkgs or nix-community or nixos would be the right place to do it. I also addressed various feedback on the initrd secrets and transition period. @sdht0 I addressed your comments. @tejing1 I addressed your comments. I added @JulienMalka as a shepherd leader (after a IRL discussion). I am still waiting feedback from the other potential nominated shepherds. |
|
I updated everyone status on the specification. Please take the time to review the RFC text again, I will soon re-address the new comments. (I prefer to do it in batch.) Once we are done with that, maybe, we can setup a quick call @JulienMalka to sync everyone on that RFC and see how we want to proceed. I am also planning to do a prototype implementation before the RFC FCP as always. |
|
When using flakes, one can add Maybe this is already covered by somehow adding the information to the "label"? |
This is already covered by the label information IMHO, bootloaders decide what is the title and what information they need to encode for the label. Adding configuration revision seems out of scope just for cosmetic purpose. |
| author: Ryan Lahfa (@raitobezarius) | ||
| co-authors: Linus Heckemann (@lheckemann) | ||
| shepherd-team: | ||
| - @06kellyjack |
There was a problem hiding this comment.
| - @06kellyjack | |
| - @06kellyjac |
|
@RaitoBezarius @GovanifY @06kellyjac @hesiod @lheckemann let's set up a first sync call either the week of the 14th of the week of the 21st. Can you please fill this when2meet so that we can find an appropriate schedule for this? I think a 45mn call should be more than enough. |
infinisil
added
status: in discussion
and removed
status: open for nominations
| // (Required) The label of the system. It should contain the operating system, kernel version, | ||
| // and other user-relevant information to identify the system. This corresponds | ||
| // loosely to `config.system.nixos.label`. | ||
| "label": "NixOS 21.11.20210810.dirty (Linux 5.15.30)", |
There was a problem hiding this comment.
I'd like this to be structured information, i.e. make label an attrset that contains distroName, release, uname` etc. This is because ultimately it is the bootloader builder's resposibility to assemble this information so that it can be displayed nicely using the facilities the specific bootloader provdes. What is "nice" might also change between bootloaders.
We have this problem in Lanzaboote already where we have a suboptimal display name for a generation because we have to just use the label as is.
There was a problem hiding this comment.
I believe this is (also kinda) answered here:
My understanding here is that providing structured data is undesirable as this represents a snapshot of the already serialized boot entry configuration. It is not intended to be re-parsed for further processing.
In this instance, it is the system.nixos.label configuration, which is an arbitrary label that can be configured by the end-user.
What, really, I'm wondering is: is the comment describing it correct? Shouldn't it be just system.nixos.label, and if not, why?
(The previously linked reply also seems to misplace the ownership of the label (or really, data saved here). “bootloaders decide what is the title and what information they need to encode for the label” is wrong, as it should be coming from the system configuration, and the bootloader would use what is coming from the system configuration... no?)
There was a problem hiding this comment.
Generally the bootloader would use the generation number, except that the generation number is nowhere to be seen in the bootspec, and for good reason, since it's the one thing that's guaranteed to change between rebuilds, while the boot.json resides within the system path itself, which is only supposed to depend on what's explicitly provided to nix at build-time. Bootspec doesn't even provide any information that other generations even exist beyond specialisations for that particular build.
So bootloaders are already forced to look elsewhere than just the provided system path in order to generate its boot entries. At that point, why not just take a peek at /etc/os-release to get more granular information and reduce some of the duplication? If the boot menu has any kind of hierarchy, then saying "NixOS" for every single subentry seems unnecessary when the version information is all you need. The only information in label that isn't in os-release is the kernel version.
There was a problem hiding this comment.
Any bootloader implementation can peek at the /etc/os-release as long as it's determined at build-time and generated an appropriate label.
Furthermore, there's no reason to make /etc/os-release a dependency, someone could use bootspec on systems without it and with a special bootloader implementation making use of another knowledge.
The kernel version is also known at build-time. There's no need to literally read /etc/os-release per se.
Ideally, the bootloader should try to honor as much as possible the label or come up with better generation of the label, sure, in lanzaboote, we have a problem that is, we need better tools to format the title because we can run in various excessive cleverness from systemd-boot that forces us to assume something about the label itself and manipulate it. As soon someone overrides forcibly that label, we are doomed.
But customization of boot title entry is a complicated matter, one, that exceeds the scope of bootspec maybe…
To put in another way, there's a tension between "bootspec's label is an opaque string that must be honored" and "my bootloader is doing weird things to the sorting order of the label / eating some words / I would like to show a 2-level menu and I don't know where to split the label string".
The resolution of this situation might be to have preferredLabel and the data @nikstur proposes to have which provides the preferred label in a structured way which the bootloader implementation may decide to reassemble for limitations of the software or to better accommodate the UI. Though, this increases the complexity of the design and opens a can of worms.
|
@06kellyjac @hesiod I invited you both to a matrix room so that we can synchronously converge towards dates for our meeting, could you join please ? |
| Moreover, an additional initrd is frequently used to store CPU microcode. To | ||
| ensure compatibility and flexibility, it is essential to rework the initrd | ||
| support in the Boot Loader Specification. The proposed changes aim to treat | ||
| initrds as a set, allowing bootloaders to handle a list of initrds or a single |
There was a problem hiding this comment.
It's more of a list than a set, right? Ordering matters.
There was a problem hiding this comment.
That's correct but I guess I should add some remarks on what is possible in terms of ordering.
For example, CPU microcode must always come first, uncompressed.
There was a problem hiding this comment.
That makes it all the more important :D I was mostly thinking of precedence with colliding paths, but microcode-first is obviously especially important. I'm not sure we even need to detail it that much in the text, but we shouldn't call it a "set" :)
There was a problem hiding this comment.
| initrds as a set, allowing bootloaders to handle a list of initrds or a single | |
| initrds as a list, allowing bootloaders to handle a list of initrds or a single |
| - Improve non-x86 support in Bootspec, emphasizing the importance of | ||
| devicetrees. | ||
| - Address and reduce the risks associated with initrd secrets, providing a | ||
| default "secure" implementation within the systemd ecosystem and offering |
There was a problem hiding this comment.
How will we deal with the scripted initrd? This RFC feels a bit like we're going to define bootspec v2 and the implement it only for the nice new stuff, but I'd expect that we want it to replace v1 in which case we'll also need to implement support for it in bootloaders other than systemd-boot and in the scripted initrd.
There was a problem hiding this comment.
My current feeling about scripted initrd is that we will start deprecation in 24.05, thus, getting the nice features for it seems to be a waste of energy, but I may be wrong (?).
We can keep v1 working as-is for scripted initrd as we go in deprecation of v1 too, no?
| // - In particular, there is no expectation that such nested specialisations will be handled by consumers of bootspec documents. | ||
| // - Each specialisation document may contain arbitrary further keys (extensions), like the top-level document. | ||
| // - The semantics of these should be the same as when these keys are used at the top level, but only apply for the given specialisation. | ||
| "org.nixos.specialisation.v2": { |
There was a problem hiding this comment.
I don't think this really needs a version bump?
There was a problem hiding this comment.
Well, each specialization must have a valid v2 bootspec document, so it's a change in the expectations of the v1 specialization specification, so it deserves a bump (?).
(I also like bumps to test fully our capabilities to avoid ossification though.)
| In practice, initrd secrets are often employed to establish stable fingerprints | ||
| for SSH servers within the initrd, or aiding in remote disk decryption on servers. | ||
| To address these issues, this RFC proposes moving away from the "appender | ||
| script" model in Bootspec v1 and instead adopting a hash map format to |
There was a problem hiding this comment.
"hash map" seems like a weird term to use here, since the format we're actually using (JSON) has nothing to do with hashes. How about "JSON object"?
| // The legacy behavior is to prepare a CPIO archive for each file and | ||
| // extend the `initrds` fields with those CPIO archives. |
There was a problem hiding this comment.
| // The legacy behavior is to prepare a CPIO archive for each file and | |
| // extend the `initrds` fields with those CPIO archives. | |
| // The legacy behavior is to prepare a CPIO archive for each file and | |
| // concatenate these with the initrds from the `initrds` field to form | |
| // a single initrd for boot. This exposes the secrets to anyone who can | |
| // read the boot partition, and is discouraged in favour of mechanisms | |
| // which provide stronger protection for the secrets. |
| // The legacy behavior is to prepare a CPIO archive for each file and | ||
| // extend the `initrds` fields with those CPIO archives. | ||
| // Make sure the location where the secrets are dropped in the initrd are visible | ||
| // for the user. |
There was a problem hiding this comment.
Unclear, what does this mean?
There was a problem hiding this comment.
Imagine you are activating the script in a rescue system and the secrets are in /mnt/etc/secrets/... instead of /etc/secrets/... and you are not fully chrooted inside /mnt or whatever, the location won't be visible at assembly time.
This sort of problem cause a lot of difficult to understand to most users because activation now depends on an invisible files to assemble bootables (arguably, this is not the only mechanism like this we have.).
How would you reframe this concept?
|
On February 22nd we had a RFC meeting. Overall, we agreed that there do not seem to be anything controversial preventing this RFC to go ahead. Most still open point are not big blockers and are handled asynchronously. The topic of providing structured information instead/alongside the label (see here) seem to be worthy of a synchronous discussion between stakeholders. We suggest scheduling a meeting to find a satisfactory way forward and this matrix channel can serve to organize such a meeting. (Ping @nikstur, @samueldr, @boomshroom, @RaitoBezarius) During the meeting, we also decided that ideally, some implementation work should be done before the RFC get merged, mainly updating https://github.com/DeterminateSystems/bootspec and the NixOS module. Migrations steps for the backends is also a topic that was discussed and that should be addressed in the RFC text, but it was discussed that to ensure backward and forward compatibility, migration between bootspec
|
|
(I'm not a shepherd... Though I can be pinged/asked questions about boot given my domain knowledge.) |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/rfcsc-meeting-2024-03-05/40851/1 |
Should this repository be moved under the NixOS organization? |
|
Hey shepherds, even if implementation is still ongoing, you can move this into FCP. The implementation doesn't need to be completed before this RFC has been merged. Seems like there's agreement on the direction of this RFC; unless there are still outstanding concerns, the RFCSC recommendation would be to move to FCP. |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/rfcsc-meeting-2024-03-19/41829/1 |
|
The FCP cannot happen before a first implementation is done in the
ecosystem, IMHO.
Le mar. 19 mars 2024, 09:02, nixos-discourse ***@***.***> a
écrit :
… This pull request has been mentioned on *NixOS Discourse*. There might be
relevant details there:
https://discourse.nixos.org/t/rfcsc-meeting-2024-03-19/41829/1
—
Reply to this email directly, view it on GitHub
<#165 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AACMZRFWDPFP7PE3G7WGV4TYZBORRAVCNFSM6AAAAAA6ZOZO42VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBXGU3TIMBXGE>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/rfcsc-meeting-2024-04-02/42643/1 |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/rfcsc-meeting-2024-04-16/43512/1 |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/rfcsc-meeting-2024-05-14/45414/1 |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/rfcsc-meeting-2024-05-28/46113/1 |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/rfcsc-meeting-2024-06-10/46817/1 |
|
RFCSC: Since there's no progress, we'll mark this as a draft. Feel free to undraft at any time when you're continuing with it. |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/rfcsc-meeting-2024-06-24/47589/1 |
This introduces the second revision of RFC-0125, Bootspec, addressing the feedback we received in #125 and building on our experience of Bootspec v1.
Rendered
TODO:
FDTDIR/devicetreeinto… nothing or justFDTDIR?