[Merged by Bors] - lib/devos: bake devos repo into live cd #168

blaggacao · 2021-03-15T04:18:05Z

This worked for me to bootstrap another machine.

blaggacao

Two specific doubts, and a general one:

Are store paths actually copied over from the iso?

I couldn't tell from: https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/installer/tools/nixos-install.sh#L143

lib/devos/devosSystem.nix

blaggacao · 2021-03-15T04:22:19Z

doc/start/iso.md

+_Note: You _could_ install another machine than the one your iso was built for,
+but the iso doesn't necesarily already carry all the necesary build artifacts._
+
+<!-- TODO: find out why --impure is necesary / PRs welcome! -->


I was stumbling on and over access to path '/mnt/nix/...' is forbidden in restricted mode

nrdxp · 2021-03-15T17:49:11Z

This is looking pretty good. Have your considered also adding isoImage.includeSystemBuildDependencies. It may help with offline installation at the cost of a bigger image.

blaggacao · 2021-03-15T18:00:23Z

This is looking pretty good. Have your considered also adding isoImage.includeSystemBuildDependencies. It may help with offline installation at the cost of a bigger image.

Cool! I'll do a couple of tests today, still, and will include that.

I also want to experiment to also set a static link local ip on the network interface when using the live cd. That way, we can feed a well known ip to the deploy when the live cd is running: so a technician can plug in the usb stick and a devops can do the depoyment "remotely" (if they are on the same link local network). At least the devops doesn't have to crawl around under the office desks.

Pacman99 · 2021-03-15T18:04:16Z

Isn't hostname usually enough for that? I'm not sure about adding a static ip, because of possible ip conflicts and other issues that can happen. Worst case you could find the ip of the machine by searching by hostname.

blaggacao · 2021-03-15T18:06:47Z

Isn't hostname usually enough for that?

Host name needs identity (which we actually have here since we build bespoke iso installers, but we also might want to work with a generic hostconfig, instead, eg. NixOS) and also dns to attest / route identities.

I still want to cover the case of an anonymous bootstrap image / iso that we can burn onto stockpiled computers and then ship to the site and initiate with an identity over the network. (kind of a rescue partition)

Pacman99 · 2021-03-15T18:10:40Z

I definitely understand the benefits of static IP. I've just had issues with it on my network and I know it can be kind of finicky. And I don't want the machine to lose network connection entirely for the convenience of a static ip.

blaggacao · 2021-03-15T18:13:16Z

I've just had issues with it on my network and I know it can be kind of finicky. And I don't want the machine to lose network connection entirely for the convenience of a static ip.

I think that is the exact use case for link local ad hoc networks

IPv4 Link-Local addresses are not suitable for communication with
devices not directly connected to the same physical (or logical)
link, and are only used where stable, routable addresses are not
available (such as on ad hoc or isolated networks)

https://tools.ietf.org/html/rfc3927

So we should be good.

Two anonymous hosts at the same time on the same link will necessarily clash.

Pacman99 · 2021-03-15T18:14:30Z

Also I was thinking that we should extract the custom system builds outside of lib, so users can add their own custom builds for all their hosts. The idea behind devosSystem is really great and I found a use for it in the home manager PR, and I'm sure others might have good ideas. And they can customize the existing systems we configure.

Pacman99 · 2021-03-15T18:15:43Z

I think that is the exact use case for link local ad hoc networks

I'll have to learn more about that. But it looks like you have a good plan!

blaggacao · 2021-03-15T19:25:09Z

isoImage.includeSystemBuildDependencies

It looks as if this easily exceeds the size of an usb flash drive.

But foremost, I hope that auto?trusted=1 substituter actually let's the bespoke iso image act as a substituter. Since the repo baked into it is the same as the moment the image was created, I suppose that all (or almost all) builds are copied over from the installer iso and no build is done on the target.

nrdxp · 2021-03-15T19:50:04Z

It looks like this is getting pretty close. I'm going to delegate this to both of you, while I work on opening a further PR toward #152. There is no harm in trying a bors r+ to merge, but it will fail til I fix the CI runner. I have a few more emails to respond to for the day, and then I can get it fixed up. Good work.
bors delegate+
bors delegate=@Pacman99

bors · 2021-03-15T19:50:06Z

✌️ blaggacao can now approve this pull request. To approve and merge a pull request, simply reply with bors r+. More detailed instructions are available here.

bors · 2021-03-15T19:50:06Z

✌️ Pacman99 can now approve this pull request. To approve and merge a pull request, simply reply with bors r+. More detailed instructions are available here.

blaggacao · 2021-03-15T20:42:50Z

I'm checking out MulticastDNS as an alternative to static IPs. that might be slightly friendlier to the end user, althouth I'm not sure if that depends on fancy router support...

that would look something like deploy ... bare-host.local. or deploy ... bootstrapping.local.

EDIT: instead of a ipv4 link-local, as a fall back to MulticastDNS, we can use ipv6which has it built-in, like in: https://networklessons.com/ipv6/ipv6-eui-64-explained

blaggacao · 2021-03-16T02:01:38Z

doc/start/iso.md

+The latter option is currently blocked by [Nix Issue 4643](https://github.com/NixOS/nix/issues/4643).
+
+TODO: The former did not complete on my machine as the MulticastDNS seemed to be too unstable.


blocked by NixOS/nix#4643

The NixOS.local MulticastDNS didn't seem to be utterly stable on my network and disappeared rather quickly (between the deploy's nix copy — suceeded — and remote activation — failed)

lib/devos/devosSystem.nix

blaggacao · 2021-03-16T12:30:04Z

I'll do another manual test, cleanup & rebase today, so we should be close to ".good to go".

Pacman99 · 2021-03-16T16:32:26Z

This looks really good. I think the only thing I would like to change is to use self instead of ../.. to reference the flake. But that could be done separately too.

Pacman99 · 2021-03-16T16:36:01Z

Also I had another idea for the live cd. You could include all the inputs in the ISO's registry. So while installing you don't need to download the inputs again. Once all inputs get passed to lib you could do something like:

nix.registry = mapAttrs (n: v: ${n}.flake = ${v}) inputs;

This would also require adding inputs@{...} to the top.

But not sure this would work, and probably best to save it for a future PR.

It is generally useful to acess the top level flake from library functions or hosts. This not only simplifies the mental model and code but also provides additional context and not least a handle to the repo source code in the nix store. closes divnix#169

Copy over the nix storepath contents of the top level flake in order to be able to boostrap a host from a live iso installer conforming to the devos documentation using it's devshell. closes divnix#167

Replace default networking with a specially configured systemd-networkd network which is responsible for setting up DHCP and also provide a well known ipv6 link-local address as well as enable multicastDNS to expose the hostname on the local link over the reserved `local.` top level domain.

blaggacao · 2021-03-16T18:52:11Z

Lat's call this a day! We might consider opening issues / discussions for further improvements:

use ipv4 link-local to tie it to a cachix derviation that represents the host's profile and let the stick autoprovision itself (unattended mode).
put the inputs in the registry so that they don't need downloading (aka. aire gapped installation — to be checked if that is actually enough)

This includes #189

blaggacao · 2021-03-16T18:52:54Z

bors r+

bors · 2021-03-16T18:52:56Z

👎 Rejected by too few up-to-date approved reviews (some of the PR reviews are stale)

Pacman99 · 2021-03-16T19:50:22Z

bors r+

fix #167 This worked for me to bootstrap another machine.

Pacman99 · 2021-03-16T19:53:50Z

Ohh right this is going to do a squash merge :(

blaggacao · 2021-03-16T19:56:13Z

Ohh right this is going to do a squash merge :(

Never mind, I think Tim is going to change that soon...

nrdxp · 2021-03-16T20:58:07Z

sorry for delay, I believe I figured out how to fix the ci runner while keeping everything up to date (import for users who want to run their own ci). Essentially we will use the stable nix daemon, and only rely on nixosUnstable for the client side. On my tests last night this seemed to fix things, but a lot of derivations were failing (I believe due to updates to nixpkgs). BORS won't commit with failing tests, so I'll see what I can do to fix them, or we may just have to disable some of the failing archtiectures for a while.`

blaggacao · 2021-03-16T21:03:45Z

@nrdxp If you exceptionally can merge this manually, we would save my carefully crafted git history for once. 😉

I built everything on my machine and tested end to end on target host.

Only caveat: deploy doesn't know how to sudo with password, but this needs improvment/ clarification in doc/integrations/deploy.md in a separate PR.

nrdxp · 2021-03-16T21:07:35Z

Well, give me about half an hour to implement the fixes in the CI runner, and if it still doesn't work, I'll just merge manually.

bors · 2021-03-16T21:14:08Z

Pull request successfully merged into core.

Build succeeded:

nrdxp · 2021-03-16T21:16:46Z

So it merged. There is the bors r- to cancel a merge before tests finish for future reference. I just removed squash merge manually, but it was too late.

Pacman99 · 2021-03-16T21:18:10Z

Yeah apologies, I will remember that for the future.

nrdxp · 2021-03-16T21:20:42Z

Not a big deal, it is inevitable with any workflow shift to have a few kinks to work out. I am glad that @blaggacao at least got credit in the commit message though.

blaggacao · 2021-03-16T21:27:19Z

I'm not after any credit whatsoever 😉

blaggacao force-pushed the da/add-devos-env-to-iso branch from 894dd40 to c164fe0 Compare March 15, 2021 04:19

blaggacao commented Mar 15, 2021

View reviewed changes

This comment has been minimized.

Sign in to view

blaggacao mentioned this pull request Mar 15, 2021

Pass 'self' to lib #169

Closed

This comment has been minimized.

Sign in to view

Pacman99 mentioned this pull request Mar 15, 2021

Extract custom system builds from devosSystem out of lib #170

Closed

nrdxp requested a review from Pacman99 March 15, 2021 19:46

This comment has been minimized.

Sign in to view

blaggacao commented Mar 16, 2021

View reviewed changes

Pacman99 reviewed Mar 16, 2021

View reviewed changes

lib/devos/devosSystem.nix Show resolved Hide resolved

Pacman99 approved these changes Mar 16, 2021

View reviewed changes

blaggacao mentioned this pull request Mar 16, 2021

add hosts module arg #164

Merged

blaggacao force-pushed the da/add-devos-env-to-iso branch 2 times, most recently from 6fd68bd to 5452058 Compare March 16, 2021 18:30

blaggacao force-pushed the da/add-devos-env-to-iso branch 2 times, most recently from 2b55d42 to 6e011bf Compare March 16, 2021 18:43

David Arnold added 3 commits March 16, 2021 13:49

lib/devos: bake devos repo into live cd

65f3fd7

Copy over the nix storepath contents of the top level flake in order to be able to boostrap a host from a live iso installer conforming to the devos documentation using it's devshell. closes divnix#167

blaggacao force-pushed the da/add-devos-env-to-iso branch from 6e011bf to a09c1a9 Compare March 16, 2021 18:50

blaggacao requested a review from Pacman99 March 16, 2021 18:54

Pacman99 approved these changes Mar 16, 2021

View reviewed changes

bors bot pushed a commit that referenced this pull request Mar 16, 2021

lib/devos: bake devos repo into live cd (#168)

3d324e7

fix #167 This worked for me to bootstrap another machine.

blaggacao mentioned this pull request Mar 16, 2021

pin inputs into iso live registry #190

Closed

blaggacao mentioned this pull request Mar 16, 2021

general: pass self to ./lib & ./hosts #189

Closed

bors bot changed the title ~~lib/devos: bake devos repo into live cd~~ [Merged by Bors] - lib/devos: bake devos repo into live cd Mar 16, 2021

bors bot closed this Mar 16, 2021

		The latter option is currently blocked by [Nix Issue 4643](https://github.com/NixOS/nix/issues/4643).

		TODO: The former did not complete on my machine as the MulticastDNS seemed to be too unstable.

[Merged by Bors] - lib/devos: bake devos repo into live cd #168

[Merged by Bors] - lib/devos: bake devos repo into live cd #168

Conversation

blaggacao commented Mar 15, 2021 • edited Loading

blaggacao left a comment

Choose a reason for hiding this comment

blaggacao Mar 15, 2021

Choose a reason for hiding this comment

This comment has been minimized.

This comment has been minimized.

nrdxp commented Mar 15, 2021 • edited Loading

blaggacao commented Mar 15, 2021

Pacman99 commented Mar 15, 2021

blaggacao commented Mar 15, 2021 • edited Loading

Pacman99 commented Mar 15, 2021

blaggacao commented Mar 15, 2021 • edited Loading

Pacman99 commented Mar 15, 2021

Pacman99 commented Mar 15, 2021

blaggacao commented Mar 15, 2021

nrdxp commented Mar 15, 2021

bors bot commented Mar 15, 2021

bors bot commented Mar 15, 2021

blaggacao commented Mar 15, 2021 • edited Loading

This comment has been minimized.

blaggacao Mar 16, 2021

Choose a reason for hiding this comment

blaggacao commented Mar 16, 2021

Pacman99 commented Mar 16, 2021

Pacman99 commented Mar 16, 2021

blaggacao commented Mar 16, 2021 • edited Loading

blaggacao commented Mar 16, 2021

bors bot commented Mar 16, 2021

Pacman99 commented Mar 16, 2021

Pacman99 commented Mar 16, 2021

blaggacao commented Mar 16, 2021

nrdxp commented Mar 16, 2021 • edited Loading

blaggacao commented Mar 16, 2021 • edited Loading

nrdxp commented Mar 16, 2021

bors bot commented Mar 16, 2021

nrdxp commented Mar 16, 2021

Pacman99 commented Mar 16, 2021

nrdxp commented Mar 16, 2021

blaggacao commented Mar 16, 2021

blaggacao commented Mar 15, 2021 •

edited

Loading

nrdxp commented Mar 15, 2021 •

edited

Loading

blaggacao commented Mar 15, 2021 •

edited

Loading

blaggacao commented Mar 15, 2021 •

edited

Loading

blaggacao commented Mar 15, 2021 •

edited

Loading

blaggacao commented Mar 16, 2021 •

edited

Loading

nrdxp commented Mar 16, 2021 •

edited

Loading

blaggacao commented Mar 16, 2021 •

edited

Loading