calamares: fix calamares modules, add nixos modules, and add new iso

[vlinkz]

Motivation for this change

Currently, installing NixOS is rather difficult for new users, and the new experience when booting into one of the graphical ISOs is a blank GNOME or Plasma desktop with no super obvious indication of what to do next. Even though following the guide is not all that hard, I believe that integrating Calamares into the current NixOS isos would help make the installation experience better for new users.

Closes #15573
Closes #21662
Closes #100475

Things done

This draft PR contains 3 things:

1. Updates the calamares package to fix the functionality of multiple modules.
2. Adds a package for calamares-nixos-extensions, which implements NixOS backend for some calamares modules
3. Adds configuration for a GNOME and Plasma5 ISOs with calamares installer

The GNOME test ISO can be downloaded here nixos-gnome-22.05pre375061.c777cdf5c56-x86_64-linux.iso
The Plasma5 test ISO is also available here: nixos-plasma5-22.05pre375061.c777cdf5c56-x86_64-linux.iso.

All feedback is appreciated!

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 22.05 Release Notes (or backporting 21.11 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
  • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
• Fits CONTRIBUTING.md.

Current Issues

• No way to enable unfree drivers
  • Laptops with Broadcom wifi cards will not have it enabled after installation

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/considering-new-installation-approaches/17778/16

[bobvanderlinden]

I simplified the calamares-nixos-extensions a bit: bobvanderlinden@c34a688

I also worked on making sure I could run calamares inside a VM more easily without rebuilding the ISO (it was slow on my laptop): https://github.com/bobvanderlinden/nixpkgs/tree/calamares-nixos

[vlinkz]

Merged your changes. Looks great!

[davidak]

This are things i noticed when testing:

• When it's booted, i see the window overview. No idea how that happened. I should only see the Installer window.
• installing with german locales work
• Check if there is a locale present in Calamares that is not in NixOS and lead to failure

"Welcome to Calamares" is not user friendly. A user does not have to know which software we use for the Installer.

• Suggestion: "Welcome to the NixOS Installation."
• The window should also be called: "NixOS Installation"

• The install actually uses 6.6G! Also have some reserve for installing apps. Probably multiply by 2 to allow updates.

We still don't have official system requirements :(

• LUKS setup works
• but i have to type the password 2 times. that's annoying for the user and will discourage them from using a secure setup

that's different from NixOS install following manual. usually i'm asked only in stage 1, not grub

• manual partitioning with one XFS partition works

• manual partitioning with one btrfs partition works

Instructions for manual partitioning should be added to the manual. At least before stable release!

• Install on GPT failed (BIOS)

Not sure if that dialog is true or default Calamares. In any case, the option name has a typo!

• use actual option bios-grub (upstream issue Typo in dialog (bios-grub) calamares/calamares#1894)

A more useful and not so scary error dialog would be great!

Error:

=== START CALAMARES 3.2.44.3
2022-03-01 - 21:42:17 [1]: OsproberEntryList PartUtils::runOsprober(DeviceModel*)
    ERROR: os-prober cannot start. 
2022-03-01 - 21:59:28 [1]: void Calamares::ViewManager::onInstallationFailed(const QString&, const QString&)
    ERROR: Installation failed: "Boost.Python error in job \"nixos-customize-config\"." 

• Install on btrfs + swap failed

It should swapon manually created swap partitions even for the install. nixos-generate-config should do the rest...

Error:

=== START CALAMARES 3.2.44.3
2022-03-01 - 21:42:17 [1]: OsproberEntryList PartUtils::runOsprober(DeviceModel*)
    ERROR: os-prober cannot start. 
2022-03-01 - 21:59:28 [1]: void Calamares::ViewManager::onInstallationFailed(const QString&, const QString&)
    ERROR: Installation failed: "Boost.Python error in job \"nixos-customize-config\"." 

• Install on UEFI system with GPT works

• Autologin works

The showcases switch too fast. I learned from conference streaming that i as a video mixer have to read the text slowly, twice and then switch the scene.

• So please make the switch so slow that everyone can actually read it!

Better showcases for desktop users can be added in a later PR. Should be done before shipping stable release!

• Can you add a button to see what's going on, for e.g. in case of an error. Even elementary OS has that!

• The end screen should include instruction how a user should proceed. 1. read manual if anything is unclear 2. edit configuration.nix to change the system. Use different desktop etc.

We can add selection of a desktop in a new PR.

Thanks for your work. This is a huge milestone for NixOS! ✨

[bobvanderlinden]

Can you add a button to see what's going on, for e.g. in case of an error. Even elementary OS has that!

I was hoping this would be easy, but apparently elementary OS isn't based on Calamares. Their installer seems custom: https://github.com/elementary/installer. I haven't found an option to see the live log in Calamares. I've submitted a feature request: calamares/calamares#1896

[vlinkz]

I'm working on the errors with luks and swap. Figured out what's needed in the configuration, now in the process of automating it in the modules. Will push changes soon

[vlinkz]

• but i have to type the password 2 times. that's annoying for the user and will discourage them from using a secure setup

Install on btrfs + swap failed

• Install on GPT failed (BIOS)

Should all be fixed in the new version of calamares-nixos-extensions. Currently uploading new test image.

[vlinkz]

• When it's booted, i see the window overview. No idea how that happened. I should only see the Installer window.

I believe that this is intended behavior in GNOME 40 and above. Unsure if there is a way to disable it for the intaller

[davidak]

I believe that this is intended behavior in GNOME 40 and above.

That was already fixed in the image yesterday. It looks like this after boot:

I thought we might want to maximize the window also, but it would look bad on bigger screens.

I check the boxes when i have retested the reported issues and they are fixed.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/what-would-you-do-if-you-were-the-bdfl-of-nix/6949/17

[vlinkz]

Ok, updated licenses and added networking on "No Desktop" options. Building new ISOs, but everything seems good to go!

[davidak]

Result of nixpkgs-review pr 161788 run on x86_64-linux 1

2 packages marked as broken and skipped:

• libsForQt512.kpmcore
• libsForQt514.kpmcore

2 packages blacklisted:

• nixos-install-tools
• tests.nixos-functions.nixos-test

5 packages built:

• calamares
• calamares-nixos
• calamares-nixos-extensions
• libsForQt5.kpmcore (libsForQt515.kpmcore ,plasma5Packages.kpmcore)
• partition-manager

[vlinkz]

ISOs uploaded!

[davidak]

What happened (to the Calamares 3.2.56 update)?

[vlinkz]

Think something went wrong with a push, lemme check

[vlinkz]

Ok, fixed

[davidak]

Changes look good. Checked also the used repository. Only the patches are too much for me to review, but it does work.

Tested install

• with Plasma ISO on UEFI hardware, LUKS, Pantheon, autologin
• with GNOME ISO on BIOS VM, LUKS, MATE

i have not found any bigger issue

[davidak]

We can now wait for the channel to advance to include the changes and announce it then, so more people can test it. https://nixpk.gs/pr-tracker.html?pr=161788

If you read this, feel free to test using the last ISOs linked in the description.

[vlinkz]

Looks great, thanks for your help! I'll continue testing as we approach 22.05 release!

[Raagh]

Thanks for all the hard work folks, hopefully, this will bring nixos to more people.

[jonringer]

Very nice, :)

Proud of all the great work, will definitely diminish some of the overhead of adopting NixOS :)

[davidak]

It is now in the unstable channel and i was able to boot the gnome image.

I announced this achievement on my social media: https://chaos.social/@davidak/108253717600383941

@dasJ i suggest to use my image for the release announcement post. I gave it a very permissive license, so media can use it too.

[dasJ]

I think this is relevant for @Mic92

[06kellyjac]

Just trying out the installer (which is great) but I noticed I don't get any logs within "toggle logs" because os-prober cannot start

IDK if it being a VM caused any of the issues here

I am using https://hydra.nixos.org/build/175427824 so maybe I should grab a newer copy from hydra

[davidak]

@06kellyjac that is the expected output right now (calamares/calamares#1921)