Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Manual: reviews partitioning steps #47697

Merged
merged 4 commits into from
Oct 4, 2018

Conversation

samueldr
Copy link
Member

@samueldr samueldr commented Oct 3, 2018

I want to read

The contents under 2.2 is the main star; that's what changed.

There were some fixups in 2.1 to make it more truthful to our modern times, but 2.3 wasn't touched a bit. The summaries under 2.4 are updated to match.

FAQ

With UEFI partitioning, why is the ESP created last?

This is so that both the swap and root partition share the same index (1 and 2) for the following installation instructions. Otherwise, it will become messy and full of "if you're using an ESP, use /dev/sdX1, otherwise use /dev/sdX2". This, also, was how it was previously handled: the ordering was manually fudged using fdisk to make the ESP /dev/sdX3.



Main feature

doc: Reviews partitioning instructions to use parted.

The tests in <nixos/tests/installer.nix> are using parted, so they are
bound to be better tested than fdisk.

This is brought on by a couple issues, plus reports on IRC that the
fdisk instructions didn't work as expected.

Care was taken so that the other documented steps did not need changes.

In all this kerfufle, a slight re-organization of the Chapter has been
made, allowing better deep linking.

Additional changes

doc: installing-usb: removes notes about unetbootin.

They are known to cause more issues than solving issues; futhermore
using dd should work everywhere without fail.

doc: installing-usb make macOS note a note.

While it seemingly brings more attention to the macOS notes with the
default docbook template, it better represents which parts of the
section are about macOS, and which parts are simply in the flow of the
text; otherwise the last paragraph may be lost into the details for
macOS.

Things done
  • ✔️ Built the manual nix-build nixos/release.nix -A manual.x86_64-linux.
  • ✔️ make format.
  • ✔️ Tested the partitioning instructions to give the expected results in the expected order for the following steps.

Everyone listed: please comb and find all nits to pick, please and thank you!

cc @vcunat @edolstra changes a bunch of text on a tight schedule.

cc @grahamc as it touches documentation

cc @7c6f434c @matthewbauer @timokau @ryantm who looked at the blocking issue and left comments or thumb ups.

Fixes #47602

@samueldr samueldr added the 1.severity: blocker This is preventing another PR or issue from being completed label Oct 3, 2018
@samueldr samueldr added this to the 18.09 milestone Oct 3, 2018
@GrahamcOfBorg GrahamcOfBorg added 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: documentation labels Oct 3, 2018
@samueldr
Copy link
Member Author

samueldr commented Oct 3, 2018

Here's some details about the better deep-linking mentioned:

The current manual has this weird wart (imho) where the table of contents seems to show sections of the installation guide, but they instead link to additional notes about the installation process. I myself remember having trouble finding the right parts when diving in initially.

image

The new structure actually shows the main "movements" in the installation as discrete sections, with a new section describing additional notes. Furthermore, it will help helpers help helpees (!!) by allowing them to link a specific subsection of the installation section.

image

@GrahamcOfBorg GrahamcOfBorg added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux labels Oct 3, 2018
@worldofpeace
Copy link
Contributor

it will help helpers help helpees (!!)

I appreciate the neutral pronouns 😄

@vcunat
Copy link
Member

vcunat commented Oct 3, 2018

What I like about unetbootin: if you have a large flash, you can have it as both bootable and a storage for random files. (With dd the drive gets read-only.) ... but yes, I also noticed unetbootin is less reliable and can't start memcheck.

@samueldr
Copy link
Member Author

samueldr commented Oct 3, 2018

@vcunat I could revert that change as it is a bit out of scope, but I would prefer a well-tested rewrite, rewriting it in a way that makes it obvious it's an alternative step to dd. The last I know about unetbootin is that every time a user has trouble booting the usb installer, the first question is "was this made using unetbootin?" followed by "don't use unetbootin".

@vcunat
Copy link
Member

vcunat commented Oct 3, 2018

It might better be undocumented in the official manual, as it's not reliable.

I wonder why make format did so many formatting changes to the parts that you didn't touch, but that seems not important at all...

@samueldr
Copy link
Member Author

samueldr commented Oct 3, 2018

I did touch those part: re-doing the sections of the chapter added wrapper tags, which bumped the indent a tiny amount, requiring all those changes.

Copy link
Member

@timokau timokau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bit of a brain-dump, some of the remarks are probably out of scope here.

If there is no policy against it I would also recommend to link to the relevant arch wiki page which is probably more complete than we can ever hope to be here. Having simple examples in the manual is important, but the option for further reading would be good too.

@@ -9,13 +9,12 @@
For systems without CD drive, the NixOS live CD can be booted from a USB
stick. You can use the <command>dd</command> utility to write the image:
<command>dd if=<replaceable>path-to-image</replaceable>
of=<replaceable>/dev/sdb</replaceable></command>. Be careful about specifying
of=<replaceable>/dev/sdX</replaceable></command>. Be careful about specifying
the correct drive; you can use the <command>lsblk</command> command to get a
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Depending on how beginner-friendly we want to be we may want to be more specific: "Be careful about specifying the correct drive instead of sdX".

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At least sdX doesn't normally exist.

<listitem>
<para>
Add a <emphasis>swap</emphasis> partition. The size required will vary
according to needs, here a 8GiB one is created.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a new user I would wonder "according to what needs? What swap size do I need?" (In fact I still kind of wonder that and just do something in between 1x and 2x my RAM).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also why no swap file?

Copy link
Member Author

@samueldr samueldr Oct 3, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good comments: I added text around the existing defaults to 8GiB and this is something I had noted in my actual refactor of the installation instructions; right now those instructions are the same setup as previously done, except it's parted instead of fdisk, and adds a bit more context around the instructions.

So the fact that these are the previous instructions is the reason it's using this 8GiB swap partition without questioning its use, and without checking into using a swap file instead.

nixos/doc/manual/installation/installing.xml Outdated Show resolved Hide resolved
nixos/doc/manual/installation/installing.xml Show resolved Hide resolved
nixos/doc/manual/installation/installing.xml Show resolved Hide resolved
nixos/doc/manual/installation/installing.xml Show resolved Hide resolved
@7c6f434c
Copy link
Member

7c6f434c commented Oct 3, 2018

Do we want to say something about interactive use of parted?

@7c6f434c
Copy link
Member

7c6f434c commented Oct 3, 2018

Out of scope but related: re: swap: there is hibernation to swap (swap file is complicates for that); memory leak handling via swap is an expert topic; do we assume the Nix version in use has O(n)-space nix copy?

@samueldr
Copy link
Member Author

samueldr commented Oct 3, 2018

If anyone was inconvenienced by the tag soup, I have taken the time to copy the built manual to my server:

@samueldr
Copy link
Member Author

samueldr commented Oct 3, 2018

Do we want to say something about interactive use of parted?

I can't as I don't know parted much; I looked at the manual while verifying the steps, and everything I learned was learned through the installer tests.

Though, if there are good things to say about interactive parted, I'm all ears, I already intended to do a more in-depth installation section rewrite in the future.

Out of scope but related: re: swap: there is hibernation to swap (swap file is complicates for that); memory leak handling via swap is an expert topic; do we assume the Nix version in use has O(n)-space nix copy?

I'm not sure what "O(n)-space nix copy" would mean here.

As said, and I think you already know and understand, the size and presence of a discrete swap partition is totally arbitrary and is from the existing installation guide. I had in mind (in that future rewrite) to add a separate annex with a FAQ for different common system setups, and for common frequently questioned answers ;). In that annex there would be both "Do I need swap?" and "Should I prefer a swap file?" questions (among others).


<para>
The NixOS installer ships with different tools and interfaces to partition
your drive. They include <command>fdisk</command>, <command>gdisk</command>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest naming the default first, then enumerating alternatives:

The NixOS installer ships with multiple partitioning tools. The examples below use <command>parted</command>, but we also provide foo, bar, baz and quux.

your drive. They include <command>fdisk</command>, <command>gdisk</command>,
<command>cfdisk</command>, <command>cgdisk</command> and
<command>parted</command>. While the example partitioning schemes use
parted, any other tool can achieve the same results.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

<command>?

<para>
Add a <emphasis>swap</emphasis> partition. The size required will vary
according to needs, here a 8GiB one is created.
<screen language="commands"># parted /dev/sda -- mkpart primary linux-swap 1M 8GiB</screen>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are the <screen> tags indented correctly?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this since their white-space matters, they are cattywampus across the manual. Though the initial tag could be left indented, make format changes that.

<listitem>
<para>
Add a <emphasis>swap</emphasis> partition. The size required will vary
according to needs, here a 8GiB one is created. The space left in front
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Vary how? Suggest providing a rule of thumb, or a link to a wiki somewhere if there are no good NixOS-specific heuristics.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A simple way to expand this further would be to drop the second sentence and say something like "You will need at least as much swap space as RAM if you want to hibernate."

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't hibernation somewhat sketchy if you just have the same amount of swap as you have RAM?

</term>
<listitem>
<para>
You <emphasis>must</emphasis> set the option
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possible reword; we want this option to be set, but it may not need user action?

<para>
The system <emphasis>will not boot</emphasis> if <xref linkend="opt-boot.loader.systemd-boot.enable" /> is not <literal>true</literal>. If you booted in UEFI mode, <command>nixos-generate-config</command> should have done this for you, but you must check this now.
</para>

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of scope, installation instructions were left as-is, make format re-indented the lot.

Copy link
Member

@grahamc grahamc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mbr:

[nix-shell:~]$ du -h ./rootdisk
16G	./rootdisk

[nix-shell:~]$ parted ./rootdisk -- mklabel msdos
WARNING: You are not superuser.  Watch out for permissions.

[nix-shell:~]$ parted ./rootdisk -- mkpart primary linux-swap 1M 8GiB
WARNING: You are not superuser.  Watch out for permissions.

[nix-shell:~]$ parted ./rootdisk -- mkpart primary 8GiB -1s
WARNING: You are not superuser.  Watch out for permissions.

[nix-shell:~]$ fdisk ./rootdisk 

Welcome to fdisk (util-linux 2.32.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): p
Disk ./rootdisk: 15.6 GiB, 16777216000 bytes, 32768000 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xd8b64975

Device      Boot    Start      End  Sectors  Size Id Type
./rootdisk1          2048 16777215 16775168    8G 82 Linux swap / Solaris
./rootdisk2      16777216 32767999 15990784  7.6G 83 Linux

gpt

[nix-shell:~]$ du -h ./rootdisk
16G	./rootdisk

[nix-shell:~]$ parted ./rootdisk -- mklabel gpt
WARNING: You are not superuser.  Watch out for permissions.
Warning: The existing disk label on /home/grahamc/rootdisk will be destroyed and all data on this disk will be lost. Do you want to continue?
Yes/No? yes                                                               

[nix-shell:~]$ parted ./rootdisk -- mkpart primary linux-swap 512MiB 8.5GiB
WARNING: You are not superuser.  Watch out for permissions.

[nix-shell:~]$ parted ./rootdisk -- mkpart primary 8.5GiB -1MiB
WARNING: You are not superuser.  Watch out for permissions.

[nix-shell:~]$ parted ./rootdisk -- mkpart ESP fat32 1M 512MiB
WARNING: You are not superuser.  Watch out for permissions.

[nix-shell:~]$ parted ./rootdisk  -- set 3 boot on
WARNING: You are not superuser.  Watch out for permissions.

[nix-shell:~]$ fdisk ./rootdisk 

Welcome to fdisk (util-linux 2.32.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): p
Disk ./rootdisk: 15.6 GiB, 16777216000 bytes, 32768000 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 3C1902C9-0C36-46F2-A948-DFEFD173ED86

Device         Start      End  Sectors  Size Type
./rootdisk1  1048576 17825791 16777216    8G Linux swap
./rootdisk2 17825792 32765951 14940160  7.1G Linux filesystem
./rootdisk3     2048  1048575  1046528  511M EFI System

Partition table entries are not in disk order.

^ this bit about being not in order seems like another reason to just go ahead and make /boot first.

No blocking comments. Way to go, Samuel, this is a huge improvement. +1

<title>Partitioning and formatting</title>

<para>
The NixOS installer doesn’t do any partitioning or formatting yet, so you
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

delete yet

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

Though yes, I agree.

</para>

<para>
The NixOS installer ships with different tools and interfaces to partition
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merge the first two sentences:

The installation environment comes with fdisk, gdisk, cfdisk, cgdisk and parted for partitioning your drive.

(edit later: endgame has good feedback here!)

<para>
The NixOS installer doesn’t do any partitioning or formatting yet, so you
need to do that yourself. Use the following commands:
<itemizedlist>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An orderedlist here will cause the steps to be enumerated as incremental steps.

</para>
</section>

<section xml:id="sec-installation-partitioning-UEFI">
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should put UEFI first?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I alphabetized the listing! (Not really!)

Yeah, it might as well be first, since it's (now) the most common type.

<para>
Finally, the <emphasis>boot</emphasis> partition. NixOS by default uses
the ESP (EFI system partition) as its <emphasis>/boot</emphasis>
partition. It uses the intiially reserverd 512MiB at the start of the
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

intiially

-> initially

reservd -> reserved

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we create the boot partition last, instead of first? Making it last causes the first step to be a bit awkward -- having to explain the gap.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's created last so it gets /dev/sda3 so the instructions match up later, sda1 as swap and sda2 as the root filesystem for both partitioning scheme. Otherwise, we need to go all meta and never refer to the partitions again (and rewrite what I didn't want to touch).

While it seemingly brings more attention to the macOS notes with the
default docbook template, it better represents which parts of the
section are about macOS, and which parts are simply in the flow of the
text; otherwise the last paragraph may be lost into the details for
macOS.
They are known to cause more issues than solving issues; futhermore
using `dd` should work everywhere without fail.
The tests in <nixos/tests/installer.nix> are using `parted`, so they are
bound to be better tested than `fdisk`.

This is brought on by a couple issues, plus reports on IRC that the
`fdisk` instructions didn't work as expected.

 * NixOS#39354
 * NixOS#46309
 * NixOS#39942
 * NixOS#45478

Care was taken so that the other documented steps did not need changes.

In all this kerfufle, a slight re-organization of the Chapter has been
made, allowing better deep linking.
@samueldr samueldr force-pushed the fix/47602-new-partitioning-steps branch from fbdaf21 to 6cfbf40 Compare October 4, 2018 02:35
Copy link
Member

@grahamc grahamc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

@grahamc grahamc merged commit a8878f4 into NixOS:master Oct 4, 2018
@grahamc
Copy link
Member

grahamc commented Oct 4, 2018

Backported in:

91c3bc1 doc: installing-usb: removes notes about unetbootin.
9da278d doc: Reviews partitioning instructions to use parted.
292d4fc nixos/doc: Adds sub-folder to input files.
59f5dea doc: installing-usb make macOS note a note.

<command>cfdisk</command>, <command>cgdisk</command> and
<command>parted</command>. While the example partitioning schemes use
parted, any other tool can achieve the same results.
</para>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to say that nix-env lets one install into RAM even more various partitioning tools? Or is it said enough times by that point?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noted for the eventual installation guide refactor. It is a good point.

@7c6f434c
Copy link
Member

7c6f434c commented Oct 4, 2018

I'm not sure what "O(n)-space nix copy" would mean here.

I mean there is a range of Nix versions that wants to download and hash in memory — these versions benefit a lot from having swap.

interactive use of parted

More or less similar to fdisk, but uses long-form commands.

@samueldr samueldr deleted the fix/47602-new-partitioning-steps branch February 12, 2019 02:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
1.severity: blocker This is preventing another PR or issue from being completed 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: documentation 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Tracking issue for partitioning instructions issues
8 participants