GPG User Authentication: In-memory gpg keygen + keytocard and GPG key material backup enabling (plus a lot of code cleanup and UX improvements)

[tlaurion]

Goal of this PR: Permit User Authentication prior of going to Recovery Shell or USB boot. But prior of enabling that functionality, we need to make sure that the user has a backup of his key material. And to have a backup of key material, we need to generate master key and subkeys in memory and backup over encrypted partition and then keytocard the subkeys to the USB Security dongle. This PR does that and clean the code that was reviewed along the way.

EDIT: this comment from @JonathonHall-Purism shows the new flow graphically: #1515 (comment)

Ok! from #1515 (comment):

• oem-factory-reset script can be asked to be launched early at boot with the 'o' key (just as 'r' key goes to recovery shell if permitted in boot policy)
• Purism OEM Factory Reset default workflow is now promoted by default, leaving advanced prompts to acknowledge not using defaults.
• A questionnaire is introduced when defaults are not accepted, guiding users into properly answer prompts or entering Enter key when answer is unknown.
• Questionnaire asks if the user wants to create an encrypted thumb drive backup of GPG key material, which when enabled currently enables GPG Authentication by default
• Next question in questionnaire is by default Y to copy keys to USB Security dongle. p256 is enabled automatically when a NK3 is connected.
• When enabled, GPG Authentication now authenticates prior of recovery shell access or USB boot.

Unrelated work under this PR:

• Streamlining of tpm owner password usage across codebase, where if invalid, is now cleaned by tpmr directly.
• When TPM Disk Unlock Key is enabled on either TPM1/TPM2, upgrading firmware past TPM TOTP/HOTP resealing reuses encrypted disks configuration and only prompts to reseal DUK with Disk Recovery Key passphrase and DUK passphrase, and then uses GPG authentication method to detach sign required changes and reboots.
• A lot of cleanup and wording consolidation has been done.

Discussions on how GPG Authentication and OEM Factory Reset/Re-Ownership features should evolve are happening under #1520 and #1521 respectively.

Please participate, your voice is important to be considered!

---

Old:
Creating WiP PR to track changes.

WARNING: At current stage, prior merged PR #1476 makes it possible (warning with Intel 3rd gen I3 laptops without RDRAND: crng is not ready early on boot, will need to update board configs for notice even though those CPUs are not widely used).

Raw notes:

• oem-factory-reset on extended provisioning mode (non-default) gives possibility of creating in-memory key generation for GPG key material (m option), creates a GPG backup thumb drive with selection of percentage of drive to be assigned to LUKS container's ext4 partition (private key container) and leftover of drive assigned to exfat partition (which contains public key)
• oem-factory-reset also does proper keytocard of subkeys to USB Security dongle (warning: NK3 supports RSA only under pre-release version >= v1.3.1-test.20230417 where upgrading NK3 firmware is not so straightforward (impossible under debian-12, was successful with workarounds under debian-11
  • TODO: support in-memory p256 GPG algo and revert PoC changes under usb_security_token_capabilities_check (My only testing key is NK3 and I wanted to support RSA first.... so stay tuned here for latest changes and squashing prior of testing this if you have NK3 and not testing pre-releases, this is not ready to use).
• When a GPG key material thumb drive backup is created, codebase activates Heads authentication on usb boot and recovery shell codepaths. It works in current state if you own a NitrokeyPro/Librem Key/Nitrokey Storage you are willing to wipe and dedicate for testing of this PR
• Consequently, this PoC PR permits in-memory key gen of RSA key (2048 bits for testing here both smartcard keygen/in-memory keygen + keytocard, which sets CONFIG_HAVE_GPG_KEY_BACKUP and saves it in config.user, injects it and then checks against in in codebase to automatically activate authentication on recovery shell and usb boot codepath

Creating PR since I came across changes needed and wanted to be able to discuss them with stakeholders prior of proposing final version. Will comment on those in next comments, and will come back to OP and modify it with current state as I progress up to proposing final version. Will set PR as draft after creating PR.

[tlaurion]

As you can see in this screenshot, some guidelines were added in warnings so that the user is made aware of signing errors that can happen under Heads if TPM reset is needed to re-create TPM counter which otherwise sealing operations fail. As of now, the user faces a loop when doing TPM Disk Unlock Key resealing and is now informed that he should do a TPM reset in such corner case.

[tlaurion]

As of now, after having re-owned/oem factory reset secrets accepting to create a GPG thumb drive backup, the user can use either the backup or the USB security dongle, interchangeably:

On recovery/usb boot, we now have maybe too verbose output but those access are authenticated with either USB Security dongle or the Thumb drive backup:

[tlaurion]

Some unrelated changes in vocabulary for consistency in current state of PR when resealing TPM Disk Unlock Key

[tlaurion]

Latest commit fixes output and forces dmesg output pushed in console to be of warning level and lower if no debugging mode set in board config/through configuration menu.

Accessing the recovery shell gives something like this in production environement:

[tlaurion]

Last commit removes unnecessary output. Just tracing here, will hide messages later on

[tlaurion]

With last commit b7c4f1a

NK3 is supported with ECC p256 in-memory keygen and keytocard (as well as non-officially supported RSA2048 subkeys).

It looks like this when asking recovery shell access or USB boot (non-debug qemu tpm2 whiptail):

Normal behavior (default) is to authenticate against USB Security dongle (NK3 here):

Otherwise GPG Thumb drive key material backup looks like this:

[tlaurion]

Next step is to reprovision USB Security dongle (keytocard) from GPG key material backup thumb drive.
All the TODO in code will be cleaned prior of asking for final review, but implementation skeleton should be in place now, so comments welcome @daringer @JonathonHall-Purism

[tlaurion]

As current state:

• code can generate both in memory+ keytocard or under USB Security dongle's smartcard
• tested with tpm1/tpm2
• with NK3 to be consistent on shipped state (p256 with current dongle detection code based on lsbuid
• works with RSA 2048 (if detection code hacked out) and p256 inmemory and on smartcard only

A lot of code cleanup has been made along the way.

@JonathonHall-Purism Please check left TODO in code. The check for TPM2 primary handle has been removed from now since if causes a lot of issues linked to default boot and it not being signed. That would need to be fixed on master separately and would need some advice/coop.

• oem-factory-reset has been modified to warn qemu users that internal flashing is not supported. We could extend that, since setting config options in board config is needed alongside injecting public key manually at build time with proper make options as written in md file alongside config file. Suggestions welcome
• oem-factory-reset now includes a questionnaire warning when default oem-factory-reset is not enforced (only librems as of now, qemu enforces it).

Here is a quick preview of what to expect:
TPM2 warning to TPM reset when primary handle is not existing:

Questionnaire changes:

[JonathonHall-Purism]

Thanks @tlaurion. I think the ideas here and general structure are good. I haven't tested any of it yet, but I did review it thoroughly keeping in mind it's still WIP.

Specific comments above, but as some key highlights:

• I suggest moving the "boot authentication" (gpg_auth) to a separate PR so we can more easily discuss the goals and behavior of that feature (but I think it is a great idea and on a good path).
• I suggested wording improvements for the new prompts in OEM reset but I think the strategy makes sense
• I'll look into the issues with the TPM2 primary handle hash as there are some clear gaps there as we've discussed before

I looked over the new GPG functionality and don't have any problems with it, with that said I'm not really a GPG expert at this point so testing will be pretty important there IMO.

[tlaurion]

@JonathonHall-Purism

*** WARNING: GPG key material will be generated on USB Security Dongle's smartcard wtihout backup *** I don't think this should be a warning. This is normal for a lot of users that just use the GPG key on the dongle with Heads, I would tell them to ignore this warning. If we get users accustomed to ignoring warnings, the warnings lose their meaning.

Fixed under 160367d

OEM reset failed when attempting to use a single password for everything and with USB dongle only (no flash drive). Said Error signing kexec boot files: gpg: signing failed: Bad PIN (qemu-coreboot-fbwhiptail-tpm1-hotp)

Saying N to encrypted USB thumb drive should say GPG User PIN is getting set GPG_GEN_KEY_IN_MEMORY=0 doesn't align with -z (empty) tests later. Suggest sticking with the "y or not-y" convention for booleans we use elsewhere for the new booleans in this file

Fixed under 3787293

[JonathonHall-Purism]

The refactors all look great, confirmed everything reported above was fixed. Testing the new functionality now on L1UM v2 (TPM2).

@tlaurion I didn't unplug my flash drive when it asked me to remove all external USB devices (to see what happens). It still selected that flash drive automatically when I continued. Isn't this supposed to exclude USB devices that remained connected? I figured this was to exclude internal USB devices, like the Librem 14's SD card reader.

edit: Then when I looked back after posting this, it had errored out and sent me back to the main menu (no error prompt). This was on the console:

750 MB is right, it's an "8GB" flash drive and I selected 10%. So it had the right device, I'll pull it and see what it put on there.

[tlaurion]

@tlaurion I didn't unplug my flash drive when it asked me to remove all external USB devices (to see what happens). It still selected that flash drive automatically when I continued. Isn't this supposed to exclude USB devices that remained connected? I figured this was to exclude internal USB devices, like the Librem 14's SD card reader.

@JonathonHall-Purism I'll rework the removal/insert prompt to list all usb connected block devices and list capacity for user to select disk prior of confirming once after selection for wiping.

edit: Then when I looked back after posting this, it had errored out and sent me back to the main menu (no error prompt). This was on the console:

newer librems based on config/linux-librem_common-6.1.8.config didn't have EXFAT support in the kernel. That was fixed under 23c967f

(Note that CircleCI is having temporarily difficulties fetching tarballs on coreboot build as of now, and also flash boards seem to have race condition (what? Why!) so I might disable the builds in CircleCI here if they fail on clean cache. Nitrokey deprecated them and I will do the same, moving them to boards/UNSUPPORTED_ boardname if they cause problem on a rerun. No time for that anymore. Was told before, might be the time for it now.)

[JonathonHall-Purism]

Thanks @tlaurion. Kconfig changes look good, saved as defconfig locally to compare, we'll stick to oldconfig going forward. Let me know when USB device selection is ready for review again 👍

[tlaurion]

Let me know when USB device selection is ready for review again

@JonathonHall-Purism that would be that moment :)

[JonathonHall-Purism]

@tlaurion I tested this a lot and made some changes to smooth out the OEM reset flow with flash drive format:

https://github.com/JonathonHall-Purism/heads/commits/oem_reset_fd_sel_improvement

Changes:

commit a925219efb526188780ecba8d3fc8c8961405816
Author: Jonathon Hall <jonathon.hall@puri.sm>
Date:   Mon Nov 13 13:54:37 2023 -0500

    oem-factory-reset: Improve prompt flow formatting flash drive
    
    Combine prompt to disconnect other devices with prompt to connect the
    desired device.
    
    Show block device sizes in MB/GB when selecting device so it is easier
    to select.  file_selector now supports --show-size to include block
    device sizes in menu.
    
    Rework file_selector so menu options can contain spaces (use bash
    array) and to simplify logic.
    
    Prompt to select flash drive and LUKS percentage in OEM reset before
    actually taking any actions, so aborting doesn't half-reset the system.
    
    Abort OEM reset if user aborts the flash drive selection instead of
    looping forever.  (Canceling the confirmation still loops to retry but
    it is possible to exit by aborting the repeated menu.)
    
    Signed-off-by: Jonathon Hall <jonathon.hall@puri.sm>

commit d39fc26dd95ae4b7a82376ede49e9a2612cafa36
Author: Jonathon Hall <jonathon.hall@puri.sm>
Date:   Mon Nov 13 14:37:19 2023 -0500

    oem-factory-reset: Move format confirmation before resetting anything
    
    Move confirmation of formatting flash drive with LUKS percentage
    selection before any reset actions have been taken, so aborting does
    not result in a half-reset system.  Combine with the more basic
    "confirm" prompt that existed after selecting the device (but did not
    include the LUKS size information).
    
    Split up prepare_flash_drive into interactive_prepare_flash_drive (both
    prompts and formats as before), confirm_thumb_drive_format (just
    confirms the selections), and prepare_thumb_drive (now noninteractive).
    
    Signed-off-by: Jonathon Hall <jonathon.hall@puri.sm>

commit 97d903f22a7ab1b4abb3eca09dbf7db165cb065b (HEAD -> oem_reset_fd_sel_improvement, github-jh/oem_reset_fd_sel_improvement)
Author: Jonathon Hall <jonathon.hall@puri.sm>
Date:   Mon Nov 13 14:52:09 2023 -0500

    oem-factory-reset: Don't repeat "insert flash drive" message
    
    Don't repeat this message if the user says "no" to the confirmation
    prompt.  Go directly to the menu.
    
    Signed-off-by: Jonathon Hall <jonathon.hall@puri.sm>

This is the resulting sequence. All prompts occur before OEM reset starts to reset anything, so aborting will not leave a half-reset system. You can now abort by aborting the flash drive selection (no longer loops in this case). Deduplicated some prompts. Put some of the better-size-display enhancements in the more specific confirmation prompt and used that instead of the more basic one.

[tlaurion]

Wow @JonathonHall-Purism . Merging disk sizes with prompts is a really good flow improvement, and your changes to parametrize functions makes it easier to understand, follow and improve further as well. Wow!

I think it's a merge day! Doing one last whole test case on real hardware and as discussed I will merge (reviewing your changes on top of the changes you reviewed and applied changes you saw fit better.)

I really love the graphics and your work on this! Awesome collaboration! I'm in awwwwwwwe! :)

[JonathonHall-Purism]

Thanks @tlaurion ! I think this is a great result. Merge when ready! 💯