Implement automatic unlocking though a systemd service

[josephlr]

Right now, both automatic unlocking of directories and rewrapping of the user's login protector are done though the PAM module pam_fscrypt.so. Specifically the following PAM types do the following things:

• session
  • tracks of the number of open sessions for that user
  • automatically unlocks user encrypted directories
  • automatically locks user encrypted directories (on the last logout)
• auth forwards the AUTHTOK object as part of the PAM data (if necessary)
• password rewarps the login protector (if necessary)

The basic plan is to keep the auth and password types in the pam modules (with modifications), while doing all the stuff we do in session in fscrypt@.service.

@ebiggers, @tyhicks: I'd love your feedback on the basic idea (and if it would improve things).
@fancytenseletters, @Minecrell, @sebadoom: This should (hopefully) fix some of the bugs you've reported.

Justification

Bugs #66, #77, #93, #196 (#57 and #34 are also related) seem to be bugs around what order certain operations run in. Ideally, we want all of our setup to happen before any user task starts. Similarly, all our teardown should occur after all user tasks complete. To do this well, we need to integrate with the OS's init service. This is because the init service:

• knows when the first session of a user begins
• knows when the last session ends
• allows us to run fscrypt commands at specific points to prevent things from breaking

eCryptfs has been running into issues with systemd and automatic mounting via PAM modules. See: Arch Bug, Debian Bug, Ubuntu Bug, systemd Bug. pam_mount.so, which can do similar stuff for device encryption, has also had issues.

Details

This change would require changing or adding three things.

• Remove functionality from pam_fscrypt.so

  • Remove session hooks, keyring linking code, caches code, etc..
  • For auth, if the user has a login protector, use the passphrase to unlock the login protector, and store it in the root user keyring (with type user). We could also prompt the user here to enter their old login password if things have gotten out of sync.
  • For password, if we find a login protector, use the passphrase hashes of AUTHTOK and OLDAUTHTOK to update the login protector to update the login protector

• Add new commands to fscrypt that must be run as root:

  • fscrypt system start <uid> --unlock=[none|login] [--link-to-root]
    • --unlock adds policies to the user's keyring
      • login gets the login protector key from the root user keyring and uses it to unlock the corresponding policies
    • --link-root-keyring makes sure the user keyring is linked into the root user keyring
  • fscrypt system stop <uid> --lock=[none|login|all] [--drop-caches] [--unlink-from-root]
    • --lock removes policies from the user's keyring:
      • login removes just those protected by the login passphrase
      • all removes all policies in the user's keyring
    • --drop-caches calls DropFilesystemCache after all keys are removed
    • --unlink-from-root removes the user keyring from the root user keyring

• Add a systemd service to trigger the appropriate commands. For example:

[Unit]
Description=Unlock fscrypt directories with login credentials
Before=user@%i.service

[Service]
Type=oneshot
RemainAfterExit=yes
Slice=user-%i.slice
RemainAfterExit=yes
StandardOutput=jouranal
StandardError=syslog
SyslogIdentifier=fscrypt
ExecStart=/usr/bin/env fscrypt system start %i --unlock=login --link-to-root
ExecStop=/usr/bin/env fscrypt system stop %i --lock=all --drop-caches --unlink-from-root

[Install]
WantedBy=user@%i.service

Benefits

• Way less PAM code to maintain
  • No more tracking the number of open user sessions
  • No locking/unlocking of directories
  • No keyrings or caching related code
  • Could replace how we currently check the user password eliminating our custom conversation function
  • Keeps the Pluggable Authentication Module focused on authentication related stuff.
• More stability
  • Should load before a lot of login jobs, preventing bugs with SDDM/GDM
  • Better setup to handle user session failures (right now we just never trigger cleanup)
  • Will eliminate issues with user processes keeping files open (mentioned here).
• Allows for more complex configuration than the PAM config files
• Allows behavior in [question] user keyring still linked to root keyring after fscrypt purge command #60 to be easily configurable
• No ties to systemd in particular. Any init service can call fscrypt system [start|stop] <uid>.

[ghost]

It should be password pam module type, not passwd.

[josephlr]

@fancytenseletters Fixed, Thanks!

[lathiat]

I'm not sure if this is the right place to put this, but I wanted to comment here because it may be relevant to whether this is the correct fix path.

I've discovered the main pain point for getting libpam-fscrypt to work on Ubuntu currently is that /etc/pam.d/systemd-user does not call into pam_keyinit.so - so the session keyring is not linked to the user keyring, and that causes libpam-fscrypt to appear to fail.

Something non-obvious about this, is that many desktop session processes are started under 'systemd-user' instead of the actual 'desktop session' - this includes gnome-terminal-server which means any gnome-terminal shell runs under this context. If you start xterm instead of gnome-terminal, you get a process under the session slice and a different keyring and this can cause confusion when debugging the issue as some processes are in one state and the others are in another including your primary debug tool gnome-terminal. You can verify this by running 'systemctl status $(pidof gnome-terminal)' and 'systemctl status $(pidof xterm)' and note the different hierachy.

In this example case within gnome-terminal you have no access to your homedir but within xterm you do.

The change to add pam_keyinit.so was made upstream in December 2016:
systemd/systemd@ab79099

However it is not currently reflected in Debian Stable or Ubuntu 16.04 through 18.04. You can find related bugs here and I also put a lot more detail into the launchpad bug below:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1754270
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=846377

With some feedback I'd be happy to open this as a new issue or similar as I guess at a minimum documentation probably could be updated to reflect needing to ensure pam_keyinit.so is in /etc/pam.d/systemd-user -- but otherwise unsure how this might effect these plans to change the mechanism entirely.

I am far from a PAM expert, learnt most of it debugging this issue, but I'd be happy to have a conversation with anyone not quite understanding how systemd-user and the other session processes are interacting and how it's all relevant. Otherwise perhaps check my full write-up at https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1754270

[ebiggers]

I'm not sure this should be fixed by adding a systemd service for fscrypt. Other than people just not configuring pam_fscrypt correctly, I think the main problem here is that systemd is opening a PAM session for each systemd-user "session", but then not closing it properly. There are two problems with how systemd is closing the PAM session. First, it is incorrectly being closed with dropped privileges, which breaks PAM modules that need to do privileged operations when closing the session, such as unmounting filesystems; or, in the case of fscrypt, writing to /var/run/fscrypt/ or executing the drop_caches sysctl. And secondly, there is a race condition where systemd sends SIGKILL to the process "(sd-pam)" which is executing pam_close_session(), so not all the PAM modules are guaranteed to be run.

It's possible to partially work around this for pam_fscrypt by changing /etc/pam.d/systemd-user to not run the pam_fscrypt hook. For example, on Arch Linux put the pam_fscrypt hooks in system-login (not system-auth), then replace the 'session include system-login' line in systemd-user with the corresponding 'session' lines from system-login, with pam_fscrypt removed. That way, systemd-user would neither increment nor decrement the refcount for fscrypt, so your directories will be "locked" at logout again (although even if you are actually still running a background process through systemd-user, which isn't right).

I think this really needs to be fixed in systemd so that it works for everyone including other PAM modules such as pam_ecryptfs, pam_mount, etc...

[ebiggers]

@josephlr, @tyhicks: I've opened an issue for the underlying systemd bug here: systemd/systemd#8598

[ghost]

For example, on Arch Linux put the pam_fscrypt hooks in system-login (not system-auth), then replace the 'session include system-login' line in systemd-user with the corresponding 'session' lines from system-login, with pam_fscrypt removed

@ebiggers There are cleaner tricks already discussed in #77 (comment)

session [success=1 default=ignore]  pam_succeed_if.so  service = systemd-user quiet
session    optional   pam_fscrypt.so       drop_caches lock_policies

[HerroBert]

No, don't wire that to systemd!

[ghost]

I think it would be great to have fscrypt@.service regardless of linked systemd ticket status - it would make troubleshooting user's setup much easier because 'journalctl --user-unit fscrypt@user.service' is much less arcane than debugging PAM module logic.

[robszy]

Is my understanding correct that issue with users who have systemd and don't follow #196 will have still issue with locking ?

[ebiggers]

Yes, that's correct.

[robszy]

Without systemd how it is guaranteed that after logout user will be sure that his folders ale for sure locked ? It is only by knowledge that nothing kills pam_modules execution at session end ? How user is notified about errors when locking folders won't go according to plan (logging to journal for average user is not enough for me ) ?

[ebiggers]

There is a log message, both from fscrypt and from the kernel. I'm not sure what other sort of notification you would expect; it's not like we can prevent the user from logging out. You could also log in as another user (such as root) and run fscrypt status $dir to check the status of the directory(s) after the user has logged out.

Also note that even with this systemd issue, the directories are always at least "incompletely locked"; they never remain fully unlocked.

[robszy]

But there is impression that when user logs out his data is protected (even root can;t see ) which isn't at least when there are situation when encrypted files are open so root could see those files not mention when something will go terribly wrong :).

Maybe some screen like in windows when app is preventing shutdown ?

When we have no guarantee to lock there is not much to choose between full disc encryption which is so much easier to comprehend for average user and there is so much less space for compromising security. Ok we have greater performance with fs encryption but protection from other users is done by access control and after shutdown we are sure we have locked files like in full disc encryption solutions which doesn't have problem of locking disks because turning off pc solve any error code :). Also in fscrypt we cannot encrypt entire partitions if we want to look like full disk encryption because of ./fscrypt folder not to mention lost+found and other problems so i wonder who use it profesionaly ?

[ebiggers]

The main point of filesystem-level encryption is to allow different files on the same filesystem to be protected by different keys. This is why Android and Chrome OS use it (not the fscrypt userspace tool, as they have their own userspace, but they use the same kernel feature). Being able to "lock" directories is good, and it is unfortunate that it doesn't work properly in some cases, but it's not actually the main point.

If you are using a "regular" single-user system you should certainly consider LUKS/dm-crypt first. We never claimed that fscrypt should always be used. Note that it is also possible to use LUKS/dm-crypt in combination with fscrypt.

[robszy]

Thanks for pointing it out that main focus in on different keys on same filesystem but plain access control or selinux isn't enough ? Or maybe one user have one password and when he compromise it not all users are compromised ? What is use case for this feature to enable different keys for files because I'm not aware of any other solution supporting it ?

[ebiggers]

The threat model primarily includes offline attacks, which OS-level access control doesn't protect against. Having different files protected by different keys is useful in that scenario since different keys can have different levels of protection. In addition, it is useful to allow an encrypted directory to be securely deleted -- for example when a user is deleted, or when their work profile is deleted, etc.

If you have more questions, please take a look at the kernel documentation, as it goes into more detail. Android's File Based Encryption documentation might also be helpful.

What is use case for this feature to enable different keys for files because I'm not aware of any other solution supporting it ?

There are many other implementations of filesystem-level encryption on Linux that support multiple keys per underlying filesystem, including eCryptfs and various FUSE filesystems.

[robszy]

I don't understand this line:

"Having different files protected by different keys is useful in that scenario since different keys can have different levels of protection"

Everything it cyphered by aes so on kernel level there is one level of protection.

Keys in Android are stored in TEE so I think there will be only good level of protection of those keys :)

So where we have different levels of protection ? It will go down to user password ?

[ebiggers]

I'm talking about the information needed to unlock the keys, not how many layers of encryption there are. For example, Android has Device Encrypted storage which is bound to the Verified Boot state only, not to any user's credentials; as well as per-user Credential Encrypted storage which is bound to the Verified Boot state as well as the credentials (PIN/pattern/password) for a particular user.

[robszy]

Ok so for only boot up there is lower level of protection than for unlocking user data like in CE storage because DE only need secureboot so it is unlocked for all mainly but DE and CE for offline are equally hard to decrypt. Thats right ?

So Android can boot up without user key because DE is unlocked by secure boot but to get user data user must login but for offline attacks it doesn't matter if user give pass before boot or after. Unlocking userdata before boot or after isn't relevant for offline attacks and also me because there isn't much i can do without unlocking my Android.

Summing up filesytem encryption is not better from whole disk encryption from point of offline attack. As there is usually one user of phone there is no need for locking feature as you said but it can be useful if you don't want root to have access to your files.

Goggle only advertise about DirectBoot feature:

" Android 5.0 up to Android 9 support full-disk encryption. Full-disk encryption uses a single key—protected with the user’s device password—to protect the whole of a device’s userdata partition. Upon boot, the user must provide their credentials before any part of the disk is accessible.

While this is great for security, it means that most of the core functionality of the phone is not immediately available when users reboot their device. Because access to their data is protected behind their single user credential, features like alarms could not operate, accessibility services were unavailable, and phones could not receive calls. "

So much work for that little ?

[ebiggers]

DE and CE for offline are equally hard to decrypt. Thats right ?

No, because the credentials are needed to unlock CE but not DE.

Summing up filesytem encryption is not better from whole disk encryption from point of offline attack.

It depends on how well the whole-disk key is protected. Is it protected in the weakest way or in the strongest way?

So much work for that little ?

The switch to file-based encryption greatly increased the amount of users who actually protect their personal data with their credentials. With full-disk encryption, in practice most users didn't require a PIN to decrypt because of the usability problems (e.g. after a reboot, alarms don't go off and can't receive phone calls). With file-based encryption it is much easier to get users to actually use strong encryption.

Anyway, this has gotten off-topic from this issue. If you have more questions, please file a separate issue.

[robszy]

alarms don't go off and can't receive phone calls

Ok it is good in case of phone device where we receive phone calls and others and Android manages filesystem automatically so that all user data is encrypted so he doesn't copy from encrypted storage to unencrypted like in normal linux only by mistake compromising data.

In linux it will be very hard to keep encrypted and unencrypted files mixed together unless there is a automatic management like in Android which is impossible so in my view in README should be warning that firstly you should try FDE and if it totally doesn't work get fscrypt but it will be very hard not to make mistake and use it properly. Also ecryptfs is dying for 5 years or so in which case there is no real usecase in linux for FBE as ubuntu abandoned it and after years of void fscrypt seams to fill it.

ALso i think we need warning in README that systemd users should be warned about problems with locking files on logout because they may think that on logout everything is locked as with CE in Android I think or maybe not (really it doesn't matter as Android is mostly single user system) ? but it can be other way so BE WARNED that your data is not protected from online attack from root :) to date that this issue is resolved.

[Redsandro]

When we have no guarantee to lock there is not much to choose between full disc encryption which is so much easier to comprehend for average user

Consider that there are still average households where average users share a desktop or laptop. Full disk encryption means that everyone needs to share the same decryption key. You can set multiple passwords in LUKS, but they all unlock the same key. This means all users can access each others' files using a bootable USB stick.

I'm not sure it is easier to comprehend that people need to share a key and that it's somehow safer. Family members want to protect their privacy from each other by encrypting their personal data (home directory) using file based encryption with their own key. Now no one can bypass access control using a bootable USB stick.

(..) so root could see those files not mention when something will go terribly wrong.

This issue is similar to and arguably worse on LUKS. Once you have a root user that you don't trust, since they share the LUKS key, they can see all files from all users all the time. With fscrypt, a user can shut down the computer and everything is locked.

really it doesn't matter as Android is mostly single user system

Perhaps in the smartphone market. For Android tablets it's more common for households have a single tablet with multiple user accounts. It's the popularity of tablets that caused the multiple account feature to be introduced with Android 4.2 iirc. That's 2012ish.

In linux it will be very hard to keep encrypted and unencrypted files mixed together

If you just migrated from Windows yes, although arguably the possibility to mix encrypted and unencrypted files is messy. I think the way things work with fscrypt is clear for example when you "encrypt a home directory". It makes logical sense that files you drag into your home get encrypted, and when you move them out they get unencrypted.

[robszy]

I was thinking mostly about one user desktop.

Now no one can bypass access control using a bootable USB stick.

If users want to protect from each other there is nothing to defend about LUKS in multiuser pc: all is decrypted so fscrypt is better for sure but be aware that root will see everything when you are logged in so you won't be protected all the time as in one user case.

But when you can turn off pc and get files using bootable usb you can also change root pass and look at others files so it is dangerous to use fscrypt because anyone can be root and look at someone else files when those users are logged in so it is compromising security totally :(.
fscrypt will be good only for offline attacks from other users which isn't needed in household as it can be bypassed so easly getting root from usbstick. Root could be logged by ssh and steal everything you have :(.
It is very risky to use fscrypt in that environment. Of couse there are means of preventing this it is not simple task and someone can hijack those prevention to be able to do it.

So there is no prevention from root and anyone else looking at your files and if user doesn't want anyone to look in his files the simplest way is LUX and his only PC with his only password to luks.

For Android tablets it's more common for households have a single tablet with multiple user accounts

Android as I;ve said it totally diffrent pair of socks as it is closed box case without root access.

It makes logical sense that files you drag into your home get encrypted, and when you move them out they get unencrypted

Is is always room for error, catastrophic sometimes and here one user PC with luks wins security competition because you just can;t make mistake.
This is impossible in fscrypt.

Cheers Robert

[josephlr]

#350 (comment) brings up another reason why we might want to avoid using PAM modules written in Go. Having fscrypt run in a systemd service might help resolve some of these issues.

[mikunimaru]

Relocking using systemd can be achieved without major code changes. #196 (comment)
There are too many bug reports regarding PAM relock processing, so I think it would be a good idea to replace it first.
In a sense, the relocking process is already broken, so even if a bug is discovered after migrating to systemd, it has been proven that the impact will be small, so it is ideal for testing.