Strategy to enroll existing manually created user datasets (like encrypted ones) #81

nielsk · 2020-04-06T05:51:13Z

I created a new dataset for a user, so that I can use zfs native encryption for the home. When I did zsysctl save it told me that there is no association for that user and did some autosave of the old filesystem. Now I destroyed the old file system and now zsysctl errors out because the file system does not exist.
But there is the encrypted file system and its mount point is the home of the user.
How do I associate a zfs with a user, so that zsysctl knows what to do?

Even more fun - and I did not try it yet: ubuntu does not create new zfs for new users but just creates new home-directories under /home which is in rpool/USERDATA thus they will not be able to save their homes. But that is a problem of ubuntu.

didrocks · 2020-04-06T07:45:26Z

Even more fun - and I did not try it yet: ubuntu does not create new zfs for new users but just creates new home-directories under /home which is in rpool/USERDATA thus they will not be able to save their homes. But that is a problem of ubuntu.

Oh, this isn’t supposed to happen, Let me have a look there, adduser and gnome-control-cener and anything that is using useradd should create the new datasets. That may be a regression in focal compared to eoan. I’m having a look.

How do I associate a zfs with a user, so that zsysctl knows what to do?
However, indeed, if you want special options like per dataset encryption, this isn’t supported directly with zsysctl yet for focal. You can associate it quite easily though:

get your current root dataset path: (with zfs mount for instance), let’s say it’s rpool/ROOT/ubuntu_12345678
Tag your newly created user dataset (which should start with username, for instance rpool/USERDATA/user1_abcdef to associate with the root pool: zfs set com.ubuntu.zsys:bootfs-datasets=rpool/ROOT/ubuntu_12345678 rpool/ROOT/ubuntu_12345678
That should be it, if you want to check that everything is alright:

if no zsysd is running, just run zsysctl show --full to list user datasets
if zsysd was still running (it’s exiting after one minute of idling), zsysctl service refresh && zsysctl show --full.
We should make an utility for this until we have a better user-experience for it.

Keep us posted!

didrocks · 2020-04-06T07:53:32Z

Even more fun - and I did not try it yet: ubuntu does not create new zfs for new users but just creates new home-directories under /home which is in rpool/USERDATA thus they will not be able to save their homes. But that is a problem of ubuntu.

Oh, this isn’t supposed to happen, Let me have a look there, adduser and gnome-control-cener and anything that is using useradd should create the new datasets. That may be a regression in focal compared to eoan. I’m having a look.

You’re right, I pushed a fix which will be building in focal shortly (shadow package 4.8.1-1ubuntu4) once accepted by the freeze.

nielsk · 2020-04-06T08:48:23Z

Thanks. I will try that out later. I just broke an installation and have to re-install because I played around too much.

…

On 6. Apr 2020, at 09:45, Didier Roche ***@***.***> wrote: Even more fun - and I did not try it yet: ubuntu does not create new zfs for new users but just creates new home-directories under /home which is in rpool/USERDATA thus they will not be able to save their homes. But that is a problem of ubuntu. Oh, this isn’t supposed to happen, Let me have a look there, adduser and gnome-control-cener and anything that is using useradd should create the new datasets. That may be a regression in focal compared to eoan. I’m having a look. How do I associate a zfs with a user, so that zsysctl knows what to do? However, indeed, if you want special options like per dataset encryption, this isn’t supported directly with zsysctl yet for focal. You can associate it quite easily though: get your current root dataset path: (with zfs mount for instance), let’s say it’s rpool/ROOT/ubuntu_12345678 Tag your newly created user dataset (which should start with username, for instance rpool/USERDATA/user1_abcdef to associate with the root pool: zfs set com.ubuntu.zsys:bootfs-datasets=rpool/ROOT/ubuntu_12345678 rpool/ROOT/ubuntu_12345678 That should be it, if you want to check that everything is alright: if no zsysd is running, just run zsysctl show --full to list user datasets if zsysd was still running (it’s exiting after one minute of idling), zsysctl service refresh && zsysctl show --full. We should make an utility for this until we have a better user-experience for it. Keep us posted! — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

didrocks · 2020-04-06T08:55:26Z

Please keep breakikng it and report anything that is weird! :)

nielsk · 2020-04-07T05:06:55Z

Tag your newly created user dataset (which should start with username, for instance rpool/USERDATA/user1_abcdef to associate with the root pool: zfs set com.ubuntu.zsys:bootfs-datasets=rpool/ROOT/ubuntu_12345678 rpool/ROOT/ubuntu_12345678

I guess this is

zfs set com.ubuntu.zsys:bootfs-datasets=rpool/ROOT/ubuntu_12345678 rpool/USERDATA/user1_abcdef

We should make an utility for this until we have a better user-experience for it.

Or mention it in the man-page of zsys. I didn't look up the zfs-properties because I totally didn't expect it there.

nielsk · 2020-04-07T05:28:27Z

meh. There are no real man-pages. And it seems you can only find out about that property with a

zfs get all

on a pre-existing dataset (preferably a home)

nielsk · 2020-04-07T05:35:36Z

This seems to work.
I had to destroy the old dataset and do another refresh to get no errors anymore.

didrocks · 2020-04-07T06:48:40Z

Oh right, sorry about the typo :) And indeed, you need to deassociate curent dataset.
There will be a series of blog posts (more and less technical) once 20.04 LTS is out to cover those aspects.

The shadow package is uploaded which will fix "create userdataset on demand", but manual ones like you did won't until we have a better way to enroll user dataset. I’ll retitle this bug for this.
Keep bug reports coming if you spot anything else bad!

nielsk · 2020-04-07T06:53:56Z

Blog posts are great but having manuals on my system without the need to google are far better because I do not have to rely on search engines but use the tools available on my system.

…

On 7. Apr 2020, at 08:48, Didier Roche ***@***.***> wrote: Oh right, sorry about the typo :) And indeed, you need to deassociate curent dataset. There will be a series of blog posts (more and less technical) once 20.04 LTS is out to cover those aspects. The shadow package is uploaded which will fix "create userdataset on demand", but manual ones like you did won't until we have a better way to enroll user dataset. I’ll retitle this bug for this. Keep bug reports coming if you spot anything else bad! — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

didrocks · 2020-04-07T06:56:20Z

It’s hard as implementer to know what is really technical details or what could be interesting to advanced users who wants to understand a little bit more and to some manual work, while still being compatible with the design.

If I write those blog posts would you be interested in helping crafting those manuals (probably in markdown) here so that we can ship them as man pages in the distro?

nielsk · 2020-04-07T07:01:42Z

I might try to help out. I am not a technical writer but I can give it a a shot.

…

On 7. Apr 2020, at 08:56, Didier Roche ***@***.***> wrote: It’s hard as implementer to know what is really technical details or what could be interesting to advanced users who wants to understand a little bit more and to some manual work, while still being compatible with the design. If I write those blog posts would you be interested in helping crafting those manuals (probably in markdown) here so that we can ship them as man pages in the distro? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

FliegenKLATSCH · 2020-05-01T09:34:09Z

I first tried zsysctl userdata set-home but finally found this issue...
Some zsysctl userdata xxx alias would be great for this simple task.
Maybe set-home should be named rename and the set-home task should set this property?

didrocks · 2020-05-01T09:43:12Z

Those are hidden commands on purpose: just to change the user home directory, please use rather usermod -m -d which is the command intended to it (it refreshes AccountServices and other things) to change the user home directory.

The command is hooked up to zsysctl userdata set-home which DTRT.

FliegenKLATSCH · 2020-05-01T10:10:44Z

I had the same use-case as the issue creator: I manually created a new encrypted home volume and renamed it with zfs to match the old name. I just noticed on apt update's a warning and investigated... I think encrypted home folders will become a popular feature because with zfs it's quite easy to archive..

didrocks · 2020-05-01T10:31:21Z

Agreed! But the fix is thus different: set-home is just a workaround, we need a dedicated command to enroll existing home directory (if created under USERDATA) filesystem datasets.

I think something like zsysctl machine add-user-dataset or something like that? (This command would check if it’s under USERDATA, and then just tag it to associate to current system + unassociate (after a prompt, potenially) any other dataset associated for this user with this machine.
None of this check is done with set-home as its purpose is different and some checks are already done beforehand by the usermod command.

On encryption, agreed, and this is why we are working on bringing to 20.10 encryption as an easy option by default! (This will be probably backported to 20.04.x release).

Deleted user have all their datasets with an empty bootfs dataset and ends up in the unmanaged datasets list. We clean up any of them using GC when the value com.ubuntu.zsys:bootfs-datasets is empty, but was set. We ensure that any dependencies follow the same deletion rule first. Fixes: #81 Co-authored-by: Jean-Baptiste Lallement <jean-baptiste@ubuntu.com>

mcamou · 2020-05-29T12:37:24Z

I have a similar problem. I installed Ubuntu 19.10 from scratch and set it up with ZFS. Since I wanted to encrypt my home directory (and possibly others) I set up an rpool/encrypted dataset, moved my home directory there, and did zfs set mountpoint=/home/mario rpool/encrypted/mario. I also have other datasets under my home directory so that I can snapshot certain parts of it.

I also moved rpool/ROOT/var/lib/docker to rpool/docker just to keep the output of zfs list more manageable, since I am constantly running different Docker containers and zfs list ends up really cluttered.

I have now updated to 20.04 and find that the choices I made with 19.04 result in zsys not picking up my home directory and /var/lib/docker. I would really like to add directories under rpool/encrypted and rpool/docker to zsys.

I tried zfs set com.ubuntu.zsys:bootfs-datasets=rpool/ROOT/ubuntu_mjgbmr rpool/encrypted/mario as mentioned above, but it did not help.

zsysctl show --full gives me the following (which is completely non-obvious):

... (system datasets)

Persistent Datasets:
                        - &{rpool/docker %!!(MISSING)s(bool=false) {/var/lib/docker on %!!(MISSING)s(bool=true) %!!(MISSING)s(bool=false) %!!(MISSING)s(int=0)  rpool/ROOT/ubuntu_mjgbmr  {local     local}} [] {%!!(MISSING)s(*zfs.Dataset=&{0x257cd60 1 map[1:{1590752657 none} 6:{yes none} 13:{/var/lib/docker local} 27:{rpool/docker none} 28:{on default}] []})}}
                        - &{rpool/encrypted %!!(MISSING)s(bool=false) {none on %!!(MISSING)s(bool=false) %!!(MISSING)s(bool=false) %!!(MISSING)s(int=0)  rpool/ROOT/ubuntu_mjgbmr  {local     local}} [%!!(MISSING)s(*zfs.Dataset=&{rpool/encrypted/mario false {/home/mario on true false 0  rpool/ROOT/ubuntu_mjgbmr  {local     local}} [0xc00019e500 0xc00019e900] {0xc000181f20}}) %!!(MISSING)s(*zfs.Dataset=&{rpool/encrypted@pre-20.04 true {  false false 1590748562    {     }} [] {0xc0001a6300}})] {%!!(MISSING)s(*zfs.Dataset=&{0x2572380 1 map[1:{1571340696 none} 6:{no none} 13:{none local} 27:{rpool/encrypted none} 28:{on default}] []})}}
                        - &{rpool/encrypted/mario %!!(MISSING)s(bool=false) {/home/mario on %!!(MISSING)s(bool=true) %!!(MISSING)s(bool=false) %!!(MISSING)s(int=0)  rpool/ROOT/ubuntu_mjgbmr  {local     local}} [%!!(MISSING)s(*zfs.Dataset=&{rpool/encrypted/mario/actyx false {/home/mario/work/actyx on true false 0  rpool/ROOT/ubuntu_mjgbmr  {local     inherited}} [0xc00019e600 0xc00019e800] {0xc0001a6030}}) %!!(MISSING)s(*zfs.Dataset=&{rpool/encrypted/mario@pre-20.04 true {  false false 1590748562    {     }} [] {0xc0001a62a0}})] {%!!(MISSING)s(*zfs.Dataset=&{0x26034b0 1 map[1:{1571413005 none} 6:{yes none} 13:{/home/mario local} 27:{rpool/encrypted/mario none} 28:{on default}] []})}}

... (history)

Users:
  - Name:    root
    History: 
     - rpool/USERDATA/root_rlz0uc@autozsys_4ynidi (2020-05-29 14:41:43): rpool/USERDATA/root_rlz0uc@autozsys_4ynidi
     - rpool/USERDATA/root_rlz0uc@pre-20.04 (2020-05-29 12:36:02): rpool/USERDATA/root_rlz0uc@pre-20.04

didrocks changed the title ~~Created new dataset for user (for encryption), won't be saved~~ Strategy to enroll existing manually created user datasets (like encrypted ones) Apr 7, 2020

didrocks added the enhancement New feature or request label Jun 8, 2020

taegge mentioned this issue Feb 1, 2022

USERDATA datasets removed #218

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Strategy to enroll existing manually created user datasets (like encrypted ones) #81

Strategy to enroll existing manually created user datasets (like encrypted ones) #81

nielsk commented Apr 6, 2020

didrocks commented Apr 6, 2020

didrocks commented Apr 6, 2020

nielsk commented Apr 6, 2020 via email

didrocks commented Apr 6, 2020

nielsk commented Apr 7, 2020

nielsk commented Apr 7, 2020 •

edited

Loading

nielsk commented Apr 7, 2020

didrocks commented Apr 7, 2020

nielsk commented Apr 7, 2020 via email

didrocks commented Apr 7, 2020

nielsk commented Apr 7, 2020 via email

FliegenKLATSCH commented May 1, 2020

didrocks commented May 1, 2020

FliegenKLATSCH commented May 1, 2020

didrocks commented May 1, 2020

mcamou commented May 29, 2020 •

edited

Loading

Strategy to enroll existing manually created user datasets (like encrypted ones) #81

Strategy to enroll existing manually created user datasets (like encrypted ones) #81

Comments

nielsk commented Apr 6, 2020

didrocks commented Apr 6, 2020

didrocks commented Apr 6, 2020

nielsk commented Apr 6, 2020 via email

didrocks commented Apr 6, 2020

nielsk commented Apr 7, 2020

nielsk commented Apr 7, 2020 • edited Loading

nielsk commented Apr 7, 2020

didrocks commented Apr 7, 2020

nielsk commented Apr 7, 2020 via email

didrocks commented Apr 7, 2020

nielsk commented Apr 7, 2020 via email

FliegenKLATSCH commented May 1, 2020

didrocks commented May 1, 2020

FliegenKLATSCH commented May 1, 2020

didrocks commented May 1, 2020

mcamou commented May 29, 2020 • edited Loading

nielsk commented Apr 7, 2020 •

edited

Loading

mcamou commented May 29, 2020 •

edited

Loading