Feature: replace root by a custom user with root privileges #1009

aleksei-burlakov · 2022-08-16T07:59:33Z

It enables creating the cluster under a custom user
Assume the users are alice@node1 and non@node2.

Bring the alice into sudoers file on the node1
and bob on the node2
for example
$ sudo echo "alice ALL=(ALL:ALL) NOPASSWD: ALL" >> /etc/sudoers
(use visudo rather than echo)
add /usr/sbin to the PATH for alice and bob (writ it in the ~/.bashrc)
$ export PATH=$PATH:/usr/sbin

And now you can do the usual routines under alice

alice@node1$ crm cluster init -y
bob@node2$ crm cluster join -c alice@node1 -y
or
alice@node1$ crm cluster init -N bob@node2 -y

The crm will start as a usual process with the same context as the
current user (alice/bob). However when it would lack permissions
for example to read/write file it would try again as the super-user.

liangxin1300 · 2022-08-22T06:29:02Z

Thanks @aleksei-burlakov !

In my environment

sudo mv /sbin/crm* /sbin/cibadmin /sbin/stonith \
        /sbin/corosync-keygen /sbin/corosync-cfgtool /usr/bin/

crm* cibadmin stonith corosync-keygen corosync-cfgtool are under /usr/sbin

And my other concern is it might impact on customers existing scripts if these commands change the path

And not sure the customers will allow to change the mod like /etc/sysconfig /etc/lvm /etc/samba

@zzhou1 @gao-yan @nicholasyang2022 What do you think?

nicholasyang2022 · 2022-08-22T07:24:12Z

Instead of moving these binaries, we may create symlinks.

Changing the mode of any directory to 777 is dangerous. And changing the owner of base system directories, i.e. /etc/sysconfig, /etc/lvm, /run may breaks other programs.

liangxin1300 · 2022-12-21T01:17:23Z

Hi @aleksei-burlakov
I mean at least use 3 commits in this PR

The codes under crmsh/crmsh directory
The unittest code under crmsh/test/unittests directory, the commit log should starts with Dev: unittest:
The functional test code under crmsh/test/features directory, the commit log should starts with Dev: behave:

And for the PR title, keep the original one is good enough Feature: replace root by a custom user with root provileges

Thank you!

liangxin1300 · 2022-12-21T02:28:51Z

alice@15sp4-1:~> crm cluster init -s /dev/sda1 -y
INFO: alice tried to read cib.xml, but : [Errno 13] Permission denied: '/var/lib/pacemaker/cib/cib.xml'
INFO: alice gonna try it with sudo
INFO: alice successfully read the cib.xml
INFO: Loading "default" profile from /etc/crm/profiles.yml
WARNING: chronyd.service is not configured to start at system boot.
INFO: Configuring csync2
INFO: Starting csync2.socket service on 15sp4-1
INFO: BEGIN csync2 checking files
INFO: END csync2 checking files
INFO: Configuring corosync (unicast)
INFO: Initializing SBD
WARNING: Failed to reset password of hacluster user: passwd: You may not view or modify password information for hacluster.
INFO: Hawk cluster interface is now running. To see cluster status, open:
INFO:   https://192.168.122.202:7630/
INFO: Log in with username 'hacluster'
INFO: BEGIN Starting pacemaker
INFO: END Starting pacemaker
INFO: BEGIN Waiting for cluster
...........                                                                                                                                                                                   INFO: END Waiting for cluster
INFO: Loading initial cluster configuration
INFO: Done (log saved to /var/log/crmsh/crmsh.log)

In above process, for this info log:

INFO: alice tried to read cib.xml, but : [Errno 13] Permission denied: '/var/lib/pacemaker/cib/cib.xml'
INFO: alice gonna try it with sudo
INFO: alice successfully read the cib.xml

I think user is not interest with these, maybe we can put these 3 lines into debug log?

And I noticed this warning

WARNING: Failed to reset password of hacluster user: passwd: You may not view or modify password information for hacluster.

Is it possible to fix this?

@zzhou1 What do you think?

liangxin1300 · 2022-12-21T03:45:15Z

For the join process, I noticed that there is a failed log:

alice@15sp4-2:~> crm cluster join -c alice@15sp4-1 -y
WARNING: chronyd.service is not configured to start at system boot.
INFO: SSH key for alice does not exist, hence generate it now
INFO: Configuring SSH passwordless with alice@15sp4-1
Password: 
INFO: SSH key for hacluster does not exist, hence generate it now
INFO: Failed to create ~hacluster/.ssh

Failed to create ~hacluster/.ssh
Could you please take a look at this?

liangxin1300 · 2022-12-21T09:00:03Z

So many places add sudo before command, in case further PR forget to add sudo, how about add sudo inside these functions?

utils.get_stdout_stderr
utils.get_stdout
utils.get_stdout_or_raise_error

And how about using utils.add_sudo function?

aleksei-burlakov · 2022-12-21T14:14:51Z

For the join process, I noticed that there is a failed log:

alice@15sp4-2:~> crm cluster join -c alice@15sp4-1 -y
WARNING: chronyd.service is not configured to start at system boot.
INFO: SSH key for alice does not exist, hence generate it now
INFO: Configuring SSH passwordless with alice@15sp4-1
Password: 
INFO: SSH key for hacluster does not exist, hence generate it now
INFO: Failed to create ~hacluster/.ssh

Failed to create ~hacluster/.ssh Could you please take a look at this?

AFAI remember it fails to do the /usr/bin/env python3 -m crmsh.healthcheck fix-cluster PasswordlessHaclusterAuthenticationFeature I didn't know how to fix it and I simply gave up on it

aleksei-burlakov · 2022-12-21T14:29:21Z

So many places add sudo before command, in case further PR forget to add sudo, how about add sudo inside these functions?
utils.get_stdout_stderr
utils.get_stdout
utils.get_stdout_or_raise_error

I want to use sudo as less as possible, only when it doesn't work completely without it. It's not all in this PR, but at least I set the direction.

And how about using utils.add_sudo function?

I don't have a strong opinion about it, I think it was simpler to put sudo directly into the command literals.

aleksei-burlakov · 2022-12-21T14:34:01Z

alice@15sp4-1:~> crm cluster init -s /dev/sda1 -y
INFO: alice tried to read cib.xml, but : [Errno 13] Permission denied: '/var/lib/pacemaker/cib/cib.xml'
INFO: alice gonna try it with sudo
INFO: alice successfully read the cib.xml
INFO: Loading "default" profile from /etc/crm/profiles.yml
WARNING: chronyd.service is not configured to start at system boot.
INFO: Configuring csync2
INFO: Starting csync2.socket service on 15sp4-1
INFO: BEGIN csync2 checking files
INFO: END csync2 checking files
INFO: Configuring corosync (unicast)
INFO: Initializing SBD
WARNING: Failed to reset password of hacluster user: passwd: You may not view or modify password information for hacluster.
INFO: Hawk cluster interface is now running. To see cluster status, open:
INFO:   https://192.168.122.202:7630/
INFO: Log in with username 'hacluster'
INFO: BEGIN Starting pacemaker
INFO: END Starting pacemaker
INFO: BEGIN Waiting for cluster
...........                                                                                                                                                                                   INFO: END Waiting for cluster
INFO: Loading initial cluster configuration
INFO: Done (log saved to /var/log/crmsh/crmsh.log)

In above process, for this info log:

INFO: alice tried to read cib.xml, but : [Errno 13] Permission denied: '/var/lib/pacemaker/cib/cib.xml'
INFO: alice gonna try it with sudo
INFO: alice successfully read the cib.xml

I think user is not interest with these, maybe we can put these 3 lines into debug log?

Sure

And I noticed this warning
WARNING: Failed to reset password of hacluster user: passwd: You may not view or modify password information for hacluster.
Is it possible to fix this?

@zzhou1 What do you think?

I think it should be possible, but I'm in a trip and this fixing would take at least a couple of days.

It enables creating the cluster under a custom user Assume the users are alice@node1 and non@node2. 1) Bring the alice into sudoers file on the node1 and bob on the node2 for example $ sudo echo "alice ALL=(ALL:ALL) NOPASSWD: ALL" >> /etc/sudoers (use visudo rather than echo) 2) Bring the alice and bob into the haclient group (group 90 in the /etc/passwd) 3) add /usr/sbin to the PATH for alice and bob $ export PATH=$PATH:/usr/sbin 4) Change the owner of the /var/log/crmsh to hacluster:haclient (this is an unwanted feature and should be fixed in future) And now you can do the usual routines under alice alice@node1$ crm cluster init -y bob@node2$ crm cluster join -c alice@node1 -y or alice@node1$ crm cluster init -N bob@node2 -y The crm will start as a usual process with the same context as the current user (alice/bob). However when it would lack permissions for example to read/write file it would try again as the super-user. Demo: https://asciinema.org/connect/088c5b1b-0fdc-4686-9a20-80fad830fca2

This is just the first step to anchor the PR#1009. There will follow other more advanced tests.

aleksei-burlakov · 2022-12-21T15:40:31Z

Hi @aleksei-burlakov I mean at least use 3 commits in this PR

The codes under crmsh/crmsh directory

The unittest code under crmsh/test/unittests directory, the commit log should starts with Dev: unittest:

The functional test code under crmsh/test/features directory, the commit log should starts with Dev: behave:

And for the PR title, keep the original one is good enough Feature: replace root by a custom user with root provileges

Thank you!

Changed!

nicholasyang2022 · 2022-12-22T03:00:03Z

In above process, for this info log:
INFO: alice tried to read cib.xml, but : [Errno 13] Permission denied: '/var/lib/pacemaker/cib/cib.xml'
INFO: alice gonna try it with sudo
INFO: alice successfully read the cib.xml
I think user is not interest with these, maybe we can put these 3 lines into debug log?

@liangxin1300 Maybe privilege escalation should not be used in silent. Users need to be aware that we are calling sudo.

zzhou1 · 2022-12-22T03:47:53Z

alice@15sp4-1:~> crm cluster init -s /dev/sda1 -y
INFO: alice tried to read cib.xml, but : [Errno 13] Permission denied: '/var/lib/pacemaker/cib/cib.xml'
INFO: alice gonna try it with sudo
INFO: alice successfully read the cib.xml

I think user is not interest with these, maybe we can put these 3 lines into debug log?

Partially.

In this case, I see the value to message the end users, but in the very beginning. Something like,

alice@15sp4-1:~> crm cluster init -s /dev/sda1 -y
INFO: crmsh bootstrap requires sudo access to run as root.

Then, followed by validation sudo -v. crmsh should immediately exit if sudo -v fails, which returns 1.

And I noticed this warning
WARNING: Failed to reset password of hacluster user: passwd: You may not view or modify password information for hacluster.
Is it possible to fix this?
@zzhou1 What do you think?
I think it should be possible, but I'm in a trip and this fixing would take at least a couple of days.

Great, take your time. This indeed need fix.

aleksei-burlakov · 2022-12-22T18:12:24Z

@nicholasyang2022 would you please help me with this https://github.com/ClusterLabs/crmsh/pull/1009/files#diff-4487c485cac79de78577193a11ac9f6aaf58c386a694bb96588d4a822974122bR1520 ?

```
continue_with_hacluster = True
try:
    utils.get_stdout_or_raise_error(
            '/usr/bin/env python3 -m crmsh.healthcheck fix-cluster PasswordlessHaclusterAuthenticationFeature',
            user=remote_user, remote=seed_host,
        )
except ValueError as err:
    continue_with_hacluster = False # at least we tried
    logger.info("Failed to create ~hacluster/.ssh")


It fails when being called under alice. We would add it as a separate commit by your authorship.

nicholasyang2022 · 2022-12-30T03:04:58Z

@aleksei-burlakov This piece of code is not expected to be called when initializing a new cluster. It works only on a running cluster, and depends on crm cluster init ssh -y and crm cluster join ssh -y -c. To generate keys for hacluster during initialization, you should fix configure_ssh_key("hacluster") for that.

The healthcheck module also needs changes to work with non-root privileged user, and I am working on it now. You can remove the codes above, it should not affect initializing a new cluster. And when I finish, I will add it back.

nicholasyang2022 · 2022-12-30T03:54:11Z

crmsh/bootstrap.py

+    # Make sure ~hacluster/.ssh exist remotely (idempotent)
+    continue_with_hacluster = True
+    try:
+        utils.get_stdout_or_raise_error(
+                '/usr/bin/env python3 -m crmsh.healthcheck fix-cluster PasswordlessHaclusterAuthenticationFeature',
+                user=remote_user, remote=seed_host,
+            )
+    except ValueError as err:
+        continue_with_hacluster = False # at least we tried
+        logger.info("Failed to create ~hacluster/.ssh")
+
+    if continue_with_hacluster:
+        swap_public_ssh_key(local_user, remote_user, 'hacluster', seed_host)
+        swap_public_ssh_key('hacluster', remote_user, 'hacluster', seed_host)


Suggested change

# Make sure ~hacluster/.ssh exist remotely (idempotent)

continue_with_hacluster = True

try:

utils.get_stdout_or_raise_error(

'/usr/bin/env python3 -m crmsh.healthcheck fix-cluster PasswordlessHaclusterAuthenticationFeature',

user=remote_user, remote=seed_host,

)

except ValueError as err:

continue_with_hacluster = False # at least we tried

logger.info("Failed to create ~hacluster/.ssh")

if continue_with_hacluster:

swap_public_ssh_key(local_user, remote_user, 'hacluster', seed_host)

swap_public_ssh_key('hacluster', remote_user, 'hacluster', seed_host)

swap_public_ssh_key('hacluster', remote_user, 'hacluster', seed_host, add=True)

liangxin1300 · 2023-01-05T02:45:35Z

crmsh/utils.py

@@ -2999,9 +3162,9 @@ def get_property(name, property_type="crm_config"):
    """
    if property_type == "crm_config":
        cib_path = os.getenv('CIB_file', constants.CIB_RAW_FILE)
-        cmd = "CIB_file={} crm configure get_property {}".format(cib_path, name)
+        cmd = "CIB_file={} sudo -E CIB_file crm configure get_property {}".format(cib_path, name)


This line might has issue

# CIB_file=/var/lib/pacemaker/cib/cib.xml sudo -E CIB_file crm configure get_property stonith-timeout sudo: CIB_file: command not found

The result is the return value of get_property is None

My mistake. The parameter should be --preserve-env=CIB_file

Fixed in #1109

liangxin1300 · 2023-01-05T09:06:54Z

Current master code, in Azure, since PasswordAuthentication is disabled, join process will failed with this error hint:

 # crm cluster join -c crmsh-node1 -y
INFO: Configuring SSH passwordless with root@crmsh-node1
ERROR: cluster.join: Failed to run "ssh-copy-id -i /root/.ssh/id_rsa.pub root@crmsh-node1": /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@crmsh-node1: Permission denied (publickey).

    crmsh has no way to help you to setup up passwordless ssh among nodes at this time.
    As the hint, likely, `PasswordAuthentication` is 'no' in /etc/ssh/sshd_config.
    Given in this case, users must setup passwordless ssh beforehand, or change it to 'yes' and manage passwords properly

Applied with this PR, this error hint gone:

# crm cluster join -c crmsh-node1 -y
INFO: Configuring SSH passwordless with root@crmsh-node1
ERROR: cluster.join: Failed to run "cat ~root/.ssh/id_rsa.pub | ssh -o StrictHostKeyChecking=no root@crmsh-node1 'cat >> ~root/.ssh/authorized_keys'": root@crmsh-node1: Permission denied (publickey).

Suggest to keep this error hint unchanged

liangxin1300 · 2023-01-05T11:44:44Z

@aleksei-burlakov
Thanks for your great work!

aleksei-burlakov force-pushed the nonprivileged-user branch 9 times, most recently from 7278508 to d35e38c Compare August 18, 2022 15:13

zzhou1 requested review from gao-yan, liangxin1300, zzhou1 and nicholasyang2022 August 19, 2022 08:03

aleksei-burlakov force-pushed the nonprivileged-user branch 6 times, most recently from 1be5559 to ed2a9e7 Compare August 21, 2022 16:51

aleksei-burlakov force-pushed the nonprivileged-user branch 9 times, most recently from a1b6ef8 to f2419d6 Compare August 25, 2022 12:08

aleksei-burlakov force-pushed the nonprivileged-user branch 2 times, most recently from 4d74918 to 5ae3326 Compare December 20, 2022 12:38

aleksei-burlakov force-pushed the nonprivileged-user branch 2 times, most recently from 697b813 to 1dcb53e Compare December 21, 2022 15:12

Aleksei Burlakov added 3 commits December 21, 2022 16:35

Dev: unittest: replace root by alice and bob

3e97b74

Dev: behave: add basic alice-bob init-join test

424c897

This is just the first step to anchor the PR#1009. There will follow other more advanced tests.

aleksei-burlakov force-pushed the nonprivileged-user branch from 1dcb53e to 424c897 Compare December 21, 2022 15:39

zzhou1 changed the title ~~Dev: behave: replace root by a custom user with root provileges~~ Dev: Feature: replace root by a custom user with root privileges Dec 22, 2022

zzhou1 changed the title ~~Dev: Feature: replace root by a custom user with root privileges~~ Feature: replace root by a custom user with root privileges Dec 22, 2022

nicholasyang2022 reviewed Dec 30, 2022

View reviewed changes

liangxin1300 reviewed Jan 5, 2023

View reviewed changes

liangxin1300 approved these changes Jan 5, 2023

View reviewed changes

liangxin1300 merged commit 6c47d1c into ClusterLabs:master Jan 5, 2023

liangxin1300 mentioned this pull request Jan 9, 2023

Dev: behave: Run all cases twice under normal user #1110

Merged

aleksei-burlakov deleted the nonprivileged-user branch February 9, 2023 09:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature: replace root by a custom user with root privileges #1009

Feature: replace root by a custom user with root privileges #1009

aleksei-burlakov commented Aug 16, 2022 •

edited

Loading

liangxin1300 commented Aug 22, 2022 •

edited

Loading

nicholasyang2022 commented Aug 22, 2022

liangxin1300 commented Dec 21, 2022

liangxin1300 commented Dec 21, 2022 •

edited

Loading

liangxin1300 commented Dec 21, 2022

liangxin1300 commented Dec 21, 2022 •

edited

Loading

aleksei-burlakov commented Dec 21, 2022

aleksei-burlakov commented Dec 21, 2022

aleksei-burlakov commented Dec 21, 2022

aleksei-burlakov commented Dec 21, 2022

nicholasyang2022 commented Dec 22, 2022

zzhou1 commented Dec 22, 2022 •

edited

Loading

aleksei-burlakov commented Dec 22, 2022 •

edited

Loading

nicholasyang2022 commented Dec 30, 2022 •

edited

Loading

nicholasyang2022 Dec 30, 2022

liangxin1300 Jan 5, 2023

nicholasyang2022 Jan 5, 2023

liangxin1300 Jan 5, 2023

liangxin1300 commented Jan 5, 2023

liangxin1300 commented Jan 5, 2023

Feature: replace root by a custom user with root privileges #1009

Feature: replace root by a custom user with root privileges #1009

Conversation

aleksei-burlakov commented Aug 16, 2022 • edited Loading

liangxin1300 commented Aug 22, 2022 • edited Loading

nicholasyang2022 commented Aug 22, 2022

liangxin1300 commented Dec 21, 2022

liangxin1300 commented Dec 21, 2022 • edited Loading

liangxin1300 commented Dec 21, 2022

liangxin1300 commented Dec 21, 2022 • edited Loading

aleksei-burlakov commented Dec 21, 2022

aleksei-burlakov commented Dec 21, 2022

aleksei-burlakov commented Dec 21, 2022

aleksei-burlakov commented Dec 21, 2022

nicholasyang2022 commented Dec 22, 2022

zzhou1 commented Dec 22, 2022 • edited Loading

aleksei-burlakov commented Dec 22, 2022 • edited Loading

nicholasyang2022 commented Dec 30, 2022 • edited Loading

nicholasyang2022 Dec 30, 2022

Choose a reason for hiding this comment

liangxin1300 Jan 5, 2023

Choose a reason for hiding this comment

nicholasyang2022 Jan 5, 2023

Choose a reason for hiding this comment

liangxin1300 Jan 5, 2023

Choose a reason for hiding this comment

liangxin1300 commented Jan 5, 2023

liangxin1300 commented Jan 5, 2023

aleksei-burlakov commented Aug 16, 2022 •

edited

Loading

liangxin1300 commented Aug 22, 2022 •

edited

Loading

liangxin1300 commented Dec 21, 2022 •

edited

Loading

liangxin1300 commented Dec 21, 2022 •

edited

Loading

zzhou1 commented Dec 22, 2022 •

edited

Loading

aleksei-burlakov commented Dec 22, 2022 •

edited

Loading

nicholasyang2022 commented Dec 30, 2022 •

edited

Loading