Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[crmsh-4.6] Dev: bootstrap: implement ssh-agent support (jsc#PED-5774) #1261

Merged
merged 14 commits into from
Nov 23, 2023

Conversation

nicholasyang2022
Copy link
Collaborator

Use Cases

In a typical cloud-based deployment, a server may have password-based
authentication disabled for ssh, and an adminstrator's ssh public key
is added to authorized_keys during initialization.

For this case, it is impossible for crmsh to log into cluster node with
interactive authentication and create new key pairs for further
operations. Instead, crmsh can make use of the administrator's key, by
authenticating ssh session with ssh-agent forwarded form the
adminstrator's PC.

Usage Example

alice@alice-pc: ~> ssh -A root@node1
root@node1:~ # crm cluster init --use-ssh-agent -y
root@node1:~ # exit
alice@alice-pc: ~> ssh -A root@node2
root@node2:~ # crm cluster join --use-ssh-agent -c node1 -y

@nicholasyang2022 nicholasyang2022 requested review from liangxin1300 and removed request for liangxin1300 October 11, 2023 08:46
@nicholasyang2022 nicholasyang2022 force-pushed the ssh-agent branch 7 times, most recently from 1cc801b to a984f5b Compare October 19, 2023 02:44
@nicholasyang2022 nicholasyang2022 force-pushed the ssh-agent branch 4 times, most recently from c05d44e to 462d1b8 Compare October 23, 2023 03:07
@codecov
Copy link

codecov bot commented Oct 23, 2023

Codecov Report

Attention: 99 lines in your changes are missing coverage. Please review.

Comparison is base (373ddb9) 52.52% compared to head (6a49af9) 53.06%.

Files Patch % Lines
crmsh/bootstrap.py 78.99% 46 Missing ⚠️
crmsh/ssh_key.py 68.90% 37 Missing ⚠️
crmsh/sh.py 83.60% 10 Missing ⚠️
crmsh/report/collect.py 25.00% 3 Missing ⚠️
crmsh/report/utillib.py 0.00% 1 Missing ⚠️
crmsh/ui_cluster.py 83.33% 1 Missing ⚠️
crmsh/utils.py 91.66% 1 Missing ⚠️
Additional details and impacted files
@@              Coverage Diff              @@
##           crmsh-4.6    #1261      +/-   ##
=============================================
+ Coverage      52.52%   53.06%   +0.53%     
=============================================
  Files             83       83              
  Lines          25354    25617     +263     
=============================================
+ Hits           13317    13593     +276     
+ Misses         12037    12024      -13     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@liangxin1300 liangxin1300 changed the title Dev: bootstrap: implement ssh-agent support (jsc#PED-5774) [crmsh-4.6] Dev: bootstrap: implement ssh-agent support (jsc#PED-5774) Oct 23, 2023
@nicholasyang2022 nicholasyang2022 force-pushed the ssh-agent branch 9 times, most recently from 7890220 to d151367 Compare November 1, 2023 04:18
@nicholasyang2022 nicholasyang2022 marked this pull request as ready for review November 1, 2023 04:48
@liangxin1300
Copy link
Collaborator

liangxin1300 commented Nov 2, 2023

After using ssh-agent to setup a cluster, some commands will fail without adding SSH_AUTH_SOCK:

  • crm cluster run "cmd"
  • crm report
  • crm cluster stop --all
    Can crmsh be aware this cluster had been set up via ssh-agent? So that crmsh can give a warning to hint use how to do?

How about check config.core.no_generating_ssh_key?
@nicholasyang2022

@nicholasyang2022
Copy link
Collaborator Author

Can crmsh be aware this cluster had been set up via ssh-agent? So that crmsh can give a warning to hint use how to do?

@liangxin1300 ClusterShell has the ability to catch an ssh call with 255 exit code and raise an AuthorizationError exception. It is easy to add diagnose messages for that exception.

However, the old get_stdout_or_raise_error method does not handle 255 specially. To avoid breaking things, I made ClusterShell to raise AuthorizationError only when requested explicitly.

I will try enabling that exception by default.

@nicholasyang2022 nicholasyang2022 force-pushed the ssh-agent branch 5 times, most recently from 544abcd to af4e45f Compare November 20, 2023 03:12
@liangxin1300
Copy link
Collaborator

liangxin1300 commented Nov 20, 2023

Compare with current crmsh-4.6 code:

# crm cluster init -y

INFO: SSH key for root does not exist, hence generate it now
INFO: SSH key for hacluster does not exist, hence generate it now

Above 2 lines are missing while not using ssh-agent;
And

INFO: SSH key for hacluster does not exist, hence generate it now

Above line is missing while using ssh-agent (but hacluster's key still be generated locally? I mean don't we consider hacluster also using ssh-agent? @zzhou1 )

@liangxin1300
Copy link
Collaborator

SSH_AUTH_SOCK=/root/ssh-auth-sock crm report will hang, please check

And, it's better to add crm report case under ssh-agent scenario

Works now, thanks!

@zzhou1
Copy link
Contributor

zzhou1 commented Nov 21, 2023

Compare with current crmsh-4.6 code:

# crm cluster init -y

...

INFO: SSH key for hacluster does not exist, hence generate it now

Above line is missing while using ssh-agent (but hacluster's key still be generated locally? I mean don't we consider hacluster also using ssh-agent? @zzhou1 )

Agree, "hacluster" ssh-key should be inside the cluster nodes since hawk might call for it.

crmsh/sh.py Outdated
if 'SSH_AUTH_SOCK' not in os.environ:
buf.write('Environment variable SSH_AUTH_SOCK does not exist.')
if 'SUDO_USER' in os.environ:
buf.write(' Please make sure environment variables are preserved across sudo calls.')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
buf.write(' Please make sure environment variables are preserved across sudo calls.')
buf.write(' Probably could consider, eg. sudo --preserve-env=SSH_AUTH_SOCK .')

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lacking --preserve-env=SSH_AUTH_SOCK is only one of the possible causes of SSH_AUTH_SOCK not found in environment variables. How about Please check whether an ssh-agent is available and consider using "sudo --preserve-env=SSH_AUTH_SOCK"

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check whether ssh-agent is available and consider using "sudo --preserve-env=SSH_AUTH_SOCK"

Use Cases
=========

In a typical cloud-based deployment, a server may have password-based
authentication disabled for ssh, and an adminstrator's ssh public key
is added to authorized_keys during initialization.

For this case, it is impossible for crmsh to log into cluster node with
interactive authentication and create new key pairs for further
operations. Instead, crmsh can make use of the administrator's key, by
authenticating ssh session with ssh-agent forwarded form the
adminstrator's PC.

Usage Example
=============

```sh
alice@alice-pc: ~> ssh -A root@node1
root@node1:~ # crm cluster init --use-ssh-agent -y
root@node1:~ # exit
alice@alice-pc: ~> ssh -A root@node2
root@node2:~ # crm cluster join --use-ssh-agent -c node1 -y
```
Newly implemented KeyFileManager and AuthorizedKeyManager allow faster
key swap for user `hacluster`.
@nicholasyang2022 nicholasyang2022 force-pushed the ssh-agent branch 2 times, most recently from 59bd4b3 to 2e6a15e Compare November 23, 2023 07:55
@liangxin1300 liangxin1300 merged commit 50296e9 into ClusterLabs:crmsh-4.6 Nov 23, 2023
30 checks passed
@nicholasyang2022 nicholasyang2022 deleted the ssh-agent branch November 24, 2023 08:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants