Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tiup deleted the ssh public key before tiup cluster prune task sucess. #1546

Closed
imoyakin opened this issue Sep 2, 2021 · 2 comments · Fixed by #1552
Closed

tiup deleted the ssh public key before tiup cluster prune task sucess. #1546

imoyakin opened this issue Sep 2, 2021 · 2 comments · Fixed by #1552
Assignees
Labels
category/stability Categorizes issue or PR as a stability enhancement. component/cluster Issues about the tiup-cluster component type/bug Categorizes issue as related to a bug.
Milestone

Comments

@imoyakin
Copy link

imoyakin commented Sep 2, 2021

Bug Report

Please answer these questions before submitting your issue. Thanks!

  1. What did you do?
    tiup cluster prune

  2. What did you expect to see?

don't delete ssh-key until all mission success.

  1. What did you see instead?
Stopping component blackbox_exporter
        Stopping instance  *.*.*.6
Failed to stop blackbox_exporter-9115.service: Unit blackbox_exporter-9115.service not loaded.

Failed to stop blackbox_exporter-9115.service: Unit blackbox_exporter-9115.service not loaded.

        Stop *.*.*.6 success
Destroying monitored  *.*.*.6
        Destroying instance  *.*.*.6
Destroy monitored on  *.*.*.6 success
Delete public key  *.*.*.6
Delete public key  *.*.*.6 success
Stopping component tikv
        Stopping instance  *.*.*.7

Error: failed to destroy tombstone: failed to stop tikv: failed to stop:  *.*.*.7 tikv-20160.service, please check the instance's log(/

try it again

afater

Error: failed to destroy tombstone: failed to stop tikv: failed to stop: *.*.*.6 tikv-20160.service, please check the instance's log(/tidb-deploy/tikv-20160/log) for more detail.: executor.ssh.execute_failed: Failed to execute command over SSH for 'tidb@*.*.*.6:22' {ssh_stderr: , ssh_stdout: , ssh_command: export LANG=C; PATH=$PATH:/usr/bin:/usr/sbin /usr/bin/sudo -H bash -c "systemctl daemon-reload && systemctl stop tikv-20160.service"}, cause: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

  1. What version of TiUP are you using (tiup --version)?
    1.5.5
@imoyakin imoyakin added the type/bug Categorizes issue as related to a bug. label Sep 2, 2021
@AstroProfundis
Copy link
Contributor

The SSH key will be deleted when the last instance on host is destroyed. When you have unreachable host (e.g., broken hardware or already deleted VMs) in your cluster and you are deleting it, you could try the --force argument.

The --force argument ignores any errors occurs during the deleting process, so only use it when some of the hosts are already unreachable, but not when some operation like transferring data timeouts.

@AstroProfundis
Copy link
Contributor

Oh, my fault, the --force argument is not available for prune subcommand, I'll fix that.

@AstroProfundis AstroProfundis self-assigned this Sep 6, 2021
@AstroProfundis AstroProfundis added this to the v1.6.0 milestone Sep 6, 2021
@AstroProfundis AstroProfundis added category/stability Categorizes issue or PR as a stability enhancement. component/cluster Issues about the tiup-cluster component labels Sep 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category/stability Categorizes issue or PR as a stability enhancement. component/cluster Issues about the tiup-cluster component type/bug Categorizes issue as related to a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants