Unexpected 'unhealthy cluster' when remove a downed member

[zzzzwc]

What happened?

I have a 3 members etcd cluster, and then stop one of the members.
Then I call MemberRemove through client to remove the downed member, the api return 'etcdserver: unhealthy cluster'.

What did you expect to happen?

MemberRemove should success because there are still 2 members active.

How can we reproduce it (as minimally and precisely as possible)?

Make a 3 members etcd cluster stop one of them and then call MemberRemove, remove the stopped etcd.

Anything else we need to know?

No response

Etcd version (please run commands below)

$ etcd --version
etcd Version: 3.5.5
Git SHA: 19002cfc6
Go Version: go1.16.15
Go OS/Arch: linux/amd64

$ etcdctl version
etcdctl version: 3.5.5
API version: 3.5

Etcd configuration (command line flags or environment variables)

paste your configuration here

Etcd debug information (please run commands below, feel free to obfuscate the IP address or FQDN in the output)

$ etcdctl member list -w table
# paste output here

$ etcdctl --endpoints=<member list> endpoint status -w table
# paste output here

Relevant log output

No response

[serathius]

Interesting, I would expect that this should work assuming that you are removing a downed member. We should be able to maintain quorum in both old configuration (2/3) and new configuration (2/2).

cc @chaochn47 as we are planning to add more tests in #15708

[chaochn47]

Thanks @zzzzwc for the report, do you have a repro and server logging? That will be very helpful.

[zzzzwc]

Thanks @zzzzwc for the report, do you have a repro and server logging? That will be very helpful.

Through logs I found I made a mistake: I actually tried to remove an up member, and then I try to remove a down member,the API success as expected....., sorry for taking your time @serathius @chaochn47