Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[discuss]: enable etcd health check #3692

Closed
tzssangglass opened this issue Feb 26, 2021 · 10 comments
Closed

[discuss]: enable etcd health check #3692

tzssangglass opened this issue Feb 26, 2021 · 10 comments

Comments

@tzssangglass
Copy link
Member

tzssangglass commented Feb 26, 2021

Issue description

I saw this issue #3673 and this pr #3676.
I think it's time to enable the etcd health check feature.
by the way, etcd's host configuration, if it is in the form of ip:port, the health check takes effect, if it is in the form of domain, the health check fails because etcd's node is hidden behind the domain and cannot be distinguished, so use the FQDN of etcd in k8s does not solve the issues #3673.

@tokers
Copy link
Contributor

tokers commented Feb 27, 2021

Yes, it just uses the FQDN since the underlying cosocket suite supports to resolve it according to the resolver directive in nginx.conf.

We may, resolving the FQDN (and cache it) by ourselves in the level of lua-resty-etcd.

@spacewander
Copy link
Member

I think we need to resolve it in the APISIX side as lua-resty-etcd knows nothing about the resolver.

@tzssangglass
Copy link
Member Author

I think we need to resolve it in the APISIX side as lua-resty-etcd knows nothing about the resolver.

agree, etcd host is meant to be each node of etcd, not a cluster, and using a k8s domain is equivalent to configuring a unique etcd cluster address, which does not match the meaning of host, so perhaps we can add an attribute to support etcd cluster address configuration

@spacewander
Copy link
Member

using a k8s domain

For this case maybe we can resolve the domain each time before retry?

@tzssangglass
Copy link
Member Author

For this case maybe we can resolve the domain each time before retry?

Maybe we can do it, but I think we can also do without. My point is: if you configure the domain, then the component responsible for domain resolution should also provide the ability to health check, and it should always return the ip of a healthy instance under that domain.

I'm not sure if this is a correct point, just a perception.

@spacewander
Copy link
Member

the component responsible for domain resolution should also provide the ability to health check

I doubt if it is a good idea to combine the health check with domain resolution, as the health check methods are various and can't detect the healthy one immediately.

@tzssangglass
Copy link
Member Author

I doubt if it is a good idea to combine the health check with domain resolution, as the health check methods are various and can't detect the healthy one immediately.

yes, you are right
only when operating etcd do we know if the node is healthy or not

@Yiyiyimu
Copy link
Member

Yiyiyimu commented Mar 12, 2021

Hi @tzssangglass are you still working on this 😀 I could take care of it if you do not got enough time

@tzssangglass
Copy link
Member Author

Hi @tzssangglass are you still working on this 😀 I could take care of it if you do not got enough time

okay, I haven't had enough time lately, so leave it to you.

@spacewander
Copy link
Member

Already done.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants