🐛 fix: improve load balancer health checks

[mnaser]

What this PR does / why we need it:

This lower the delay between checks on the health
monitors which should reduce the recovery time
during an upgrade for example.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):

Fixes #1374

Special notes for your reviewer:

1. Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.

TODOs:

• squashed commits
• if necessary:
  • includes documentation
  • adds unit tests

/hold

[netlify]

✅ Deploy Preview for kubernetes-sigs-cluster-api-openstack ready!

Name	Link
🔨 Latest commit	77ac0cf
🔍 Latest deploy log	https://app.netlify.com/sites/kubernetes-sigs-cluster-api-openstack/deploys/6367f1c60b9cf90008330b69
😎 Deploy Preview	https://deploy-preview-1375--kubernetes-sigs-cluster-api-openstack.netlify.app/print
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site settings.

[k8s-ci-robot]

Welcome @mnaser!

It looks like this is your first PR to kubernetes-sigs/cluster-api-provider-openstack 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/cluster-api-provider-openstack has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

[k8s-ci-robot]

Hi @mnaser. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[tobiasgiese]

/ok-to-test

[jichenjc]

thanks for the detailed info in the issue, it's very helpful info

/lgtm

[mdbooth]

There's a fair bit to unpack here. Firstly I wonder if this should be configurable with good defaults, but lets leave that aside for now.

Secondly we're doing a TCP check rather than hitting the heackcheck endpoint, which seems suboptimal.

Lastly, here's a really good internal OpenShift doc for platform implementors with justified recommendations: https://github.com/openshift/installer/blob/master/docs/dev/kube-apiserver-health-check.md

Approving this because it's clearly better than what's currently there, but I think there's more to do here.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mdbooth, mnaser

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [mdbooth]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[mnaser]

There's a fair bit to unpack here. Firstly I wonder if this should be configurable with good defaults, but lets leave that aside for now.

Agreed

Secondly we're doing a TCP check rather than hitting the heackcheck endpoint, which seems suboptimal.

I was going to propose making the change to HTTPS (see issue) however OVN provider for Octavia doesn't support it, so I figured that would be a bit hard for the RH side of things :)

Lastly, here's a really good internal OpenShift doc for platform implementors with justified recommendations: https://github.com/openshift/installer/blob/master/docs/dev/kube-apiserver-health-check.md

That's a really neat document.

Approving this because it's clearly better than what's currently there, but I think there's more to do here.

[mnaser]

/hold cancel

[mnaser]

/cherry-pick release-0.6

[k8s-infra-cherrypick-robot]

@mnaser: only kubernetes-sigs org members may request cherry picks. You can still do the cherry-pick manually.

In response to this:

/cherry-pick release-0.6

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.