-
Notifications
You must be signed in to change notification settings - Fork 26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
make readiness probe optional #397
Conversation
For Teztnets, it's useful to disable this probe: when I start a new testnet, sometimes I am waiting for other participants to join for the chain to kick off. When recreating dailynet or mondaynet, I need to wait for other peers to connect to me to download the most recent blocks: with a probe, that's not possible. Must be explicitly set to false, otherwise by default the probe will be present. It can only be set at node type level. Since it's processed by helm and not config-generator, I didn't see a way to make it configurable at global or instance level.
It is good to add this option. Just a few questions:
If it doesn't have blocks and so is on 0, doesn't it return success?
Does this mean that the chain already exists and has blocks? So basically blocks stopped being produced and therefore the chain gets older and probe returns false?
Can set option in those places and then mount tezos-config configMap? Then probe looks at NODE_GLOBALS and it's own node options. Can return success msg saying why skipping validation. |
Yes, but sometimes there are hiccups, for example 10 blocks go through and then nothing. Also on tenderbake, when there is no quorum, the node keeps emitting blocks at the same level but different "rounds", but only when it has rights to do so. In mondaynet for example, this results in the liveness probe "blinking". Bottom, line, I just want to remove it for the teztnets platform.
Yes, correct. And when the probe returns false, other nodes won't be able to connect to it. If other nodes know of more recent blocks, they won't be able to upload them to our node.
True. It's a different approach. In my current approach, I simply remove the sidecar and the readiness configuration. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍 👍
@@ -203,7 +203,7 @@ jobs: | |||
- name: Set up Helm | |||
uses: azure/setup-helm@v1 | |||
with: | |||
version: v3.5.3 | |||
version: v3.8.1 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is in one of my pull requests as well. I'll take it out, if we approve this one first.
For Teztnets, it's useful to disable this probe: when I start a new
testnet, sometimes I am waiting for other participants to join for the
chain to kick off.
When recreating dailynet or mondaynet, I need to wait for other peers to
connect to me to download the most recent blocks: with a probe, that's
not possible.
Must be explicitly set to false, otherwise by default the probe will be
present. It can only be set at node type level. Since it's processed by
helm and not config-generator, I didn't see a way to make it
configurable at global or instance level.
I will need this to migrate teztnets to the new aws org without disruption.