Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chain activation: avoid misfiring #601

Merged
merged 2 commits into from
Sep 12, 2023
Merged

chain activation: avoid misfiring #601

merged 2 commits into from
Sep 12, 2023

Conversation

nicolasochem
Copy link
Collaborator

@nicolasochem nicolasochem commented Sep 11, 2023

when activate job gets a successful RPC response for the first time, it should not repeat the query; instead, just parse the output again and check whether the level is zero.

This way, if the second rpc query fails, it will just restart the job instead of passing silently, resulting in a non-activated chain stuck at level 0, as happened on Mondaynet today.

I tried:

  • private chain on minikube: I see the until loop run a few times, and then the chain gets activated
  • delete the activation job and run helm upgrade: job runs again, but is satisfied that activation already happened and exits silently

when activate job gets a successful RPC response for the first time, it
should not repeat the query; instead, just parse the output again and
check whether the level is zero.

This way, if the second rpc query fails, it will just restart the job
instead of passing silently, resulting in a non-activated chain stuck at
level 0, as happened on Mondaynet today.
Copy link
Collaborator

@harryttd harryttd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

closes #600

@nicolasochem nicolasochem merged commit 5ec120a into master Sep 12, 2023
20 checks passed
@nicolasochem nicolasochem deleted the activate-misfired branch September 12, 2023 02:20
nicolasochem added a commit that referenced this pull request Sep 12, 2023
* chain activation: avoid misfiring

when activate job gets a successful RPC response for the first time, it
should not repeat the query; instead, just parse the output again and
check whether the level is zero.

This way, if the second rpc query fails, it will just restart the job
instead of passing silently, resulting in a non-activated chain stuck at
level 0, as happened on Mondaynet today.

* fix helm tests
@harryttd harryttd linked an issue Sep 12, 2023 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

pre-existing activation detection misfired on Mondaynet
2 participants