Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: adjust startup, liveness and readiness probes settings #50

Merged

Conversation

rm3l
Copy link
Member

@rm3l rm3l commented Dec 13, 2024

Description of the change

Startup probe settings seem to have been added in the upstream Backstage Chart
in [1], but the current settings do not allow the RHDH Chart for the
liveness probe to be triggered sufficiently enough for the app to be
considered live.
This adjust such settings by accounting for the worst case scenario
where the application might take a bit long to start.

This also aligns the probe endpoints with the upstream chart.

[1] backstage/charts#216

Existing or Associated Issue(s)

Additional Information

Checklist

  • Chart version bumped in Chart.yaml according to semver.
  • Variables are documented in the values.yaml and added to the README.md. The pre-commit utility can be used to generate the necessary content. Use pre-commit run -a to apply changes.
  • JSON Schema template updated and re-generated the raw schema via pre-commit hook.
  • List tests pass for Chart using the Chart Testing tool and the ct lint command.

rm3l added 2 commits December 13, 2024 11:26
Startup probe settings seem to have been added in the upstream Backstage Chart
in [1], but the current settings do not allow the RHDH Chart for the
liveness probe to be triggered sufficiently enough for the app to be
considered live.
This adjust such settings by accounting for the worst case scenario
where the application might take a bit long to start.

This also aligns the probe endpoints with the upstream chart.

[1] backstage/charts#216
@rm3l
Copy link
Member Author

rm3l commented Dec 13, 2024

/uncc @kadel
/cc @PatAKnight @gustavolira

@rm3l
Copy link
Member Author

rm3l commented Dec 13, 2024

/cherry-pick release-1.4

@openshift-cherrypick-robot

@rm3l: once the present PR merges, I will cherry-pick it on top of release-1.4 in a new PR and assign it to you.

In response to this:

/cherry-pick release-1.4

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@kadel
Copy link
Member

kadel commented Dec 13, 2024

This fixes the problem that we have, but why not just remove the probe definitions from the RHDH sub-chart and rely on the inherited values from the upstream RHDH chart? I've tested them and it should work

charts/backstage/values.yaml Outdated Show resolved Hide resolved
Co-authored-by: Patrick Knight <pknight@redhat.com>
Co-authored-by: Gustavo Lira e Silva <guga.java@gmail.com>
@nickboldt
Copy link
Member

nickboldt commented Dec 13, 2024

when cherrypicking this to the release-1.4 branch we can safely update that chart from 2.23.0 to 2.23.1 as the 2.24.0 version was created when we branched for the 1.4.0 release.

so the same patch has to be applied to https://github.com/redhat-developer/rhdh-chart/blob/release-1.4/charts/backstage/Chart.yaml 2.23.0 -> 2.23.1

https://artifacthub.io/packages/helm/rhdh-chart/backstage

@rm3l
Copy link
Member Author

rm3l commented Dec 13, 2024

This fixes the problem that we have, but why not just remove the probe definitions from the RHDH sub-chart and rely on the inherited values from the upstream RHDH chart? I've tested them and it should work

That was exactly the issue. The upstream chart defines the probes (including the startup probe) here, but the default thresholds and periods do not seem sufficient to cover the RHDH application.
Also note that this was also not always reproducible. On some clusters, it worked, but in some others, the startup probe didn't and the pod got restarted until it failed.

…liveness and readiness probes

Co-authored-by: Nick Boldt <nboldt@redhat.com>
@nickboldt nickboldt merged commit f866790 into redhat-developer:main Dec 13, 2024
5 of 6 checks passed
@openshift-cherrypick-robot

@rm3l: #50 failed to apply on top of branch "release-1.4":

Applying: Adjust startup, liveness and readiness probes settings
Applying: Update Chart version
Using index info to reconstruct a base tree...
M	charts/backstage/Chart.yaml
M	charts/backstage/README.md
Falling back to patching base and 3-way merge...
Auto-merging charts/backstage/README.md
CONFLICT (content): Merge conflict in charts/backstage/README.md
Auto-merging charts/backstage/Chart.yaml
CONFLICT (content): Merge conflict in charts/backstage/Chart.yaml
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config advice.mergeConflict false"
Patch failed at 0002 Update Chart version

In response to this:

/cherry-pick release-1.4

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@rm3l rm3l deleted the RHIDP-5106--fix-startup-probe-failures branch December 13, 2024 15:45
rm3l added a commit to rm3l/rhdh-chart that referenced this pull request Dec 13, 2024
…eveloper#50)

* Adjust startup, liveness and readiness probes settings

Startup probe settings seem to have been added in the upstream Backstage Chart
in [1], but the current settings do not allow the RHDH Chart for the
liveness probe to be triggered sufficiently enough for the app to be
considered live.
This adjust such settings by accounting for the worst case scenario
where the application might take a bit long to start.

This also aligns the probe endpoints with the upstream chart.

[1] backstage/charts#216

* Update Chart version

* Update charts/backstage/values.yaml

Co-authored-by: Patrick Knight <pknight@redhat.com>
Co-authored-by: Gustavo Lira e Silva <guga.java@gmail.com>

* Add comment explaining why there is no 'initialDelaySeconds' on both liveness and readiness probes

Co-authored-by: Nick Boldt <nboldt@redhat.com>

---------

Co-authored-by: Patrick Knight <pknight@redhat.com>
Co-authored-by: Gustavo Lira e Silva <guga.java@gmail.com>
Co-authored-by: Nick Boldt <nboldt@redhat.com>
nickboldt pushed a commit that referenced this pull request Dec 13, 2024
* Adjust startup, liveness and readiness probes settings

Startup probe settings seem to have been added in the upstream Backstage Chart
in [1], but the current settings do not allow the RHDH Chart for the
liveness probe to be triggered sufficiently enough for the app to be
considered live.
This adjust such settings by accounting for the worst case scenario
where the application might take a bit long to start.

This also aligns the probe endpoints with the upstream chart.

[1] backstage/charts#216

* Update Chart version

* Update charts/backstage/values.yaml

Co-authored-by: Patrick Knight <pknight@redhat.com>
Co-authored-by: Gustavo Lira e Silva <guga.java@gmail.com>

* Add comment explaining why there is no 'initialDelaySeconds' on both liveness and readiness probes

Co-authored-by: Nick Boldt <nboldt@redhat.com>

---------

Co-authored-by: Patrick Knight <pknight@redhat.com>
Co-authored-by: Gustavo Lira e Silva <guga.java@gmail.com>
Co-authored-by: Nick Boldt <nboldt@redhat.com>

fix URLs and bump to 2.24.2 in 1.4 branch

Signed-off-by: RHDH Build (rhdh-bot) <rhdh-bot@redhat.com>
rm3l added a commit that referenced this pull request Dec 13, 2024
* Adjust startup, liveness and readiness probes settings

Startup probe settings seem to have been added in the upstream Backstage Chart
in [1], but the current settings do not allow the RHDH Chart for the
liveness probe to be triggered sufficiently enough for the app to be
considered live.
This adjust such settings by accounting for the worst case scenario
where the application might take a bit long to start.

This also aligns the probe endpoints with the upstream chart.

[1] backstage/charts#216

* Update Chart version

* Update charts/backstage/values.yaml




* Add comment explaining why there is no 'initialDelaySeconds' on both liveness and readiness probes



---------





fix URLs and bump to 2.24.2 in 1.4 branch

Signed-off-by: RHDH Build (rhdh-bot) <rhdh-bot@redhat.com>
Co-authored-by: Armel Soro <armel@rm3l.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants