Check all container's status when calculating revision ContainerHealthy condition #14744

seongpyoHong · 2023-12-23T08:13:06Z

Release Note

Check all container's status when calculating revision ContainerHealthy condition

knative-prow · 2023-12-23T08:13:16Z

Hi @seongpyoHong. Thanks for your PR.

I'm waiting for a knative member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

codecov · 2023-12-23T08:16:56Z

Codecov Report

Attention: Patch coverage is 80.00000% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 84.13%. Comparing base (88abc84) to head (9bc98bf).
Report is 28 commits behind head on main.

Files	Patch %	Lines
pkg/reconciler/revision/reconcile_resources.go	75.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #14744      +/-   ##
==========================================
+ Coverage   84.08%   84.13%   +0.05%     
==========================================
  Files         213      213              
  Lines       16687    16687              
==========================================
+ Hits        14031    14040       +9     
+ Misses       2303     2299       -4     
+ Partials      353      348       -5

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

dprotaso · 2024-01-12T19:12:30Z

/hold

I want to fix #14660 prior

dprotaso · 2024-02-17T04:18:15Z

Still holding for now see: #14567 (comment)

A serving point release came out two weeks ago - that might have fixed the original issue.

dprotaso

It also looks like reconciling the PodAutoscaler will mask the container failures

You'll want to update resUnavailable here to also be true if the container isn't healthy

serving/pkg/apis/serving/v1/revision_lifecycle.go

Line 173 in 5fdffba

    
           resUnavailable := rs.GetCondition(RevisionConditionResourcesAvailable).IsFalse()

dprotaso · 2024-03-08T20:03:12Z

go.mod

@@ -34,7 +34,7 @@ require (
 	k8s.io/utils v0.0.0-20230209194617-a36077c30491
 	knative.dev/caching v0.0.0-20231219125158-cb270b3a43b8
 	knative.dev/hack v0.0.0-20231201014241-7030d5bf584d
-	knative.dev/networking v0.0.0-20231218143655-3f2ee2a60c6d
+	knative.dev/networking v0.0.0-20231219131858-8c7897c8b9a0


can you drop these go.mod changes from your PR

dprotaso · 2024-03-08T20:37:01Z

pkg/reconciler/revision/reconcile_resources.go

@@ -106,7 +106,7 @@ func (c *Reconciler) reconcileDeployment(ctx context.Context, rev *v1.Revision)
 			}

 			for _, status := range pod.Status.ContainerStatuses {
-				if status.Name == rev.Spec.GetContainer().Name {
+				if status.Name != resources.QueueContainerName {


You'll need to adjust the break on line 123. Otherwise if the failing sidecar container is second in the containerStatuses list it won't fail.

I think we don't need to use break anymore, but I changed the position of break for fast fail.

dprotaso · 2024-03-08T21:15:04Z

Still holding for now see: #14567 (comment)

A serving point release came out two weeks ago - that might have fixed the original issue.

The other fix didn't address the issue this PR addresses

/hold cancel

dprotaso · 2024-03-08T21:15:21Z

/ok-to-test

seongpyoHong · 2024-03-11T14:44:39Z

@dprotaso Sorry for the late reply.

I've fixed and rebased it to reflect your feedback.

dprotaso · 2024-03-26T18:55:27Z

@seongpyoHong thanks for the changes

/lgtm
/approve

Are you able to follow up by adding a unit test or an e2e test?

knative-prow · 2024-03-26T18:55:36Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dprotaso, seongpyoHong

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [dprotaso]
~~pkg/apis/OWNERS~~ [dprotaso]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

seongpyoHong · 2024-03-27T01:57:53Z

Are you able to follow up by adding a unit test or an e2e test?

Sure, I will also add the unit test and e2e test.

knative-prow bot requested review from jsanin-vmw and skonto December 23, 2023 08:13

knative-prow bot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. area/API API objects and controllers labels Dec 23, 2023

knative-prow-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 4, 2024

knative-prow bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 12, 2024

dprotaso reviewed Mar 8, 2024

View reviewed changes

knative-prow bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 8, 2024

knative-prow bot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Mar 8, 2024

dprotaso mentioned this pull request Mar 11, 2024

Add multi-container probing #14853

Merged

seongpyoHong added 3 commits March 11, 2024 23:34

Check all container's status when calculate rev status

d4caabc

Move break

19733c3

Update resUnavailable to true when containerHealthy condition is false

9bc98bf

seongpyoHong force-pushed the handle-all-containers-status-when-calculate-rev-status branch from 4a9adae to 9bc98bf Compare March 11, 2024 14:40

knative-prow-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 11, 2024

knative-prow bot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Mar 11, 2024

knative-prow bot assigned dprotaso Mar 26, 2024

knative-prow bot added the lgtm Indicates that a PR is ready to be merged. label Mar 26, 2024

knative-prow bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 26, 2024

knative-prow bot merged commit 231db15 into knative:main Mar 26, 2024
49 checks passed

SaschaSchwarze0 mentioned this pull request Aug 30, 2024

Revisions stay in ContainerHealthy=False status forever #15487

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Check all container's status when calculating revision ContainerHealthy condition #14744

Check all container's status when calculating revision ContainerHealthy condition #14744

seongpyoHong commented Dec 23, 2023

knative-prow bot commented Dec 23, 2023

codecov bot commented Dec 23, 2023 •

edited

Loading

dprotaso commented Jan 12, 2024

dprotaso commented Feb 17, 2024

dprotaso left a comment

dprotaso Mar 8, 2024

dprotaso Mar 8, 2024

seongpyoHong Mar 12, 2024

dprotaso commented Mar 8, 2024

dprotaso commented Mar 8, 2024

seongpyoHong commented Mar 11, 2024

dprotaso commented Mar 26, 2024

knative-prow bot commented Mar 26, 2024

seongpyoHong commented Mar 27, 2024

Check all container's status when calculating revision ContainerHealthy condition #14744

Check all container's status when calculating revision ContainerHealthy condition #14744

Conversation

seongpyoHong commented Dec 23, 2023

knative-prow bot commented Dec 23, 2023

codecov bot commented Dec 23, 2023 • edited Loading

Codecov Report

dprotaso commented Jan 12, 2024

dprotaso commented Feb 17, 2024

dprotaso left a comment

Choose a reason for hiding this comment

dprotaso Mar 8, 2024

Choose a reason for hiding this comment

dprotaso Mar 8, 2024

Choose a reason for hiding this comment

seongpyoHong Mar 12, 2024

Choose a reason for hiding this comment

dprotaso commented Mar 8, 2024

dprotaso commented Mar 8, 2024

seongpyoHong commented Mar 11, 2024

dprotaso commented Mar 26, 2024

knative-prow bot commented Mar 26, 2024

seongpyoHong commented Mar 27, 2024

codecov bot commented Dec 23, 2023 •

edited

Loading