fix: pod with a restart policy of Never or OnFailure stuck at 'Progressing' (#15317)

[RoelofKuijpers]

This implementation extends the health condition check for pods.
Previously the assumption was that Pods with restart policy of Never or OnFailure are hooks with a finite life, these were considered as Progressing instead of Healthy. However, this logic does not apply when the pod is managed by an operator (e.g., Flink operator) and therefore has a restart policy of Never.
We introduce a new annotation which existence is checked when the pod is Running, that allows for skipping this logic on restart policy.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 53.68%. Comparing base (8849c3f) to head (61ddfd1).
Report is 41 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #709      +/-   ##
==========================================
- Coverage   54.26%   53.68%   -0.58%     
==========================================
  Files          64       64              
  Lines        6164     6480     +316     
==========================================
+ Hits         3345     3479     +134     
- Misses       2549     2725     +176     
- Partials      270      276       +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[drewhemm]

This looks like a good approach to the problem.

[drewhemm]

The pod manifest needs the following to pass the tests:

• Compute and storage resources defined
• The alpine tag needs to use something other than latest, e.g. 3.21
• Add automountServiceAccountToken: false to the pod spec, as per the Kubernetes docs

[RoelofKuijpers]

@drewhemm I have made the changes you suggested to get a Quality Gate pass

[drewhemm]

Cool, looks like the last blocking issue is the commit sign off.

[drewhemm]

A non-blocking issue has been flagged by SonarQube, probably best to resolve it as follows:

resources:
  requests:
    ephemeral-storage: "100Mi"

[sivchari]

LGTM. left nits.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud