Watch and provide startup logs (workspace + broker pods)

[sparkoo]

This effectively replaces and enhances the original PR #15988

What does this PR do?

Watch the logs of all containers in workspace pod and plugin broker pod during workspace startup. Feature is disabled by default. It can be enabled on workspace startup REST API with new query parameter debug-workspace-start. If it is set to true, logs will be watched and printed during the startup.

so API call looks like this:
<che-host>/api/workspace/{id}/runtime?debug-workspace-start=true

What issues does this PR fix or reference?

#15983
#16117

Docs PR

will be part of followup UI-based tasks #16050 #16063 #16119

Additional context

epic: #15047

[che-bot]

E2E tests of Eclipse Che Multiuser on OCP has failed:

• Build details
• Test report

Use command [ci-test] to rerun the test.

[che-bot]

✅ E2E Happy path tests succeed 🎉

See Details

• Jenkins job

• logs and configs

• Happy path tests DevFile

• image: maxura/che-server:16126

Tested with Eclipse Che Multiuser User on K8S (minikube v1.1.1)

[che-bot]

E2E tests of Eclipse Che Multiuser on OCP has failed:

• Build details
• Test report

Use command [ci-test] to rerun the test.

[amisevsk]

* More informational than a question: I've noticed that the infrastructure for che.openshift.io sometimes randomly closes watches shortly after they're opened, so we should keep this in mind if we need to debug this feature on che.openshift.io.

Thanks, I didn't know about this. Do we have some metrics? Maybe good idea to measure it here. Do we know why it's happening?

I have no idea why it happens, but had to work around something like this in the kubernetes image puller. It may even be a k8s client-go issue, so we might not see it on the Che side.

[che-bot]

✅ E2E Happy path tests succeed 🎉

See Details

• Jenkins job

• logs and configs

• Happy path tests DevFile

• image: maxura/che-server:16126

Tested with Eclipse Che Multiuser User on K8S (minikube v1.1.1)

[che-bot]

✅ E2E Happy path tests succeed 🎉

See Details

• Jenkins job

• logs and configs

• Happy path tests DevFile

• image: maxura/che-server:16126

Tested with Eclipse Che Multiuser User on K8S (minikube v1.1.1)

[che-bot]

E2E tests of Eclipse Che Multiuser on OCP has failed:

• Build details
• Test report

Use command [ci-test] to rerun the test.

[che-bot]

E2E tests of Eclipse Che Multiuser on OCP has failed:

• Build details
• Test report

Use command [ci-test] to rerun the test.

[che-bot]

✅ E2E Happy path tests succeed 🎉

See Details

• Jenkins job

• logs and configs

• Happy path tests DevFile

• image: maxura/che-server:16126

Tested with Eclipse Che Multiuser User on K8S (minikube v1.1.1)

[che-bot]

E2E tests of Eclipse Che Multiuser on OCP has failed:

• Build details
• Test report

Use command [ci-test] to rerun the test.

[dmytro-ndp]

@sparkoo: java selenium tests failed because of known test flakiness in master.

[che-bot]

E2E tests of Eclipse Che Multiuser on OCP has failed:

• Build details
• Test report

Use command [ci-test] to rerun the test.

[che-bot]

❌ E2E Happy path tests failed ❗

See Details

• Jenkins job

• test report

• logs and configs

• Happy path tests DevFile

• image: maxura/che-server:16126

Tested with Eclipse Che Multiuser User on K8S (minikube v1.1.1)

⚠️ https://github.com/orgs/eclipse/teams/eclipse-che-qa please check this report.

ℹ️ Use comment "crw-ci-test" to rerun happy path E2E test.