Can't start workspace (Forbidden!Configured service account doesn't have access. Service account may have been revoked. pods "mkdir-workspace9i8bnzy7uvrtp60c" is forbidden: User "system:serviceaccount:eclipse-che:che" cannot get resource "pods/log" in API group "" in the namespace "mtsmfm-che".)

[mtsmfm]

Describe the bug

For some reason I met this error.

Failure executing: GET at: https://172.20.0.1/api/v1/namespaces/mtsmfm-che/pods/mkdir-workspace9i8bnzy7uvrtp60c/log?pretty=false. Message: Forbidden!Configured service account doesn't have access. Service account may have been revoked. pods "mkdir-workspace9i8bnzy7uvrtp60c" is forbidden: User "system:serviceaccount:eclipse-che:che" cannot get resource "pods/log" in API group "" in the namespace "mtsmfm-che".

Restarting workspace doesn't help.
Removing namespace can solve this problem but my persistent volumes will be lost.

Che version

7.37

Steps to reproduce

1. Start workspace
2. Stop workspace
3. (Something is needed but I'm not sure the exact step to reproduce)
4. Start workspace

Expected behavior

Workspace starts successfully

Runtime

Kubernetes (vanilla)

Screenshots

No response

Installation method

other (please specify in additional context)

Environment

Amazon

Eclipse Che Logs

No response

Additional context

Runtime is Amazon EKS.

We install che via argocd.
We use che-operator's manifests.

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: eclipse-che
resources:
  - https://raw.githubusercontent.com/eclipse-che/che-operator/7.37.0/config/crd/bases/org_v1_che_crd.yaml
  - https://raw.githubusercontent.com/eclipse-che/che-operator/7.37.0/config/crd/bases/org.eclipse.che_chebackupserverconfigurations_crd.yaml
  - https://raw.githubusercontent.com/eclipse-che/che-operator/7.37.0/config/crd/bases/org.eclipse.che_checlusterbackups_crd.yaml
  - https://raw.githubusercontent.com/eclipse-che/che-operator/7.37.0/config/crd/bases/org.eclipse.che_checlusterrestores_crd.yaml
  - https://raw.githubusercontent.com/eclipse-che/che-operator/7.37.0/config/rbac/cluster_role.yaml
  - https://raw.githubusercontent.com/eclipse-che/che-operator/7.37.0/config/rbac/cluster_rolebinding.yaml
  - https://raw.githubusercontent.com/eclipse-che/che-operator/7.37.0/config/manager/manager.yaml
  - https://raw.githubusercontent.com/eclipse-che/che-operator/7.37.0/config/rbac/role.yaml
  - https://raw.githubusercontent.com/eclipse-che/che-operator/7.37.0/config/rbac/role_binding.yaml
  - https://raw.githubusercontent.com/eclipse-che/che-operator/7.37.0/config/rbac/service_account.yaml

[tolusha]

I don't see if operator delegates this role to che service account.

[1] https://github.com/eclipse-che/che-operator/blob/main/controllers/che/workspace_namespace_permission.go#L251-L255

[tolusha]

I think we have to delegate edit cluster roles like was done here
https://github.com/eclipse-che/che-operator/blob/5c6d066f183f740be6075380026e5adc1339739d/pkg/controller/che/workspace_namespace_permission.go#L159-L168

[tolusha]

Might be related
#20396