Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Che-operator pod is getting OOMkilled on GKE #20529

Closed
coutantal opened this issue Sep 23, 2021 · 5 comments
Closed

Che-operator pod is getting OOMkilled on GKE #20529

coutantal opened this issue Sep 23, 2021 · 5 comments
Labels
area/che-operator Issues and PRs related to Eclipse Che Kubernetes Operator kind/bug Outline of a bug - must adhere to the bug report template. severity/P2 Has a minor but important impact to the usage or development of the system.
Milestone

Comments

@coutantal
Copy link

Describe the bug

che-operator pod is getting OOMKilled when installed with chectl on GKE.

NAME                           READY   STATUS      RESTARTS   AGE
che-operator-9fc88fc85-hzrlj   0/1     OOMKilled   0          42s

Che version

7.36@latest

Steps to reproduce

  1. Install eclipse-che on GKE with chectl
chectl server:deploy --platform=k8s --domain=************

Expected behavior

che-operator should boots fine to let chectl continue with the installation

Runtime

Kubernetes (vanilla)

Screenshots

No response

Installation method

chectl/latest

Environment

Linux

Eclipse Che Logs

2021-09-23T16:17:02.095Z Cause: Error: Timeout: there are no pods in the namespace: eclipse-che, selector: app=che,component=postgres. Check Eclipse Che logs for details. Consider increasing error recheck timeout with --k8spoderrorrechecktimeout flag.
2021-09-23T16:17:02.095Z     at KubeTasks.<anonymous> (~/.local/share/chectl/client/7.36.1/lib/tasks/kube.js:155:23)
2021-09-23T16:17:02.095Z     at Generator.next (<anonymous>)
2021-09-23T16:17:02.095Z     at fulfilled (~/.local/share/chectl/client/7.36.1/node_modules/tslib/tslib.js:114:62)
2021-09-23T16:17:02.095Z     at runMicrotasks (<anonymous>)

Additional context

Here's the resources definition of che-operator's deployment :

        resources:
          limits:
            cpu: 500m
            memory: 256Mi
          requests:
            cpu: 100m
            memory: 64Mi

Actually I can get around the issue by modifying these limits values.

@coutantal coutantal added the kind/bug Outline of a bug - must adhere to the bug report template. label Sep 23, 2021
@che-bot che-bot added the status/need-triage An issue that needs to be prioritized by the curator responsible for the triage. See https://github. label Sep 23, 2021
@dmytro-ndp dmytro-ndp added area/chectl Issues related to chectl, the CLI of Che team/deploy severity/P1 Has a major impact to usage or development of the system. and removed status/need-triage An issue that needs to be prioritized by the curator responsible for the triage. See https://github. labels Sep 23, 2021
@tolusha tolusha added area/che-operator Issues and PRs related to Eclipse Che Kubernetes Operator severity/P2 Has a minor but important impact to the usage or development of the system. and removed area/chectl Issues related to chectl, the CLI of Che team/deploy severity/P1 Has a major impact to usage or development of the system. labels Sep 24, 2021
@tolusha
Copy link
Contributor

tolusha commented Sep 24, 2021

@coutantal
Could you provide working limits?

@coutantal
Copy link
Author

@tolusha what do you mean by working limits ?

I have no namespace or cluster-wide limitation for pod's resources.

@tolusha
Copy link
Contributor

tolusha commented Oct 21, 2021

We were able to reproduce the issue and working on it
#20647

@tolusha tolusha closed this as completed Nov 29, 2021
@tolusha tolusha added this to the 7.40 milestone Nov 29, 2021
@coutantal
Copy link
Author

Thanks for you work !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/che-operator Issues and PRs related to Eclipse Che Kubernetes Operator kind/bug Outline of a bug - must adhere to the bug report template. severity/P2 Has a minor but important impact to the usage or development of the system.
Projects
None yet
Development

No branches or pull requests

4 participants