Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create workspace Error: Failed to run the workspace: "Server 'theia' in container 'theia-idelcn' not available." #16121

Closed
5 tasks done
mymynew opened this issue Feb 25, 2020 · 7 comments
Closed
5 tasks done

Create workspace Error: Failed to run the workspace: "Server 'theia' in container 'theia-idelcn' not available." #16121

mymynew opened this issue Feb 25, 2020 · 7 comments
Labels
area/install Issues related to installation, including offline/air gap and initial setup kind/bug Outline of a bug - must adhere to the bug report template.

Comments

@mymynew
Copy link

mymynew commented Feb 25, 2020 •
edited by l0rd
Loading

Describe the bug

Create workspace Error: Failed to run the workspace: "Server 'theia' in container 'theia-idelcn' not available."
Create Java Gradle workspace

Che version

7.8.0

Steps to reproduce

  1. minikube start --vm-driver=none --kubernetes-version=v1.17.0 --memory=4096
  2. set che tls
  3. chectl server:start --platform=minikube --multiuser --installer=helm --tls --self-signed-cert --cheimage=quay.io/eclipse/che-server:7.8.0
  4. create workspace

Expected behavior

start workspace .

Runtime

  • minikube v1.7.3
  • kubernetes v1.17.0
  • Docker version 19.03.6

Installation method

chectl chectl/0.0.20200219-next.3776510 linux-x64 node-v10.19.0

Environment

  • Linux Centos7.6 64
  • Cloud
    - [x] other (please specify)

Additional context

Command console output

chectl server:start --platform=minikube --installer=helm --tls --self-signed-cert --cheimage=quay.io/eclipse/che-server:7.8.0 
✔ Verify Kubernetes API...OK
  ✔ 👀  Looking for an already existing Eclipse Che instance
    ✔ Verify if Eclipse Che is deployed into namespace "che"...it is not
  ✔ ✈️  Minikube preflight checklist
    ✔ Verify if kubectl is installed
    ✔ Verify if minikube is installed
    ✔ Verify if minikube is running
    ↓ Start minikube [skipped]
      → Minikube is already running.
    ✔ Check Kubernetes version: Found v1.17.0.
    ✔ Verify if minikube ingress addon is enabled
    ✔ Enable minikube ingress addon
    ✔ Retrieving minikube IP and domain for ingress URLs...172.31.69.238.nip.io.
Eclipse Che logs will be available in '/tmp/chectl-logs/1582540735740'
  ✔ Start following logs
    ✔ Start following Eclipse Che logs...done
    ✔ Start following Postgres logs...done
    ✔ Start following Keycloak logs...done
    ✔ Start following Plugin registry logs...done
    ✔ Start following Devfile registry logs...done
  ✔ Start following events
    ✔ Start following namespace events...done
  ✔ 🏃‍  Running Helm to install Eclipse Che
    ✔ Verify if helm is installed
    ✔ Check Helm Version: Found v2.16.3+g1ee0254
    ✔ Create Namespace (che)...does already exist.
    ✔ Check for TLS secret prerequisites...self-signed-cert secret found.
    ✔ Check for self-signed certificate prerequisites...che-tls secret found.
    ✔ Create Tiller Role Binding...it already exists.
    ✔ Create Tiller Service Account...it already exists.
    ✔ Create Tiller RBAC
    ✔ Create Tiller Service...it already exists.
    ✔ Preparing Che Helm Chart...done.
    ✔ Updating Helm Chart dependencies...done.
    ✔ Deploying Che Helm Chart...done.
  ✔ ✅  Post installation checklist
    ✔ Devfile registry pod bootstrap
      ✔ scheduling...done.
      ✔ downloading images...done.
      ✔ starting...done.
    ✔ Plugin registry pod bootstrap
      ✔ scheduling...done.
      ✔ downloading images...done.
      ✔ starting...done.
    ✔ Eclipse Che pod bootstrap
      ✔ scheduling...done.
      ✔ downloading images...done.
      ✔ starting...done.
    ✔ Retrieving Eclipse Che Server URL...https://che-che.172.31.69.238.nip.io
    ✔ Eclipse Che status check
Command server:start has completed successfully.
kubectl --all-namespaces=true get pod,ing 
NAMESPACE     NAME                                                   READY   STATUS    RESTARTS   AGE
che-che       pod/workspacezd48k6pckfm56kbo.gradle-f78b557f6-ddc2g   4/4     Running   0          54s
che           pod/che-7487b77dd9-pbpzt                               1/1     Running   0          15h
che           pod/devfile-registry-d9fd7f648-ml5mb                   1/1     Running   0          15h
che           pod/plugin-registry-58587b799b-4scft                   1/1     Running   0          15h
kube-system   pod/coredns-6955765f44-7gdtz                           1/1     Running   0          22h
kube-system   pod/coredns-6955765f44-xwpd6                           1/1     Running   0          22h
kube-system   pod/etcd-che-test                                      1/1     Running   0          22h
kube-system   pod/kube-apiserver-che-test                            1/1     Running   0          22h
kube-system   pod/kube-controller-manager-che-test                   1/1     Running   0          22h
kube-system   pod/kube-proxy-zwrjm                                   1/1     Running   0          22h
kube-system   pod/kube-scheduler-che-test                            1/1     Running   0          22h
kube-system   pod/nginx-ingress-controller-6fc5bcc8c9-grdjc          1/1     Running   0          22h
kube-system   pod/storage-provisioner                                1/1     Running   0          22h
kube-system   pod/tiller-deploy-5dfbbc9559-gss8s                     1/1     Running   0          19h

NAMESPACE   NAME                                  HOSTS                                                                 ADDRESS         PORTS     AGE
che-che     ingress.extensions/ingress1chjftub    serverdcx40z96-theia-idelcn-server-3100.172.31.69.238.nip.io          172.31.69.238   80, 443   82s
che-che     ingress.extensions/ingressa7wkuhac    serverdcx40z96-theia-idelcn-server-3130.172.31.69.238.nip.io          172.31.69.238   80, 443   82s
che-che     ingress.extensions/ingressaosdtd0x    serverdcx40z96-theia-idelcn-server-13132.172.31.69.238.nip.io         172.31.69.238   80, 443   82s
che-che     ingress.extensions/ingressnrchwd6e    serverdcx40z96-theia-idelcn-server-13131.172.31.69.238.nip.io         172.31.69.238   80, 443   82s
che-che     ingress.extensions/ingressqex9pbwz    serverpxsqrrq7-che-machine-execoea-server-4444.172.31.69.238.nip.io   172.31.69.238   80, 443   82s
che-che     ingress.extensions/ingressv35vfzra    serverdcx40z96-theia-idelcn-server-13133.172.31.69.238.nip.io         172.31.69.238   80, 443   82s
che         ingress.extensions/che-ingress        che-che.172.31.69.238.nip.io                                          172.31.69.238   80, 443   15h
che         ingress.extensions/devfile-registry   devfile-registry-che.172.31.69.238.nip.io                             172.31.69.238   80, 443   15h
che         ingress.extensions/plugin-registry    plugin-registry-che.172.31.69.238.nip.io
kubectl -n che-che logs -f pod/workspacezd48k6pckfm56kbo.gradle-f78b557f6-ddc2g --all-containers=true --max-log-requests=6 
[GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached.

[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
 - using env:   export GIN_MODE=release
 - using code:  gin.SetMode(gin.ReleaseMode)

[GIN-debug] GET    /static/*filepath         --> github.com/eclipse/che-machine-exec/vendor/github.com/gin-gonic/gin.(*RouterGroup).createStaticHandler.func1 (3 handlers)
[GIN-debug] HEAD   /static/*filepath         --> github.com/eclipse/che-machine-exec/vendor/github.com/gin-gonic/gin.(*RouterGroup).createStaticHandler.func1 (3 handlers)
[GIN-debug] GET    /                         --> main.main.func1 (3 handlers)
[GIN-debug] GET    /connect                  --> main.main.func2 (3 handlers)
[GIN-debug] GET    /attach/:id               --> main.main.func3 (3 handlers)
⇩ Registered RPCRoutes:

Json-rpc MachineExec Routes:
✓ create
✓ check
✓ resize
[GIN-debug] Listening and serving HTTP on :4444
Starting the deployer with the list of resolvers [ LocalDirectoryPluginDeployerResolver {},
  GithubPluginDeployerResolver { unpackedFolder: '/tmp/github-remote' },
  HttpPluginDeployerResolver { unpackedFolder: '/tmp/http-remote' },
  VsCodePluginDeployerResolver { vscodeExtensionsFolder: '/tmp/vscode-extension-marketplace' } ]
Theia Endpoint 1/pid listening on port 2503
Found the list of default plugins ID on env: undefined
Found the list of plugins ID on env: local-dir:///plugins/sidecars/redhat_java11_latest
Found the list of default plugins ID from CLI: undefined
unzipping the VS Code extension 'redhat.java11.latest.dxjbrcwnpv.java-0.57.0-2029.vsix' to directory: /tmp/vscode-unpacked/redhat.java11.latest.dxjbrcwnpv.java-0.57.0-2029.vsix
unzipping the VS Code extension 'redhat.java11.latest.qvyspiczpy.vscode-java-debug-0.24.0.vsix' to directory: /tmp/vscode-unpacked/redhat.java11.latest.qvyspiczpy.vscode-java-debug-0.24.0.vsix
PluginTheiaDirectoryHandler: accepting plugin with path /tmp/vscode-unpacked/redhat.java11.latest.dxjbrcwnpv.java-0.57.0-2029.vsix
Resolving "redhat.java11.latest.dxjbrcwnpv.java-0.57.0-2029.vsix" as a VS Code extension...
Resolved "redhat.java11.latest.dxjbrcwnpv.java-0.57.0-2029.vsix" to a VS Code extension "java@0.57.0" with engines: { vscode: '^1.41.0' }
PluginTheiaDirectoryHandler: accepting plugin with path /tmp/vscode-unpacked/redhat.java11.latest.qvyspiczpy.vscode-java-debug-0.24.0.vsix
Resolving "redhat.java11.latest.qvyspiczpy.vscode-java-debug-0.24.0.vsix" as a VS Code extension...
Resolved "redhat.java11.latest.qvyspiczpy.vscode-java-debug-0.24.0.vsix" to a VS Code extension "vscode-java-debug@0.24.0" with engines: { vscode: '^1.38.0' }
the accepted plugins are []
the acceptedFrontendPlugins plugins are []
the acceptedBackendPlugins plugins are [ PluginDeployerEntryImpl {
    originId: 'local-dir:///plugins/sidecars/redhat_java11_latest',
    pluginId: 'redhat.java11.latest.dxjbrcwnpv.java-0.57.0-2029.vsix',
    map: Map { 'package.json' => [Object] },
    changes:
     [ 'PluginVsCodeFileHandler', 'PluginVsCodeDirectoryHandler' ],
    acceptedTypes: [ 1 ],
    currentPath:
     '/tmp/vscode-unpacked/redhat.java11.latest.dxjbrcwnpv.java-0.57.0-2029.vsix/extension',
    initPath:
     '/plugins/sidecars/redhat_java11_latest/redhat.java11.latest.dxjbrcwnpv.java-0.57.0-2029.vsix',
    resolved: true,
    resolvedByName: 'LocalDirectoryPluginDeployerResolver' },
  PluginDeployerEntryImpl {
    originId: 'local-dir:///plugins/sidecars/redhat_java11_latest',
    pluginId:
     'redhat.java11.latest.qvyspiczpy.vscode-java-debug-0.24.0.vsix',
    map: Map { 'package.json' => [Object] },
    changes:
     [ 'PluginVsCodeFileHandler', 'PluginVsCodeDirectoryHandler' ],
    acceptedTypes: [ 1 ],
    currentPath:
     '/tmp/vscode-unpacked/redhat.java11.latest.qvyspiczpy.vscode-java-debug-0.24.0.vsix/extension',
    initPath:
     '/plugins/sidecars/redhat_java11_latest/redhat.java11.latest.qvyspiczpy.vscode-java-debug-0.24.0.vsix',
    resolved: true,
    resolvedByName: 'LocalDirectoryPluginDeployerResolver' } ]
local path to deploy on remote instance [ '/tmp/vscode-unpacked/redhat.java11.latest.dxjbrcwnpv.java-0.57.0-2029.vsix/extension',
  '/tmp/vscode-unpacked/redhat.java11.latest.qvyspiczpy.vscode-java-debug-0.24.0.vsix/extension' ]
Deploying backend plugin "java@0.57.0" from "/tmp/vscode-unpacked/redhat.java11.latest.dxjbrcwnpv.java-0.57.0-2029.vsix/extension/dist/extension"
Deploying backend plugin "vscode-java-debug@0.24.0" from "/tmp/vscode-unpacked/redhat.java11.latest.qvyspiczpy.vscode-java-debug-0.24.0.vsix/extension/dist/extension"
2020/02/25 02:08:52 Broker configuration
2020/02/25 02:08:52   Push endpoint: wss://che-che.172.31.69.238.nip.io/api/websocket
2020/02/25 02:08:52   Auth enabled: false
2020/02/25 02:08:52   Runtime ID:
2020/02/25 02:08:52     Workspace: workspacezd48k6pckfm56kbo
2020/02/25 02:08:52     Environment: default
2020/02/25 02:08:52     OwnerId: che
2020/02/25 02:08:52   Self signed certificate /tmp/che/secret/ca.crt
2020/02/25 02:08:52 Starting plugin artifacts broker
2020/02/25 02:08:52 Fetching plugin meta.yaml from https://plugin-registry-che.172.31.69.238.nip.io/v3/plugins/redhat/java11/latest/meta.yaml
2020/02/25 02:08:52 Fetching plugin meta.yaml from https://plugin-registry-che.172.31.69.238.nip.io/v3/plugins/eclipse/che-machine-exec-plugin/7.8.0/meta.yaml
2020/02/25 02:08:52 Fetching plugin meta.yaml from https://plugin-registry-che.172.31.69.238.nip.io/v3/plugins/eclipse/che-theia/7.8.0/meta.yaml
2020/02/25 02:08:52 Processing plugin redhat/java11/latest
2020/02/25 02:08:52   Installing plugin extension 1/2
2020/02/25 02:08:52     Plugin already downloaded
2020/02/25 02:08:52   Installing plugin extension 2/2
2020/02/25 02:08:52     Plugin already downloaded
2020/02/25 02:08:52 Saving log of installed plugins
2020/02/25 02:08:52 All plugin artifacts have been successfully downloaded
Unable to create Che API REST Client: "CHE_WORKSPACE_TELEMETRY_BACKEND_PORT" is not set.
root WARN Failed to resolve module: @types/body-parser
root WARN Failed to resolve module: @types/express
root WARN Failed to resolve module: @types/fs-extra
root WARN Failed to resolve module: @types/lodash.debounce
root WARN Failed to resolve module: @types/lodash.throttle
root WARN Failed to resolve module: @types/react
root WARN Failed to resolve module: @types/react-dom
root WARN Failed to resolve module: @types/react-virtualized
root WARN Failed to resolve module: @types/route-parser
root WARN Failed to resolve module: @types/ws
root WARN Failed to resolve module: @types/yargs
root WARN Failed to resolve module: font-awesome
root WARN Failed to resolve module: react
root WARN Failed to resolve module: react-dom
root WARN Failed to resolve module: react-virtualized
root WARN Failed to resolve module: @types/rimraf
root WARN Failed to resolve module: @types/tar-fs
root WARN Failed to resolve module: @types/touch
root WARN Failed to resolve module: @types/uuid
root WARN Failed to resolve module: @typefox/monaco-editor-core
root WARN Failed to resolve module: @types/base64-arraybuffer
root WARN Failed to resolve module: @theia/monaco
root WARN Failed to resolve module: @types/p-debounce
root WARN Failed to resolve module: @types/mime-types
root WARN Failed to resolve module: @types/diff
root WARN Failed to resolve module: @types/connect
root WARN Failed to resolve module: @types/mime
root WARN Failed to resolve module: @types/serve-static
root WARN Failed to resolve module: @types/request
root WARN Failed to resolve module: @types/highlight.js
root WARN Failed to resolve module: @types/markdown-it
root WARN Failed to resolve module: @types/markdown-it-anchor
root INFO Theia app listening on http://0.0.0.0:3100.
root INFO unzipping the plug-in 'eclipse_che_ports_plugin.theia' to directory: /tmp/theia-unpacked/eclipse_che_ports_plugin.theia
root INFO unzipping the plug-in 'eclipse_che_telemetry_plugin.theia' to directory: /tmp/theia-unpacked/eclipse_che_telemetry_plugin.theia
root INFO unzipping the plug-in 'eclipse_che_theia_containers_plugin.theia' to directory: /tmp/theia-unpacked/eclipse_che_theia_containers_plugin.theia
root INFO unzipping the plug-in 'eclipse_che_theia_ssh_plugin.theia' to directory: /tmp/theia-unpacked/eclipse_che_theia_ssh_plugin.theia
root INFO unzipping the plug-in 'eclipse_che_welcome_plugin.theia' to directory: /tmp/theia-unpacked/eclipse_che_welcome_plugin.theia
root INFO unzipping the plug-in 'eclipse_che_workspace_plugin.theia' to directory: /tmp/theia-unpacked/eclipse_che_workspace_plugin.theia
root INFO unzipping the plug-in 'task_plugin.theia' to directory: /tmp/theia-unpacked/task_plugin.theia
root INFO unzipping the plug-in 'theia_yeoman_plugin.theia' to directory: /tmp/theia-unpacked/theia_yeoman_plugin.theia
root INFO unzipping the VS Code extension 'vscode-git-1.3.0.1.vsix' to directory: /tmp/vscode-unpacked/vscode-git-1.3.0.1.vsix
root INFO Configuring to accept webviews on 'serverdcx40z96-theia-idelcn-server-3100.172.31.69.238.nip.io' hostname.
root INFO PluginTheiaDirectoryHandler: accepting plugin with path /tmp/theia-unpacked/eclipse_che_ports_plugin.theia
root INFO PluginTheiaDirectoryHandler: accepting plugin with path /tmp/theia-unpacked/eclipse_che_telemetry_plugin.theia
root INFO PluginTheiaDirectoryHandler: accepting plugin with path /tmp/theia-unpacked/eclipse_che_theia_containers_plugin.theia
root INFO PluginTheiaDirectoryHandler: accepting plugin with path /tmp/theia-unpacked/eclipse_che_theia_ssh_plugin.theia
root INFO PluginTheiaDirectoryHandler: accepting plugin with path /tmp/theia-unpacked/eclipse_che_welcome_plugin.theia
root INFO PluginTheiaDirectoryHandler: accepting plugin with path /tmp/theia-unpacked/eclipse_che_workspace_plugin.theia
root INFO PluginTheiaDirectoryHandler: accepting plugin with path /tmp/theia-unpacked/task_plugin.theia
root INFO PluginTheiaDirectoryHandler: accepting plugin with path /tmp/theia-unpacked/theia_yeoman_plugin.theia
root INFO PluginTheiaDirectoryHandler: accepting plugin with path /tmp/vscode-unpacked/vscode-git-1.3.0.1.vsix
root INFO Resolved "vscode-git-1.3.0.1.vsix" to a VS Code extension "git@1.0.0" with engines: { vscode: '^1.5.0' }
root INFO PluginTheiaDirectoryHandler: accepting plugin with path /plugins/installed.json
root INFO PluginTheiaDirectoryHandler: accepting plugin with path /plugins/sidecars
root ERROR Failed to load plugin dependencies from '/plugins/installed.json' path Error: ENOTDIR: not a directory, open '/plugins/installed.json/package.json'
root ERROR Failed to load plugin dependencies from '/plugins/sidecars' path Error: ENOENT: no such file or directory, open '/plugins/sidecars/package.json'
root INFO Deploying backend plugin "@eclipse-che/ports-plugin@7.8.0" from "/tmp/theia-unpacked/eclipse_che_ports_plugin.theia/lib/ports-plugin.js"
root INFO Deploying backend plugin "@eclipse-che/telemetry-plugin@7.8.0" from "/tmp/theia-unpacked/eclipse_che_telemetry_plugin.theia/lib/telemetry-plugin.js"
root INFO Deploying backend plugin "@eclipse-che/theia-containers-plugin@7.8.0" from "/tmp/theia-unpacked/eclipse_che_theia_containers_plugin.theia/lib/containers-plugin.js"
root INFO Deploying backend plugin "@eclipse-che/theia-ssh-plugin@7.8.0" from "/tmp/theia-unpacked/eclipse_che_theia_ssh_plugin.theia/lib/ssh-plugin-backend.js"
root INFO Deploying backend plugin "@eclipse-che/welcome-plugin@7.8.0" from "/tmp/theia-unpacked/eclipse_che_welcome_plugin.theia/lib/welcome-plugin.js"
root INFO Deploying backend plugin "@eclipse-che/workspace-plugin@7.8.0" from "/tmp/theia-unpacked/eclipse_che_workspace_plugin.theia/lib/workspace-plugin.js"
root INFO Deploying backend plugin "task-plugin@7.8.0" from "/tmp/theia-unpacked/task_plugin.theia/lib/task-plugin-backend.js"
root INFO Deploying backend plugin "@theia/yeoman-plugin@0.0.1-1562576566" from "/tmp/theia-unpacked/theia_yeoman_plugin.theia/lib/theia-yeoman-plugin-backend-plugin.js"
root INFO Deploying backend plugin "git@1.0.0" from "/tmp/vscode-unpacked/vscode-git-1.3.0.1.vsix/extension/out/main"
kubectl -n che-che describe pod/workspacezd48k6pckfm56kbo.gradle-f78b557f6-ddc2g 
Name:         workspacezd48k6pckfm56kbo.gradle-f78b557f6-ddc2g
Namespace:    che-che
Priority:     0
Node:         che-test/172.31.69.238
Start Time:   Tue, 25 Feb 2020 10:08:45 +0800
Labels:       che.component.name=gradle
              che.deployment_name=workspacezd48k6pckfm56kbo.gradle
              che.original_name=gradle
              che.workspace_id=workspacezd48k6pckfm56kbo
              pod-template-hash=f78b557f6
Annotations:  org.eclipse.che.container.display-name/che-machine-execoea: che-machine-execoea
              org.eclipse.che.container.display-name/gradle: gradle
              org.eclipse.che.container.display-name/theia-idelcn: theia-idelcn
              org.eclipse.che.container.display-name/vscode-javawgk: vscode-javawgk
Status:       Running
IP:           172.17.0.8
IPs:
  IP:           172.17.0.8
Controlled By:  ReplicaSet/workspacezd48k6pckfm56kbo.gradle-f78b557f6
Init Containers:
  remote-runtime-injectorxt0:
    Container ID:   docker://d8a63c680ef9ca0e7abf84281ce56bdb123b5b54e635fe883945b78fcaeaab8b
    Image:          quay.io/eclipse/che-theia-endpoint-runtime-binary:7.8.0
    Image ID:       docker-pullable://quay.io/eclipse/che-theia-endpoint-runtime-binary@sha256:85da304326dce10a92674df73fee0e0b169b9f9bcc76231a11dd1811df433847
    Port:           <none>
    Host Port:      <none>
    State:          Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Tue, 25 Feb 2020 10:08:49 +0800
      Finished:     Tue, 25 Feb 2020 10:08:49 +0800
    Ready:          True
    Restart Count:  0
    Environment:
      PLUGIN_REMOTE_ENDPOINT_EXECUTABLE:  /remote-endpoint/plugin-remote-endpoint
      REMOTE_ENDPOINT_VOLUME_NAME:        remote-endpoint
    Mounts:
      /remote-endpoint from remote-endpoint (rw)
      /tmp/che/secret/ from che-self-signed-cert (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from che-workspace-token-xrfgb (ro)
  che-plugin-artifacts-broker-v3-1-0:
    Container ID:  docker://c09941a92ee89a5723372e4905c858163c3a0def2dded8fdd0969973e7ea8077
    Image:         quay.io/eclipse/che-plugin-artifacts-broker:v3.1.0
    Image ID:      docker-pullable://quay.io/eclipse/che-plugin-artifacts-broker@sha256:3498e4070fbda3a1b61cf518494ec1051112fc70daf64723dafbf0946a53af0e
    Port:          <none>
    Host Port:     <none>
    Args:
      -push-endpoint
      wss://che-che.172.31.69.238.nip.io/api/websocket
      -runtime-id
      workspacezd48k6pckfm56kbo:default:che
      -cacert
      /tmp/che/secret/ca.crt
      --registry-address
      https://plugin-registry-che.172.31.69.238.nip.io/v3
      -metas
      /broker-config/config.json
    State:          Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Tue, 25 Feb 2020 10:08:52 +0800
      Finished:     Tue, 25 Feb 2020 10:08:52 +0800
    Ready:          True
    Restart Count:  0
    Limits:
      memory:  250Mi
    Requests:
      memory:  250Mi
    Environment:
      CHE_AUTH_ENABLED:   false
      CHE_MACHINE_TOKEN:
    Mounts:
      /broker-config/ from broker-config-volumekptdhh (ro)
      /plugins from claim-che-workspace (rw,path="workspacezd48k6pckfm56kbo/plugins/")
      /tmp/che/secret/ from che-self-signed-cert (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from che-workspace-token-xrfgb (ro)
      /workspace_logs from claim-che-workspace (rw,path="workspacezd48k6pckfm56kbo/che-logs-gradle/")
Containers:
  gradle:
    Container ID:   docker://049aadcdb7ee88329a2e719490e767e4693f152b438819176c1246ea3f96f667
    Image:          quay.io/eclipse/che-java11-gradle:nightly
    Image ID:       docker-pullable://quay.io/eclipse/che-java11-gradle@sha256:88f9660549cda1da31d4bd19a83c1ba4572a406362d25d30a569587868780488
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Tue, 25 Feb 2020 10:08:54 +0800
    Ready:          True
    Restart Count:  0
    Limits:
      memory:  536870912
    Requests:
      memory:  536870912
    Environment:
      CHE_MACHINE_TOKEN:
      JAVA_OPTS:                     -XX:MaxRAMPercentage=50 -XX:+UseParallelGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -Dsun.zip.disableMemoryMapping=true -Xms20m -Djava.security.egd=file:/dev/./urandom
      CHE_PROJECTS_ROOT:             /projects
      CHE_API:                       https://che-che.172.31.69.238.nip.io/api
      CHE_WORKSPACE_NAME:            wksp-1gqp
      CHE_WORKSPACE_LOGS_ROOT__DIR:  /workspace_logs
      GRADLE_USER_HOME:              /home/gradle/.gradle
      CHE_WORKSPACE_NAMESPACE:       che
      JAVA_TOOL_OPTIONS:             -XX:MaxRAMPercentage=50 -XX:+UseParallelGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -Dsun.zip.disableMemoryMapping=true -Xms20m -Djava.security.egd=file:/dev/./urandom
      CHE_MACHINE_NAME:              gradle
      CHE_API_INTERNAL:              https://che-che.172.31.69.238.nip.io/api
      CHE_WORKSPACE_ID:              workspacezd48k6pckfm56kbo
      CHE_API_EXTERNAL:              https://che-che.172.31.69.238.nip.io/api
      HOME:                          /home/gradle
    Mounts:
      /etc/gitconfig from gitconfigvolume (rw,path="gitconfig")
      /etc/ssh/private from workspacezd48k6pckfm56kbo-sshprivatekeys (ro)
      /etc/ssh/ssh_config from ssshkeyconfigvolume (ro,path="ssh_config")
      /home/gradle/.gradle from claim-che-workspace (rw,path="workspacezd48k6pckfm56kbo/gradle/")
      /projects from claim-che-workspace (rw,path="workspacezd48k6pckfm56kbo/projects/")
      /tmp/che/secret/ from che-self-signed-cert (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from che-workspace-token-xrfgb (ro)
      /workspace_logs from claim-che-workspace (rw,path="workspacezd48k6pckfm56kbo/che-logs-gradle/")
  vscode-javawgk:
    Container ID:   docker://d8feac39ad4a3f8126852bdff616f84a5e8b1253de111d1fefc8b94116ae8a33
    Image:          quay.io/eclipse/che-sidecar-java:11-f76ca45
    Image ID:       docker-pullable://quay.io/eclipse/che-sidecar-java@sha256:54a5f721588455403d8c2a81d1fbc8fef2c6a32aead44f2fb3e24a8a1fddb9a2
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Tue, 25 Feb 2020 10:08:57 +0800
    Ready:          True
    Restart Count:  0
    Limits:
      memory:  1572864k
    Requests:
      memory:  1572864k
    Environment:
      CHE_WORKSPACE_NAME:                 wksp-1gqp
      THEIA_PLUGINS:                      local-dir:///plugins/sidecars/redhat_java11_latest
      CHE_WORKSPACE_LOGS_ROOT__DIR:       /workspace_logs
      PLUGIN_REMOTE_ENDPOINT_EXECUTABLE:  /remote-endpoint/plugin-remote-endpoint
      CHE_MACHINE_TOKEN:
      CHE_WORKSPACE_NAMESPACE:            che
      CHE_MACHINE_NAME:                   vscode-javawgk
      CHE_API_INTERNAL:                   https://che-che.172.31.69.238.nip.io/api
      CHE_WORKSPACE_ID:                   workspacezd48k6pckfm56kbo
      CHE_PROJECTS_ROOT:                  /projects
      CHE_API_EXTERNAL:                   https://che-che.172.31.69.238.nip.io/api
      CHE_API:                            https://che-che.172.31.69.238.nip.io/api
    Mounts:
      /etc/gitconfig from gitconfigvolume (rw,path="gitconfig")
      /etc/ssh/private from workspacezd48k6pckfm56kbo-sshprivatekeys (ro)
      /etc/ssh/ssh_config from ssshkeyconfigvolume (ro,path="ssh_config")
      /home/theia/.m2 from claim-che-workspace (rw,path="workspacezd48k6pckfm56kbo/m2/")
      /plugins from claim-che-workspace (rw,path="workspacezd48k6pckfm56kbo/plugins/")
      /projects from claim-che-workspace (rw,path="workspacezd48k6pckfm56kbo/projects/")
      /remote-endpoint from remote-endpoint (rw)
      /tmp/che/secret/ from che-self-signed-cert (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from che-workspace-token-xrfgb (ro)
      /workspace_logs from claim-che-workspace (rw,path="workspacezd48k6pckfm56kbo/che-logs-gradle/")
  che-machine-execoea:
    Container ID:   docker://5987f88c3c7d6aa824ad411d39df0070de0503518397816e38a25910a5842305
    Image:          quay.io/eclipse/che-machine-exec:7.8.0
    Image ID:       docker-pullable://quay.io/eclipse/che-machine-exec@sha256:5aebc022ee3a12baf06cb4f6f7350fc6531e18a6f6781775c47035ae6818567f
    Port:           4444/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Tue, 25 Feb 2020 10:09:00 +0800
    Ready:          True
    Restart Count:  0
    Limits:
      memory:  134217728
    Requests:
      memory:  134217728
    Environment:
      CHE_WORKSPACE_NAME:            wksp-1gqp
      CHE_WORKSPACE_LOGS_ROOT__DIR:  /workspace_logs
      CHE_MACHINE_TOKEN:
      CHE_WORKSPACE_NAMESPACE:       che
      CHE_MACHINE_NAME:              che-machine-execoea
      CHE_API_INTERNAL:              https://che-che.172.31.69.238.nip.io/api
      CHE_WORKSPACE_ID:              workspacezd48k6pckfm56kbo
      CHE_PROJECTS_ROOT:             /projects
      CHE_API_EXTERNAL:              https://che-che.172.31.69.238.nip.io/api
      CHE_API:                       https://che-che.172.31.69.238.nip.io/api
    Mounts:
      /etc/gitconfig from gitconfigvolume (rw,path="gitconfig")
      /etc/ssh/private from workspacezd48k6pckfm56kbo-sshprivatekeys (ro)
      /etc/ssh/ssh_config from ssshkeyconfigvolume (ro,path="ssh_config")
      /tmp/che/secret/ from che-self-signed-cert (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from che-workspace-token-xrfgb (ro)
      /workspace_logs from claim-che-workspace (rw,path="workspacezd48k6pckfm56kbo/che-logs-gradle/")
  theia-idelcn:
    Container ID:   docker://2f6bb162e7564b9e16ecf4af0748d4eb33160f90ce6c4017e2017ab079f81183
    Image:          quay.io/eclipse/che-theia:7.8.0
    Image ID:       docker-pullable://quay.io/eclipse/che-theia@sha256:3f630acb301316223c226bf6515776bf191f1e4fefe02066c8b7d112bbdb8aa8
    Ports:          3100/TCP, 3130/TCP, 13131/TCP, 13132/TCP, 13133/TCP
    Host Ports:     0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP
    State:          Running
      Started:      Tue, 25 Feb 2020 10:09:03 +0800
    Ready:          True
    Restart Count:  0
    Limits:
      memory:  512M
    Requests:
      memory:  512M
    Environment:
      THEIA_PLUGINS:                 local-dir:///plugins
      CHE_MACHINE_TOKEN:
      HOSTED_PLUGIN_PORT:            3130
      CHE_PROJECTS_ROOT:             /projects
      CHE_API:                       https://che-che.172.31.69.238.nip.io/api
      CHE_WORKSPACE_NAME:            wksp-1gqp
      HOSTED_PLUGIN_HOSTNAME:        0.0.0.0
      CHE_WORKSPACE_LOGS_ROOT__DIR:  /workspace_logs
      CHE_WORKSPACE_NAMESPACE:       che
      CHE_MACHINE_NAME:              theia-idelcn
      CHE_API_INTERNAL:              https://che-che.172.31.69.238.nip.io/api
      CHE_WORKSPACE_ID:              workspacezd48k6pckfm56kbo
      CHE_API_EXTERNAL:              https://che-che.172.31.69.238.nip.io/api
    Mounts:
      /etc/gitconfig from gitconfigvolume (rw,path="gitconfig")
      /etc/ssh/private from workspacezd48k6pckfm56kbo-sshprivatekeys (ro)
      /etc/ssh/ssh_config from ssshkeyconfigvolume (ro,path="ssh_config")
      /plugins from claim-che-workspace (rw,path="workspacezd48k6pckfm56kbo/plugins/")
      /projects from claim-che-workspace (rw,path="workspacezd48k6pckfm56kbo/projects/")
      /tmp/che/secret/ from che-self-signed-cert (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from che-workspace-token-xrfgb (ro)
      /workspace_logs from claim-che-workspace (rw,path="workspacezd48k6pckfm56kbo/che-logs-gradle/")
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  remote-endpoint:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:  <unset>
  broker-config-volumekptdhh:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      workspacezd48k6pckfm56kbo.broker-config-mapt0vcd3
    Optional:  false
  claim-che-workspace:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  claim-che-workspace
    ReadOnly:   false
  che-self-signed-cert:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  workspacezd48k6pckfm56kbo-che-self-signed-cert
    Optional:    false
  workspacezd48k6pckfm56kbo-sshprivatekeys:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  workspacezd48k6pckfm56kbo-sshprivatekeys
    Optional:    false
  ssshkeyconfigvolume:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      workspacezd48k6pckfm56kbo-sshconfigmap
    Optional:  false
  gitconfigvolume:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      workspacezd48k6pckfm56kbo-gitconfig
    Optional:  false
  che-workspace-token-xrfgb:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  che-workspace-token-xrfgb
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  70s   default-scheduler  Successfully assigned che-che/workspacezd48k6pckfm56kbo.gradle-f78b557f6-ddc2g to che-test
  Normal  Pulling    69s   kubelet, che-test  Pulling image "quay.io/eclipse/che-theia-endpoint-runtime-binary:7.8.0"
  Normal  Pulled     67s   kubelet, che-test  Successfully pulled image "quay.io/eclipse/che-theia-endpoint-runtime-binary:7.8.0"
  Normal  Created    67s   kubelet, che-test  Created container remote-runtime-injectorxt0
  Normal  Started    66s   kubelet, che-test  Started container remote-runtime-injectorxt0
  Normal  Pulling    66s   kubelet, che-test  Pulling image "quay.io/eclipse/che-plugin-artifacts-broker:v3.1.0"
  Normal  Pulled     63s   kubelet, che-test  Successfully pulled image "quay.io/eclipse/che-plugin-artifacts-broker:v3.1.0"
  Normal  Created    63s   kubelet, che-test  Created container che-plugin-artifacts-broker-v3-1-0
  Normal  Started    63s   kubelet, che-test  Started container che-plugin-artifacts-broker-v3-1-0
  Normal  Pulled     62s   kubelet, che-test  Container image "quay.io/eclipse/che-java11-gradle:nightly" already present on machine
  Normal  Created    61s   kubelet, che-test  Created container gradle
  Normal  Started    61s   kubelet, che-test  Started container gradle
  Normal  Pulling    61s   kubelet, che-test  Pulling image "quay.io/eclipse/che-sidecar-java:11-f76ca45"
  Normal  Pulled     59s   kubelet, che-test  Successfully pulled image "quay.io/eclipse/che-sidecar-java:11-f76ca45"
  Normal  Created    58s   kubelet, che-test  Created container vscode-javawgk
  Normal  Started    58s   kubelet, che-test  Started container vscode-javawgk
  Normal  Pulling    58s   kubelet, che-test  Pulling image "quay.io/eclipse/che-machine-exec:7.8.0"
  Normal  Pulled     55s   kubelet, che-test  Successfully pulled image "quay.io/eclipse/che-machine-exec:7.8.0"
  Normal  Created    55s   kubelet, che-test  Created container che-machine-execoea
  Normal  Started    55s   kubelet, che-test  Started container che-machine-execoea
  Normal  Pulling    55s   kubelet, che-test  Pulling image "quay.io/eclipse/che-theia:7.8.0"
  Normal  Pulled     52s   kubelet, che-test  Successfully pulled image "quay.io/eclipse/che-theia:7.8.0"
  Normal  Created    52s   kubelet, che-test  Created container theia-idelcn
  Normal  Started    52s   kubelet, che-test  Started container theia-idelcn
@mymynew mymynew added the kind/bug Outline of a bug - must adhere to the bug report template. label Feb 25, 2020
@che-bot che-bot added the status/need-triage An issue that needs to be prioritized by the curator responsible for the triage. See https://github. label Feb 25, 2020
@tolusha
Copy link
Contributor

tolusha commented Feb 25, 2020

I am wondering about step #2
Could you compare your step with #14035 (comment)

@mymynew
Copy link
Author

mymynew commented Feb 25, 2020

@tolusha
Same operation ,
Reference doc https://www.eclipse.org/che/docs/che-7/setup-che-in-tls-mode-with-self-signed-certificate/
This my test server, you can do anything:
ip: 47.91.158.95
account/password: root/Che#$8899
ca.crt in /home/yzgwork/che_ca，hosts file add "47.91.158.95 che-che.172.31.69.238.nip.io
47.91.158.95 keycloak-che.172.31.69.238.nip.io" can be access che-server

Anyone help me ? please

@l0rd l0rd added status/info-needed More information is needed before the issue can move into the “analyzing” state for engineering. and removed status/need-triage An issue that needs to be prioritized by the curator responsible for the triage. See https://github. labels Feb 25, 2020
@l0rd
Copy link
Contributor

l0rd commented Feb 25, 2020

The workspace pod starts successfully. But for some reason the workspaces loader is not able to reach che-theia. That may be related to the fact that instead of using the FQDN of the remote host you are using a nip.io address. @akurinnoy @olexii4 any idea of what's going on here?

@mymynew you can try a few of things:

@sleshchenko
Copy link
Member

That error message actually came from Che Server#server checkers.
It's Che Server who was not able to request Theia ingress.

@mymynew Are you able to run workpace again, and while it's starting - find theia ingress and try to curl it from Che Server pod? If IP resolved? If no - then probably network inside pods are not configured properly.
@AndrienkoAleksandr You had some experience on running minikube locally(--vm-driver=none ), have you faced the same issue?

@AndrienkoAleksandr
Copy link
Contributor

@AndrienkoAleksandr You had some experience on running minikube locally(--vm-driver=none ), have you faced the same issue?>

Che was working for me.

@mymynew
Copy link
Author

mymynew commented Feb 26, 2020

@sleshchenko theia container listen port 3100，but k8s ingress not create 3100 ingress

theia container use service name access ?
curl svc.namespace:3100 is ok， like this curl server9zh0eeu2-theia-ide8ne.yzg-che:3100 or curl podip:3100

kubectl get ingress,svc,pod --all-namespaces 
NAMESPACE   NAME                                  HOSTS                                                           ADDRESS         PORTS     AGE
che         ingress.extensions/che-ingress        che-che.172.31.69.238.nip.io                                    172.31.69.238   80, 443   45m
che         ingress.extensions/devfile-registry   devfile-registry-che.172.31.69.238.nip.io                       172.31.69.238   80, 443   45m
che         ingress.extensions/keycloak-ingress   keycloak-che.172.31.69.238.nip.io                               172.31.69.238   80, 443   45m
che         ingress.extensions/plugin-registry    plugin-registry-che.172.31.69.238.nip.io                        172.31.69.238   80, 443   45m
yzg-che     ingress.extensions/ingress0zvfuy2i    serverjlzu8wss-theia-ide4gm-server-13131.172.31.69.238.nip.io   172.31.69.238   80, 443   2m30s
yzg-che     ingress.extensions/ingress3i68lk2f    serverk7o4871d-jwtproxy-server-4401.172.31.69.238.nip.io        172.31.69.238   80, 443   2m30s
yzg-che     ingress.extensions/ingressit1d515j    serverk7o4871d-jwtproxy-server-4400.172.31.69.238.nip.io        172.31.69.238   80, 443   2m30s
yzg-che     ingress.extensions/ingressnokjrtqq    serverjlzu8wss-theia-ide4gm-server-13132.172.31.69.238.nip.io   172.31.69.238   80, 443   2m30s
yzg-che     ingress.extensions/ingresso7fjpryt    serverjlzu8wss-theia-ide4gm-server-3130.172.31.69.238.nip.io    172.31.69.238   80, 443   2m30s
yzg-che     ingress.extensions/ingresstnmbyqig    serverjlzu8wss-theia-ide4gm-server-13133.172.31.69.238.nip.io   172.31.69.238   80, 443   2m30s

NAMESPACE     NAME                                         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                                           AGE
che           service/che-host                             ClusterIP   10.96.3.173     <none>        8080/TCP,8087/TCP                                 45m
che           service/devfile-registry                     ClusterIP   10.96.189.72    <none>        8080/TCP                                          45m
che           service/keycloak                             ClusterIP   10.96.79.71     <none>        5050/TCP                                          45m
che           service/plugin-registry                      ClusterIP   10.96.36.66     <none>        8080/TCP                                          45m
che           service/postgres                             ClusterIP   10.96.10.255    <none>        5432/TCP                                          45m
default       service/kubernetes                           ClusterIP   10.96.0.1       <none>        443/TCP                                           53m
kube-system   service/kube-dns                             ClusterIP   10.96.0.10      <none>        53/UDP,53/TCP,9153/TCP                            53m
kube-system   service/tiller-deploy                        ClusterIP   10.96.32.123    <none>        44134/TCP                                         46m
yzg-che       service/serverfxaj37n4-che-machine-exectzx   ClusterIP   10.96.169.0     <none>        4444/TCP                                          2m31s
yzg-che       service/serverjlzu8wss-theia-ide4gm          ClusterIP   10.96.87.55     <none>        3100/TCP,3130/TCP,13133/TCP,13132/TCP,13131/TCP   2m31s
yzg-che       service/serverk7o4871d-jwtproxy              ClusterIP   10.96.144.145   <none>        4400/TCP,4401/TCP                                 2m31s

NAMESPACE     NAME                                                          READY   STATUS    RESTARTS   AGE
che           pod/che-699bddc97d-mch2c                                      1/1     Running   0          45m
che           pod/devfile-registry-d9fd7f648-r74hk                          1/1     Running   0          45m
che           pod/keycloak-5875d484d6-6fs2t                                 1/1     Running   0          45m
che           pod/plugin-registry-58587b799b-dfhdp                          1/1     Running   0          45m
che           pod/postgres-77469cbb7-rph8x                                  1/1     Running   0          45m
default       pod/dns-client-7bfdddc88f-9zcx4                               1/1     Running   0          32m
kube-system   pod/coredns-6955765f44-hrt56                                  1/1     Running   0          53m
kube-system   pod/coredns-6955765f44-n4mjv                                  1/1     Running   0          53m
kube-system   pod/etcd-che-test                                             1/1     Running   0          53m
kube-system   pod/kube-apiserver-che-test                                   1/1     Running   0          53m
kube-system   pod/kube-controller-manager-che-test                          1/1     Running   0          53m
kube-system   pod/kube-proxy-hb2fz                                          1/1     Running   0          53m
kube-system   pod/kube-scheduler-che-test                                   1/1     Running   0          53m
kube-system   pod/nginx-ingress-controller-6fc5bcc8c9-xvscn                 1/1     Running   0          49m
kube-system   pod/storage-provisioner                                       1/1     Running   0          53m
kube-system   pod/tiller-deploy-5b4685ffbf-28l69                            1/1     Running   0          46m
yzg-che       pod/workspace1689yvr3bafm030v.che-jwtproxy-5d6d95d964-ztp5v   1/1     Running   0          2m6s
yzg-che       pod/workspace1689yvr3bafm030v.gradle-9f89769c9-8vxs5          4/4     Running   0          2m6s

@mymynew
Copy link
Author

mymynew commented Feb 27, 2020 •
edited
Loading

use chectl server:start --platform=minikube --multiuser --installer=helm --domain serverip.nip.io
liak this set: -b 47.91.158.95.nip.io start successful.
thank you everybody.

@mymynew mymynew closed this as completed Feb 27, 2020
@tolusha tolusha added area/install Issues related to installation, including offline/air gap and initial setup and removed status/info-needed More information is needed before the issue can move into the “analyzing” state for engineering. labels Feb 27, 2020
@tolusha tolusha mentioned this issue Feb 27, 2020
Closed
@monaka monaka mentioned this issue Sep 14, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/install Issues related to installation, including offline/air gap and initial setup kind/bug Outline of a bug - must adhere to the bug report template.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@l0rd @tolusha @sleshchenko @AndrienkoAleksandr @mymynew @che-bot