Getting Started Samples are failing with 'user is not permitted use the container-build SCC'

[ibuziuk]

Describe the bug

• Install Eclipse Che 7.56.0 via OperatorHub
• Create default CR (container build is disabled by default)
• Start a Getting Started sample
• ERROR: Failed to start the workspace spring-petclinic, reason: Failed to update workspace 'spring-petclinic'. Unable to patch devworkspace: admission webhook "mutate.devworkspace-controller.svc" denied the request: user is not permitted use the container-build SCC

Che version

7.56@latest

Steps to reproduce

See description

Expected behavior

Workspace started without problems

Runtime

OpenShift

Screenshots

Installation method

OperatorHub

Environment

Linux

Eclipse Che Logs

No response

Additional context

Reproducible against OpenShift 4.10.36 RHPDS cluster

Release notes text

A regression has been introduced in Eclipse Che 7.56 so that when installing Che on OpenShift using operator hub, workspaces started from Getting Started section of User Dashboard would fail to start with the error 'user is not permitted use the container-build SCC'.

[amisevsk]

ERROR: Failed to start the workspace spring-petclinic, reason: Failed to update workspace 'spring-petclinic'. Unable to patch devworkspace: admission webhook "mutate.devworkspace-controller.svc" denied the request: user is not permitted use the container-build SCC

Occurs if a workspace is created (/updated to have) the controller.devfile.io/scc attribute but the user performing the action does not have permissions to use the specified SCC. Is the DevWorkspace being created with that attribute?

If container build is disabled, DevWorkspaces should not get the controller.devfile.io/scc attribute.

[ibuziuk]

@amisevsk here is how the resulting DW yaml looks like. Basically, there is no controller.devfile.io/scc attribute, which is expected:

apiVersion: workspace.devfile.io/v1alpha2
kind: DevWorkspace
metadata:
  annotations:
    che.eclipse.org/che-editor: che-incubator/che-code/insiders
    che.eclipse.org/devfile-source: |
      factory:
        params: >-
          devWorkspace=https://eclipse-che.apps.cluster-g79ql.g79ql.sandbox2866.opentlc.com/devfile-registry/devfiles/quarkus/devworkspace-che-code-insiders.yaml&storageType=per-user&url=https://github.com/che-samples/quarkus-quickstarts/tree/devfilev2
    che.eclipse.org/last-updated-timestamp: '2022-11-21T12:23:07.386Z'
  resourceVersion: '60585'
  name: quarkus-quickstart
  uid: 7f1ee779-d3e7-4243-9b09-829bea37cea8
  creationTimestamp: '2022-11-21T12:23:07Z'
  generation: 3
  managedFields:
    - apiVersion: workspace.devfile.io/v1alpha2
      fieldsType: FieldsV1
      fieldsV1:
        'f:status':
          'f:conditions': {}
          'f:message': {}
          'f:phase': {}
      manager: devworkspace-controller
      operation: Update
      time: '2022-11-21T12:23:07Z'
    - apiVersion: workspace.devfile.io/v1alpha2
      fieldsType: FieldsV1
      fieldsV1:
        'f:metadata':
          'f:annotations':
            .: {}
            'f:che.eclipse.org/che-editor': {}
            'f:che.eclipse.org/devfile-source': {}
            'f:che.eclipse.org/last-updated-timestamp': {}
        'f:spec':
          .: {}
          'f:routingClass': {}
          'f:started': {}
          'f:template':
            .: {}
            'f:attributes':
              .: {}
              'f:controller.devfile.io/devworkspace-config':
                .: {}
                'f:name': {}
                'f:namespace': {}
              'f:controller.devfile.io/storage-type': {}
            'f:commands': {}
            'f:components': {}
            'f:projects': {}
      manager: unknown
      operation: Update
      time: '2022-11-21T12:23:08Z'
  namespace: user1-che
  labels:
    controller.devfile.io/creator: 950dc6ab-c6b5-4cb6-b2eb-7c6db2741d32
spec:
  routingClass: che
  started: false
  template:
    attributes:
      controller.devfile.io/devworkspace-config:
        name: devworkspace-config
        namespace: openshift-operators
      controller.devfile.io/storage-type: per-user
    commands:
      - exec:
          commandLine: mvn package
          component: tools
          group:
            isDefault: true
            kind: build
          workingDir: '${PROJECT_SOURCE}/getting-started'
        id: package
      - exec:
          commandLine: >-
            mvn package -Dnative -Dmaven.test.skip
            -Dquarkus.native.native-image-xmx=2G
          component: tools
          group:
            kind: build
          label: Package Native
          workingDir: '${PROJECT_SOURCE}/getting-started'
        id: package-native
      - exec:
          commandLine: 'mvn compile quarkus:dev'
          component: tools
          group:
            isDefault: true
            kind: run
          label: Start Development mode (Hot reload + debug)
          workingDir: '${PROJECT_SOURCE}/getting-started'
        id: start-dev
      - exec:
          commandLine: ./getting-started-1.0.0-SNAPSHOT-runner
          component: ubi-minimal
          group:
            kind: run
          label: Start Native
          workingDir: '${PROJECT_SOURCE}/getting-started/target'
        id: start-native
    components:
      - attributes:
          che-code.eclipse.org/contribute-endpoint/code-redirect-1: 13131
          che-code.eclipse.org/contribute-memoryLimit: true
          che-code.eclipse.org/contribute-endpoint/code-redirect-2: 13132
          che-code.eclipse.org/contribute-cpuRequest: true
          che-code.eclipse.org/contribute-endpoint/code-redirect-3: 13133
          che-code.eclipse.org/original-memoryLimit: 4.7G
          che-code.eclipse.org/contributed-container: tools
          che-code.eclipse.org/contribute-cpuLimit: true
          che-code.eclipse.org/contribute-memoryRequest: true
          che-code.eclipse.org/contribute-endpoint/che-code: 3100
          che-code.eclipse.org/contribute-entry-point: true
          che-code.eclipse.org/contribute-volume-mount/checode: /checode
        container:
          cpuRequest: 30m
          command:
            - /checode/entrypoint-volume.sh
          env:
            - name: QUARKUS_HTTP_HOST
              value: 0.0.0.0
            - name: CHE_DASHBOARD_URL
              value: >-
                https://eclipse-che.apps.cluster-g79ql.g79ql.sandbox2866.opentlc.com
            - name: CHE_PLUGIN_REGISTRY_URL
              value: >-
                https://eclipse-che.apps.cluster-g79ql.g79ql.sandbox2866.opentlc.com/plugin-registry/v3
            - name: CHE_PLUGIN_REGISTRY_INTERNAL_URL
              value: 'http://plugin-registry.openshift-operators.svc:8080/v3'
            - name: OPENVSX_REGISTRY_URL
              value: 'https://open-vsx.org'
          memoryRequest: 256Mi
          sourceMapping: /projects
          cpuLimit: 500m
          volumeMounts:
            - name: m2
              path: /home/user/.m2
            - name: checode
              path: /checode
          memoryLimit: 5.77G
          image: 'quay.io/devfile/universal-developer-image:ubi8-3055e6d'
          endpoints:
            - exposure: none
              name: debug
              protocol: tcp
              targetPort: 5005
            - exposure: public
              name: hello-greeting-endpoint
              path: /hello/greeting/che-user
              protocol: http
              targetPort: 8080
            - attributes:
                contributed-by: che-code.eclipse.org
                cookiesAuthEnabled: true
                discoverable: false
                type: main
                urlRewriteSupported: true
              exposure: public
              name: che-code
              path: '?tkn=eclipse-che'
              protocol: https
              secure: false
              targetPort: 3100
            - attributes:
                contributed-by: che-code.eclipse.org
                discoverable: false
                urlRewriteSupported: true
              exposure: public
              name: code-redirect-1
              protocol: http
              targetPort: 13131
            - attributes:
                contributed-by: che-code.eclipse.org
                discoverable: false
                urlRewriteSupported: true
              exposure: public
              name: code-redirect-2
              protocol: http
              targetPort: 13132
            - attributes:
                contributed-by: che-code.eclipse.org
                discoverable: false
                urlRewriteSupported: true
              exposure: public
              name: code-redirect-3
              protocol: http
              targetPort: 13133
          mountSources: true
        name: tools
      - container:
          args:
            - '-f'
            - /dev/null
          command:
            - tail
          env:
            - name: CHE_DASHBOARD_URL
              value: >-
                https://eclipse-che.apps.cluster-g79ql.g79ql.sandbox2866.opentlc.com
            - name: CHE_PLUGIN_REGISTRY_URL
              value: >-
                https://eclipse-che.apps.cluster-g79ql.g79ql.sandbox2866.opentlc.com/plugin-registry/v3
            - name: CHE_PLUGIN_REGISTRY_INTERNAL_URL
              value: 'http://plugin-registry.openshift-operators.svc:8080/v3'
            - name: OPENVSX_REGISTRY_URL
              value: 'https://open-vsx.org'
          image: registry.access.redhat.com/ubi8/ubi-minimal
          memoryLimit: 64M
          mountSources: true
          sourceMapping: /projects
        name: ubi-minimal
      - name: m2
        volume:
          size: 1G
      - name: che-code-quarkus-quickstart
        plugin:
          kubernetes:
            name: che-code-quarkus-quickstart
    projects:
      - name: quarkus-quickstarts
        zip:
          location: >-
            http://devfile-registry.openshift-operators.svc:8080/resources/v2/quarkus-quickstarts.zip
status:
  conditions:
    - lastTransitionTime: '2022-11-21T12:23:07Z'
      message: Workspace is stopped
      status: 'False'
      type: Started
  devworkspaceId: workspace7f1ee779d3e74243
  message: Stopped
  phase: Stopped

However, the workspace is failing to start. DWOC created by the operator is the following

apiVersion: controller.devfile.io/v1alpha1
config: {}
kind: DevWorkspaceOperatorConfig
metadata:
  creationTimestamp: '2022-11-21T12:16:14Z'
  generation: 1
  managedFields:
    - apiVersion: controller.devfile.io/v1alpha1
      fieldsType: FieldsV1
      fieldsV1:
        'f:config': {}
        'f:metadata':
          'f:ownerReferences':
            .: {}
            'k:{"uid":"5ff91f17-658d-434d-b8ed-1abd9d010a2b"}': {}
      manager: manager
      operation: Update
      time: '2022-11-21T12:16:14Z'
  name: devworkspace-config
  namespace: openshift-operators
  ownerReferences:
    - apiVersion: org.eclipse.che/v2
      blockOwnerDeletion: true
      controller: true
      kind: CheCluster
      name: eclipse-che
      uid: 5ff91f17-658d-434d-b8ed-1abd9d010a2b
  resourceVersion: '56889'
  uid: e29b7323-9459-453a-ac7f-fdfd59dc0f64

Eclipse Che CR looks the following way:

 devEnvironments:
    containerBuildConfiguration:
      openShiftSecurityContextConstraint: container-build
    defaultComponents:
      - container:
          image: 'quay.io/devfile/universal-developer-image:ubi8-38da5c2'
          sourceMapping: /projects
        name: universal-developer-image
    defaultEditor: che-incubator/che-code/insiders
    defaultNamespace:
      autoProvision: true
      template: <username>-che
    disableContainerBuildCapabilities: true

The suspicious thing is that DWO version installed with 7.56.0 is 0.16.0+0.1666668361.p downstream. @tolusha is it expected?

[amisevsk]

@ibuziuk The message in the screenshot suggests that Che is attempting to patch the DevWorkspace to include the SCC -- we don't see it in the DevWorkspace because DWO blocks applying the patch. Does the DevWorkspace there start if you manually set .spec.started to true?

It seems like what's happening when the dashboard is attempting to start the workspace is that it tries to apply a patch that

1. Sets .spec.started=true
2. Adds the SCC attribute for some reason

The second point there causes the patch to be rejected, and so the workspace fails to start. The workspace never even attempts to start, as indicated by the status not having a failed condition.

[ibuziuk]

looks like there might be a problem with dashboard PR to 7.56.x has been opened eclipse-che/che-dashboard#676 and we are waiting for the final verification from QA
@nickboldt once merged we need to trigger the respin for 3.3

[ibuziuk]

eclipse-che/che-dashboard#676 has been merged. Closing
@nickboldt please trigger the 3.3 build

[l0rd]

This issue is related to downstream https://issues.redhat.com/browse/CRW-3554

[devstudio-release]

sync'd to Red Hat JIRA https://issues.redhat.com/browse/CRW-3561