Define Tekton tasks in the devfile.

[sunix]

Is your enhancement related to a problem? Please describe.

Here it is about how we could take advantage of the fact that

• Tekton is built on k8s
• Che is built on k8s

to perform builds as close as what we have in the CI but in the inner loop (before pushing to git).

Here would be the requirements:

• We want to avoid having to transfert any files elsewhere: sources and built binaries have to stay in the workspace volumes. (maybe restricted to tasks with file/folder output resources)
• Declaring a Tekton task in the devfile should have the same syntax as a classic Tekton task.
• It should be easy to trigger a tekton task from Che Theia UI (my workspace panel or command pallet)
• In the devfile, we may want to define cpu/memory requirements for a specific task ... to be discussed

[sunix]

cc @l0rd @vdemeester WDYT ?

[l0rd]

@sunix can you provide an example of such a devfile? Are you thinking about a new command type (i.e. type: tektonTask)? How would we mount the source code in the pipeline containers?

[gattytto]

maybe this can be related? #15752

I

• Declaring a Tekton task in the devfile should have the same syntax as a classic Tekton task

this part.. won't this bring a big amount of additional data to the devfile?

[sunix]

@sunix can you provide an example of such a devfile? Are you thinking about a new command type (i.e. type: tektonTask)? How would we mount the source code in the pipeline containers?

@l0rd
Could be, I am unsure what would be the implementation. A Che command would make sense, then it means that a task could bring other type of resources (container, etc ...). To me that would be fine.

[gattytto]

for getting resources in tekton I have followed a guide with this as example:

apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: github-repo
  namespace: che
spec:
  type: git
  params:
    - name: url
      value: https://github.com/user/repo

but then if the repo needs authentication it requires this:

apiVersion: tekton.dev/v1alpha1
kind: TaskRun
metadata:
  name: pipeline-test-auth
  namespace: che
spec:
  serviceAccountName: build-bot
  taskRef:
    name: read-task
  inputs:
    resources:
      - name: github-repo
        resourceRef:
          name: github-repo

---
apiVersion: v1
kind: Secret
metadata:
  name: basic-user-pass
  namespace: che
  annotations:
    tekton.dev/git-0: https://github.com/user/repo
type: kubernetes.io/basic-auth
stringData:
  username: username
  password: pass #out in the open!!

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: build-bot
  namespace: che
secrets:
  - name: basic-user-pass

[sunix]

to me these kind of secrets/config have to be enabled in Che #14680

[gattytto]

could there be a che dashboard plugin? so to have an extra option in
to manage kubeconfigs, tekton objects and other cluster custom-resources.

workspace edit section of the dashboard could have an extra tab for adding those resources to the workspace (instead of manually editing the devfile), like the plugins section.

acces to those resources could be fine-grained using keycloak?

as of now when I open the tekton workspace I need to get into the sidecar terminal and add the ./kube/config every time the IDE restarts from inactivity (token expiry/idle timeout).

if I had the che server running in a cluster, and the tekton operator running in a separate cluster of the same network, how could I use theia to push the right kubeconfig to the tekton extension?.

[gorkem]

A Tekton Task, a ConfigMap, or a ServiceBindingRequest these are all kubernetes resources that a developer may want to apply along with the workspace. Wouldn't extending the Kubernetes component type to allow us to create any resource via devfile be a more generic solution?

To author and manage Tekton resources, we have the vscode-tekton extension that is progressing well and it needs to be made available also on Che.

Overall, we have a persistence problem with kubeconfig that is effecting openshift extension but will also effect tekton and knative extensions. I think we need to find a solution to that problem.

If you are using a devfile one thing that can make life easier is to keep your ./kube/config in the git repo and set the env KUBECONFIG to point to that file in your devfile.

[gattytto]

che-dockerfiles/che-sidecar-tekton#2 is coming! and I can see "octant" in the docker file that's good.

[gattytto]

@gorkem Overall, we have a persistence problem with kubeconfig that is effecting openshift extension but will also effect tekton and knative extensions. I think we need to find a solution to that problem.

I have come with this as a temporary workaround for the kubeconfig persistence problem, also useful for bazel cache, and now gopls works so it can work for the go modules /.cache folder too

[gattytto]

@l0rd @sunix can you provide an example of such a devfile? Are you thinking about a new command type (i.e. type: tektonTask)? How would we mount the source code in the pipeline containers?

the third part of your question I have been testing, not specifically for the projects folder but for the huge cache produced from bazel sidecar when referencing many external repos in /workspace file (http_archive refs):
within the project's build folder create something.sh with:

#!/bin/sh
kubectl delete -f ../tekton/task/bazel_build_gen.yaml -f ../tekton/task/taskrun.yml
kubectl delete -f bazelcache.yml
kubectl apply -f bazelcache.yml
kubectl apply -f ../tekton/task/bazel_build_gen.yaml
kubectl apply -f ../tekton/task/taskrun.yml
tkn taskrun logs pipeline-test-auth -f

next to it in bazelcache.yaml:

apiVersion: v1  
kind: PersistentVolume  
metadata:  
  name: bazelcache  
  labels:  
    type: local  
spec:  
  storageClassName: bazelcache  
  capacity: 
    storage: 5Gi 
  accessModes:  
    - ReadWriteMany 
  hostPath:  
    path: "/data/wksp/workspaceax6ccroy58gxem8n/bazelcache/_bazel_user" #filled from sidecar, declared in bazel plugin yaml

--- 

kind: StorageClass  
apiVersion: storage.k8s.io/v1  
metadata:  
  name: bazelcache  
  annotations:  
    storageclass.kubernetes.io/is-default-class: "false"  
provisioner: k8s.io/minikube-hostpath  
reclaimPolicy: Retain 

The task that gets executed has ways to use input and output:

apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: read-task
  namespace: che
spec:
  params:
    - name: revision
      description: 'el branch'
      type: string
      default: master
  inputs:
    resources:
      - name: github-repo
        type: git
  workspaces:
    - name: ghrepo
      mountPath: /workspace
    - name: bazelcache #this is created in bazel plugin meta.yaml 
      mountpath: /tekton/home/.cache/bazel/_bazel_1724 #this is because google's bazel runner doesn't assume rootless
  steps:
    - name: catreadme
      image: 'launcher.gcr.io/google/bazel' #'gcr.io/cloud-builders/bazel:latest'
      resources:
        limits:
          cpu: 2000m
          memory: 1Gi
        requests:
          cpu: 500m
          memory: 1Gi
      script: |
        #!/usr/bin/env bash
        cd /workspace/github-repo
        bazel run -- //:build_gen
    - name: pushgo
      image: 'launcher.gcr.io/google/bazel' 
      resources:
        limits:
          cpu: 5000m
          memory: 2Gi
        requests:
          cpu: 200m
          memory: 1Gi
      script: |
        cd /workspace/github-repo
        rm -rf bazel-out/*
        rm -rf bazel-bin/*
        bazel build //whatever
        cd bazel-bin/go
        echo "module github.com/user/repo" > go.mod
        git config --global user.email ""
        git config --global user.name ""
        git init .
        git checkout -b v1alpha1 -f
        git add .
        git commit -am "-"
        git remote add origin https://github.com/user/repo
        git push -u --force origin v1alpha1

then the taskrun has to do some stuff around the volume, this is relevant to user id used in pipeline container too:

apiVersion: tekton.dev/v1alpha1
kind: TaskRun
metadata:
  name: pipeline-test-auth
  namespace: che
spec:
  params:
    - name: revision
      value: v1alpha1
  serviceAccountName: build-bot
  podTemplate:
    securityContext:
      runAsNonRoot: true
      runAsUser: 1724
  taskRef:
    name: read-task
  workspaces:
    - name: ghrepo
      emptyDir: {}
    - name: bazelcache
      volumeClaimTemplate:
        spec:
          accessModes: 
            - ReadWriteMany
          volumeName: bazelcache
          storageClassName: bazelcache
          resources:
            requests:
              storage: 5Gi
  inputs:
    resources:
      - name: github-repo
        resourceRef:
          name: github-repo

THEN run the bash script after cding to build folder inside the tekton sidecar container shell.

[che-bot]

Issues go stale after 180 days of inactivity. lifecycle/stale issues rot after an additional 7 days of inactivity and eventually close.

Mark the issue as fresh with /remove-lifecycle stale in a new comment.

If this issue is safe to close now please do so.

Moderators: Add lifecycle/frozen label to avoid stale mode.

[vdemeester]

/remove-lifecycle stale

[che-bot]

Issues go stale after 180 days of inactivity. lifecycle/stale issues rot after an additional 7 days of inactivity and eventually close.

Mark the issue as fresh with /remove-lifecycle stale in a new comment.

If this issue is safe to close now please do so.

Moderators: Add lifecycle/frozen label to avoid stale mode.