Cherry pick KFP multi user changes to 1.1 branch (#1383)

* feat: KFP multi user mode PR1 - enable multi user mode without istio authorization (#1342)

* Add argo to stacks/generic

* Pull pipelines manifest from upstream

* Updated kfp

* Minio v3 manifests

* Rename minio configmap

* Add generic minio install

* Generate new test data

* Mysql kustomize v3 manifest - generic install

* Add mysql gcp pd install

* Generate test data

* Pipelines kustomize v3 manifests

* Add kfp ui virtual service

* Add metadata deployment to stacks/generic

* Use common cluster domain

* Deploy metadata writer

* Add kfp cache server

* Update test data

* Enable KFP multi user mode without istio security

* Fix persistence agent watch namespace

* Fix namespace env for some deployments

* Fix cluster roles and bindings

* fix rename

* Fix pipelines ui role

* Updated kfp to rc2

* simplify pipeline v3 manifest using updated kfp rc2 manifest

* Fix pipeline-install-config

* remove redundant configmap

* update tests

* updated to kfp 1.0.0-rc.3

* Adapt to kfp 1.0rc3 refactoring

* update test snapshots

* fix pull kfp script to detect empty dir

* fix example ref

* update snapshot

* fix gcp pd manifest

* Update stacks ref

* revert alice example to gcp stack

* update snapshot

* fix profile controller iam binding

* Update kfp profile controller can be configured to different images and
istio sidecar

* add missing viewer controller cluster roles

* Use python3 for sync.py

* Revert gcp stack back to use non multi user kfp

* revert unintended changes

* revert upstream changes

* Use kubeflow userid header and prefix config for KFP servers (#1365)

* feat: KFP multi user mode PR2 - secure KFP with istio mTLS and authz (#1368)

* Add argo to stacks/generic

* Pull pipelines manifest from upstream

* Updated kfp

* Minio v3 manifests

* Rename minio configmap

* Add generic minio install

* Generate new test data

* Mysql kustomize v3 manifest - generic install

* Add mysql gcp pd install

* Generate test data

* Pipelines kustomize v3 manifests

* Add kfp ui virtual service

* Add metadata deployment to stacks/generic

* Use common cluster domain

* Deploy metadata writer

* Add kfp cache server

* Update test data

* Enable KFP multi user mode without istio security

* Fix persistence agent watch namespace

* Fix namespace env for some deployments

* Fix cluster roles and bindings

* fix rename

* Fix pipelines ui role

* Updated kfp to rc2

* simplify pipeline v3 manifest using updated kfp rc2 manifest

* Fix pipeline-install-config

* remove redundant configmap

* update tests

* updated to kfp 1.0.0-rc.3

* Adapt to kfp 1.0rc3 refactoring

* update test snapshots

* fix pull kfp script to detect empty dir

* fix example ref

* update snapshot

* fix gcp pd manifest

* Update stacks ref

* revert alice example to gcp stack

* update snapshot

* fix profile controller iam binding

* Update kfp profile controller can be configured to different images and
istio sidecar

* add missing viewer controller cluster roles

* Use python3 for sync.py

* Revert gcp stack back to use non multi user kfp

* revert unintended changes

* revert upstream changes

* Secure kfp multi user mode with istio authorization

* patch minio to disable istio sidecar injection

* fix cache server istio authz

* enable istio sidecar for profiles deploy

* enable istio sidecar for centraldashboard

* Do not protect profile controller with istio

* Allow admission webhook traffic to cache-server

* revert gcp stack back to pipeline generic

* Reuse minio generic install as base for gcp-pd and ibm

* update snapshot

* refactor: pipelines profile controller should get minio access keys from the secret (#1372)

* refactor: pipelines profile controller should get minio access keys from the secret

* do not print secrets in log

* feat: Use KFP multi user mode for GCP (#1373)

* refactor: pipelines profile controller should get minio access keys from the secret

* do not print secrets in log

* use kfp multi user mode for gcp stacks

* update snapshot

* feat: Add application and common labels to KFP and various fixes (#1374)

* Add common labels to kfp components

* Add KFP application

* update snapshot

* Use json format for json patch, because yaml will look like a resource and fail tests

* Remove part of label

* update snapshots

* Fix profile controller deployment version

* update snapshot

* Fix userid-header for gcp

* update snapshot

* Fix b64encode exception

* update snapshot

* update snapshot

hack/pull_kfp_upstream.sh

@@ -10,11 +10,15 @@ set -ex
 # pipelines version.
 export PIPELINES_VERSION=1.0.0-rc.3
 export PIPELINES_SRC_REPO=https://github.com/kubeflow/pipelines.git
-# Pulling for the first time
-# kpt pkg get $PIPELINES_SRC_REPO/manifests/kustomize@$PIPELINES_VERSION pipeline/upstream
 
-# Updates
-kpt pkg update pipeline/upstream/@$PIPELINES_VERSION --strategy force-delete-replace
+if [ -d pipeline/upstream ]; then
+    # Updates
+    kpt pkg update pipeline/upstream/@$PIPELINES_VERSION --strategy force-delete-replace
+else
+    # Pulling for the first time
+    kpt pkg get $PIPELINES_SRC_REPO/manifests/kustomize@$PIPELINES_VERSION pipeline/upstream
+fi
+
 # Before kubeflow/pipelines/manifests/kustomize supports kustomize v3.5+, we
 # have to convert kustomization.yaml env to envs syntax, so that it is compatible
 # with latest kustomize used in kubeflow/manifests.

pipeline/installs/generic/application.yaml

@@ -0,0 +1,43 @@
+apiVersion: app.k8s.io/v1beta1
+kind: Application
+metadata:
+  name: kubeflow-pipelines
+  annotations:
+    kubernetes-engine.cloud.google.com/icon: >-
+      data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAADMAAAAyCAYAAADx/eOPAAALuUlEQVRogd2afWxd9XnHP99bK4pS3yhiGUIRiiJUVVVqbq8ppdR20ibqpuIMtDRkUYERp29J57gMZVuxsrZiK7oZXVv5re1AiOuoG+N1DMkuytprsGPEVMouxqQZJVHEWIdQlGVxZlmZdb/747zcc869jpMO+seOdPz7nd/L83tev89zzjX8Bq795Rq9o17zXp+Tey+Ijkyboela29DRWkhffyT733pH/Z3este9F2cC6N0kNjxtjD+FdRD8UF9X7u97y7UbQFPAivC0BdllS381slun3s3z3xVhhqeds90tqR/oMB7u68z19ZZra0E/l1if3WOziPx3skrDPTr+bvDxfxImEIJbgX6gGBJ7EfHJX/ySReDHwO9KYAenyWCMFKw21GSeslwa2Z17+TcuzPBRr7B8m6Df5oOJqdPAR/u6cm/2lmv3At+IT3GiXZqbcaxSLsfRoTsvn7XL2jE87ZXGnwf+VGiDY86ETM1wU1+XjvSW/RlgTJADQ2QaCZKWcX1/aDIcjE8i3SdzZLjn0lm8pJXD02417BM+gLmq2Rqjr/d16Vu95dp6wc8Ra5O8NrPIcoZCvIR1H+KZkd2qLcfnRYUZOuorJO+3uQt0RerolGYZR7r5+C9ZATwPviGyQprd6Liszy3bnwVKwGMjPbnFyxJmeNpX2T4gaR/QmmSpyYZTho/2depMb9k/kNh3KawuJ1bWauHzUcyXRpZAv5Zmg7aHBLcmNN9ECAFeAO3s69KZ3nLtDuF9dnBs0IT9JO24rbPb0JfP2syCZpFfE5q1mRWcvlgMNcwMTRq9z/+OWXdx4AGjvX1deqC37DbwPwOrMgsufol5mWMWs1ivEbjTrOCtLNNb+udygqsNbUBtopR/NkuuwTJ6Hxsw67KSuvH5MPDA/nJttfGTdUFCMUlp/ALwOtIs9muBxpnFnBzuSQf21oP/BbXclVvumWuTaDN8WNBm2GizJkxPM0CDMA2WGZ72bbb/Njue2TRj9Il/PcG87SeBz4ZTNaSTsmctHcO8SqDp14d7dCFLZ2v/3OpQ023Ah4n65kohvETUCdcsfmuilD+bpNdgGZvOODuHqYGIVGCec9g7+7o031v2jaBTiD0ysxbHRnZrPktzyz1zK7f0z10nh5pWwLRhvZro1KqznVJhNB8UyDeSsU4zAOiIXV1OuEqQ2AR79nflXgcY6dGLwIvR8q39cy1b+uc2Emo6dI824BpMSxz8iVhy4m/2WiYHdV5UmOHp2mpwm52ESCdwRn+9v0tPAWzpn9sAFAQbMdc60PaHsFZEWd9uxk4z8G3seykECfObTEd2KmuZG4CWyLXkYLMwtiYt+hMsTUdAEZQzjs9apv66SHJRk73ZjBQ+iRu29s+1VEr5OImmXs4MHUahVoLWgK23wbv6OrU4OulcuHYehWsVHhpXwpE2FNRayTszX2cwDpQEzTB+QvrJHCXUaigk+c++aXZiE98YmUVgV19X7u3ypH/fgfUA5h2usY2jNjmWoGVn50nvC9T2NviA5OPBGPW91OlG+0Xa1WJhhqadk3WjpKCilQIQFP19XZocnfIHgIeFWyNh6goXyX6gdNWfU8aJ5tNjEheAHZVS/ruGj0s8k6VPhh6ms6kwgoLl1aGuCEuSpwXfHZ2qrTJ+HHkNCpOjmbdFcEcGUIhUSj/H65rPO6j+766U8i/QXV0z8cqJc4btwF8AtWgtMb1wj+j41Df/s1EYQwdEDiqM3hDes9quGY3IKoYOvCrU7HlCoZtEWapPkzEpsU8uq8b36a6uBqaBv5l45URLpZT/pmGH8LnkvlAdAOt1oeXqRsuYTjlEMJiXvWN/Z+5szfqioKcOKo7qr/nAEesKiOyv2A/q88rOx8+8bPhK5dUTAA8jbUT6MuKnbKteNVHKP23xCeD1LC0F2TWOmzoAKEiWxmC+sr8rN1OerF2HGaqXFcZhDWaYj11S4ZxcXxVqyKqPZOeNTwM7Jkr5BeDPQJ8NFQaoC/gZ26rXT5TyxxAfRx6P94d0gU0pYYama+tsbwix/AHM4fKUrwAeB68kRJ5AZsWWieGTjLipsVCgrKCwKHF7pZQ/RXf104j76i4ZMmquxkzRXb2zUsqfxdxsfCiA70hRjZbpCDHmJcRdeZPDHkVck0Ul5PeHZ81DgHxKtglXaHCxVN9fr5TyR9hW3QA8Amqp5526SyKtBEbZVv1eZeZkbqKU7xfsFJwPqRW29s+11oUxnUhnkHf2dWoB+R5Jv5dNaGHh1wog8d/ZAI+0GgVpFPTp4AfJT2Hup7u6EvMk0tpkboutEz0HMPzHyD+mu3pFpZR/Aug0Pgm0RLkvFzLWYfjDvs7cqfKUt2LuXTLhue5mdWhVDJdEzxDDcRKawceN9lRePVkDfgBcR/LKVqNpz/s08DO6q4VKKT8j8zHgJ1HyzA1P11YZjfV1arw85auBR4RalDB5lEjDKi0CgPPphKZ0QiNRwUQeg88B2ydKreew9yH1NCxe/r4GaZpt1Vsrh/JnDDcBLwPkbLVgf6s86RXYj4KvtJKJM8KsGLkSlsmUL6mSg1RJY1xD7KmU8sfprnYgBqJsGVsiEfupsca7FfMo26p/OfHKiVqllB8HyPV16VxfV66G/G1QBwY5xvCgTT7X3/MTaBbFVr0fJvqw2ASZ+yul/FN0V68CHsesiDl3UopM3CwhDZDD/Dnwj3S/sjoYAMqTtc1YX02jVqYOiuuqsAKIkqZCfFIz/IrfFY8gDrKt2gI8irSuwQezyTeNaOl+6qYb+fpYGKEXJE9GSTObK5ItrheaLHE5/XRKcHul+kYN8x2kzWlLNNuVtUqibzKW5CBjxUoszO7NWrS1E/xWvMeJjck2WQHEKJeMD+qH4gWCSvg00m3AVxv5TMRKsp9Cs0Q/Ka/1BOZQNBSXMz2b9Q5oO9JCKgkqg2aKofl8uvTPeE1w3t5KKf8y26pFxINhLRa5R9JV6huT/aZuFu7Ds+A9jBdj+VIvZz2b9BL2Xi5yJQEgUFqinI9SZBDx358o5Q/HiRGtquOEmxJu6DcbC/afQWxnvHg+Odrwm2bP5txh5OEYjOM3vaiu8qqHJw1mPmK/Xs7HJf0LRncDMF5cAL6NWUxDrX/duwbczljxjSzvTX+gtXU3MBlrRCltrsxBTgorACKrRGf5bczOiVLrhUL74B2F9oHVjBd/iLwTWEhr+CIWaLYumDjIWLHha+aSwvRs1iJmJ9Kb9ZJRETS3ACsMC8i1ZNwgXZDYWTmU/1WhfeAW8Cjo+UL7wDrGik8jfid0kYz/Z2ODepv+GPIY+FAznpcUJhAo9w5mh81CFtEsWieCTzwXkogmfKBSyh8ttA98EDPqoPouYqYLxYEPMVY8itmEeTM+KEaqZhVAkiPPIL6QDPhLFiYQSC9J7M3mGlF/24zWSvwIM1xoH2gF/sFiTcSPxQakqUJxsIPx4jGCr0AzCUYTbROJ7DPAdsbSAX9ZwgDs3qTDiMGUOxF/1DgfekLVsPf0sw8DPARsDNwy8iYBXov4p0L7wC2MF99CfBJ4rqmbJbO/qYE+x1jx5HK8Xtp/aFgHDM/FX+RM9FFjHjjj4NV3HvlPsP4g+SqQgm6zCuvJQnHgi4wVz2JuAj8RnLGEVaCf8Y8cuRQ2L0mYEBB2Gb8ZHKD4NQBx+0Qpf7LQPrAVVGqiiWTpCcEn4QcLxcF7C7+aXMDahT1YX5IS5DHE/ZfC4yULEwr0DtIOWwuuvwZ8rVLKP1soDqzHPGJoyRao9b4SXiQQ30A8eO1/PJ8D7gK+BtQSJcQM8AXGlg747LUkmi91lad8J3CuZ5OeBii0D64ET2FdH1N0omWJvgLPkvwM8LmZf7lrnm3VO4CHsM4DH2P8I8vGSfK67P9q8v9wWPAcQLH4PbBHbK6Pq+3M9+Ml+6FL2dyC+WmhOLiWseKPMDeDd12uIPBrWCZ5Xds++AHsAwGlBKnoB5747c2J+aSJEuvRL8CDv/2Zz+cqh/LL/gPD//vrfwFjcI5oX6jDBwAAAABJRU5ErkJggg==
+spec:
+  addOwnerRef: true
+  selector:
+    matchLabels:
+      app.kubernetes.io/application: kubeflow-pipelines
+  descriptor:
+    version: $(kfp-app-version)
+    type: Kubeflow Pipelines
+    description: |-
+      Reusable end-to-end ML workflow
+    maintainers:
+    - name: Kubeflow Pipelines
+      url: https://github.com/kubeflow/pipelines
+    links:
+    - description: 'Kubeflow Pipelines Documentation'
+      url: https://www.kubeflow.org/docs/pipelines/
+  componentKinds:
+  - group: v1
+    kind: ServiceAccount
+  - group: rbac.authorization.k8s.io/v1
+    kind: Role
+  - group: rbac.authorization.k8s.io/v1
+    kind: RoleBinding
+  - group: v1
+    kind: Service
+  - group: v1
+    kind: PersistentVolumeClaim
+  - group: v1
+    kind: ConfigMap
+  - group: v1
+    kind: Secret
+  - group: apps/v1
+    kind: Deployment
+  - group: networking.istio.io/v1alpha3
+    kind: VirtualService
+

pipeline/installs/generic/kustomization.yaml

@@ -1,15 +1,22 @@
 apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
 namespace: kubeflow
+commonLabels:
+  app.kubernetes.io/name: kubeflow-pipelines
+  app.kubernetes.io/component: ml-pipeline
 resources:
 - ../../upstream/base/pipeline/cluster-scoped
 - ../../upstream/base/pipeline
 - virtual-service.yaml
 - ../../cache/base_v3
+- application.yaml
 configMapGenerator:
 - name: pipeline-install-config
   envs:
   - ./params.env
+- name: pipeline-upstream-install-config
+  envs:
+  - ../../upstream/base/params.env
 secretGenerator:
 - name: mysql-secret
   envs:
@@ -22,5 +29,12 @@ vars:
     apiVersion: v1
   fieldref:
     fieldpath: metadata.namespace
+- name: kfp-app-version
+  objref:
+    kind: ConfigMap
+    name: pipeline-upstream-install-config
+    apiVersion: v1
+  fieldref:
+    fieldpath: data.appVersion
 configurations:
 - params.yaml

pipeline/installs/generic/params.yaml

@@ -1,3 +1,5 @@
 varReference:
 - path: spec/http/route/destination/host
   kind: VirtualService
+- path: spec/descriptor/version
+  kind: Application

pipeline/installs/multi-user/api-service/cluster-role-binding.yaml

@@ -0,0 +1,11 @@
+apiVersion: rbac.authorization.k8s.io/v1beta1
+kind: ClusterRoleBinding
+metadata:
+  name: ml-pipeline
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: ml-pipeline
+subjects:
+- kind: ServiceAccount
+  name: ml-pipeline

pipeline/installs/multi-user/api-service/cluster-role.yaml

@@ -0,0 +1,34 @@
+apiVersion: rbac.authorization.k8s.io/v1beta1
+kind: ClusterRole
+metadata:
+  name: ml-pipeline
+rules:
+- apiGroups:
+  - argoproj.io
+  resources:
+  - workflows
+  verbs:
+  - create
+  - get
+  - list
+  - watch
+  - update
+  - patch
+  - delete
+- apiGroups:
+  - kubeflow.org
+  resources:
+  - scheduledworkflows
+  verbs:
+  - create
+  - get
+  - list
+  - update
+  - patch
+  - delete
+- apiGroups:
+  - ""
+  resources:
+  - pods
+  verbs:
+  - delete

pipeline/installs/multi-user/api-service/deployment-patch.yaml

@@ -0,0 +1,23 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: ml-pipeline
+spec:
+  template:
+    spec:
+      containers:
+      - name: ml-pipeline-api-server
+        envFrom:
+        - configMapRef:
+            name: pipeline-api-server-config
+        env:
+        - name: KUBEFLOW_USERID_HEADER
+          valueFrom:
+            configMapKeyRef:
+              name: kubeflow-config
+              key: userid-header
+        - name: KUBEFLOW_USERID_PREFIX
+          valueFrom:
+            configMapKeyRef:
+              name: kubeflow-config
+              key: userid-prefix

pipeline/installs/multi-user/api-service/kustomization.yaml

@@ -0,0 +1,8 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+- cluster-role-binding.yaml
+- cluster-role.yaml
+configMapGenerator:
+- name: pipeline-api-server-config
+  env: params.env

pipeline/installs/multi-user/api-service/params.env

@@ -0,0 +1,4 @@
+MULTIUSER=true
+DEFAULTPIPELINERUNNERSERVICEACCOUNT=default-editor
+VISUALIZATIONSERVICE_NAME=ml-pipeline-visualizationserver
+VISUALIZATIONSERVICE_PORT=8888

pipeline/installs/multi-user/application-patch.json

@@ -0,0 +1,34 @@
+[
+    {
+        "op": "add",
+        "path": "/spec/componentKinds/0",
+        "value": {
+            "group": "metacontroller.k8s.io/v1alpha1",
+            "kind": "CompositeController"
+        }
+    },
+    {
+        "op": "add",
+        "path": "/spec/componentKinds/0",
+        "value": {
+            "group": "rbac.istio.io/v1alpha1",
+            "kind": "ServiceRole"
+        }
+    },
+    {
+        "op": "add",
+        "path": "/spec/componentKinds/0",
+        "value": {
+            "group": "rbac.istio.io/v1alpha1",
+            "kind": "ServiceRoleBinding"
+        }
+    },
+    {
+        "op": "add",
+        "path": "/spec/componentKinds/0",
+        "value": {
+            "group": "networking.istio.io/v1alpha3",
+            "kind": "DestinationRule"
+        }
+    }
+]

pipeline/installs/multi-user/cache/cluster-role-binding.yaml

@@ -0,0 +1,11 @@
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+  name: kubeflow-pipelines-cache-binding
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: kubeflow-pipelines-cache-role
+subjects:
+- kind: ServiceAccount
+  name: kubeflow-pipelines-cache

pipeline/installs/multi-user/cache/cluster-role.yaml

@@ -0,0 +1,31 @@
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  name: kubeflow-pipelines-cache-role
+rules:
+- apiGroups:
+  - ""
+  resources:
+  - pods
+  verbs:
+  - get
+  - list
+  - watch
+  - update
+  - patch
+- apiGroups:
+  - ""
+  resources:
+  - configmaps
+  verbs:
+  - get
+- apiGroups:
+  - argoproj.io
+  resources:
+  - workflows
+  verbs:
+  - get
+  - list
+  - watch
+  - update
+  - patch

pipeline/installs/multi-user/cache/deployment-patch.yaml

@@ -0,0 +1,13 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: cache-server
+spec:
+  template:
+    spec:
+      containers:
+      - name: server
+        env:
+          - name: NAMESPACE_TO_WATCH
+            value: ''
+            valueFrom: null

pipeline/installs/multi-user/cache/kustomization.yaml

@@ -0,0 +1,5 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+- cluster-role.yaml
+- cluster-role-binding.yaml