Set s3 endpoint in workflow-controller-configmap from pipeline-instal…

[rrrkharse]

Which issue is resolved by this Pull Request:
Resolves #257

Description of your changes:
Passes the data.minioServiceHost from the pipeline-install-config config map into the workflow-controller-configmap config map via kustomization vars.

The data.minioServiceHost field in pipeline-install-config is pulled from the params.env file in awsconfigs/apps/pipeline/s3

Testing:

Case 1: Before this PR (s3 endpoint is static in the workflow config map)

params.env

bucketName=
minioServiceHost=s3.amazonaws.com
minioServiceRegion=

kustomize build output

---
apiVersion: v1
data:
  ConMaxLifeTime: 120s
  appName: pipeline
  appVersion: 1.8.2
  autoUpdatePipelineDefaultVersion: "true"
  bucketName: ""
  cacheDb: cachedb
  cacheImage: gcr.io/google-containers/busybox
  cacheNodeRestrictions: "false"
  cronScheduleTimezone: UTC
  dbHost: mysql
  dbPort: "3306"
  defaultPipelineRoot: ""
  minioServiceHost: s3.amazonaws.com                                   # same value as minioServiceHost=s3.amazonaws.com
  minioServiceRegion: ""
  mlmdDb: metadb
  pipelineDb: mlpipeline
  warning: |
    1. Do not use kubectl to edit this configmap, because some values are used
    during kustomize build. Instead, change the configmap and apply the entire
    kustomize manifests again.
    2. After updating the configmap, some deployments may need to be restarted
    until the changes take effect. A quick way to restart all deployments in a
    namespace: `kubectl rollout restart deployment -n <your-namespace>`.
kind: ConfigMap
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: ml-pipeline
    app.kubernetes.io/name: kubeflow-pipelines
    application-crd-id: kubeflow-pipelines
  name: pipeline-install-config
  namespace: kubeflow
---
apiVersion: v1
data:
  config: |
    {
    artifactRepository:
    {
        s3: {
            bucket: ,
            keyPrefix: artifacts,
            endpoint: s3.amazonaws.com,                                    # static value
            insecure: true,
            accessKeySecret: {
                name: mlpipeline-minio-artifact,
                key: accesskey
            },
            secretKeySecret: {
                name: mlpipeline-minio-artifact,
                key: secretkey
            }
        },
        archiveLogs: true
    }
    }
kind: ConfigMap
metadata:
  annotations: {}
  labels:
    application-crd-id: kubeflow-pipelines
  name: workflow-controller-configmap
  namespace: kubeflow
---

Case 2: After this PR, using default minioServiceHost in params.env

params.env

bucketName=
minioServiceHost=s3.amazonaws.com
minioServiceRegion=

kustomize build output

---
apiVersion: v1
data:
  ConMaxLifeTime: 120s
  appName: pipeline
  appVersion: 1.8.2
  autoUpdatePipelineDefaultVersion: "true"
  bucketName: ""
  cacheDb: cachedb
  cacheImage: gcr.io/google-containers/busybox
  cacheNodeRestrictions: "false"
  cronScheduleTimezone: UTC
  dbHost: mysql
  dbPort: "3306"
  defaultPipelineRoot: ""
  minioServiceHost: s3.amazonaws.com                                   # same value as minioServiceHost=s3.amazonaws.com
  minioServiceRegion: ""
  mlmdDb: metadb
  pipelineDb: mlpipeline
  warning: |
    1. Do not use kubectl to edit this configmap, because some values are used
    during kustomize build. Instead, change the configmap and apply the entire
    kustomize manifests again.
    2. After updating the configmap, some deployments may need to be restarted
    until the changes take effect. A quick way to restart all deployments in a
    namespace: `kubectl rollout restart deployment -n <your-namespace>`.
kind: ConfigMap
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: ml-pipeline
    app.kubernetes.io/name: kubeflow-pipelines
    application-crd-id: kubeflow-pipelines
  name: pipeline-install-config
  namespace: kubeflow
---
apiVersion: v1
data:
  config: |
    {
    artifactRepository:
    {
        s3: {
            bucket: ,
            keyPrefix: artifacts,
            endpoint: s3.amazonaws.com,                                    # same value as minioServiceHost=s3.amazonaws.com
            insecure: true,
            accessKeySecret: {
                name: mlpipeline-minio-artifact,
                key: accesskey
            },
            secretKeySecret: {
                name: mlpipeline-minio-artifact,
                key: secretkey
            }
        },
        archiveLogs: true
    }
    }
kind: ConfigMap
metadata:
  annotations: {}
  labels:
    application-crd-id: kubeflow-pipelines
  name: workflow-controller-configmap
  namespace: kubeflow
---

Case 3: After this PR, using custom minioServiceHost in params.env

params.env

bucketName=
minioServiceHost=s3.otherendpoint.amazonaws.com
minioServiceRegion=

kustomize build output

---
apiVersion: v1
data:
  ConMaxLifeTime: 120s
  appName: pipeline
  appVersion: 1.8.2
  autoUpdatePipelineDefaultVersion: "true"
  bucketName: ""
  cacheDb: cachedb
  cacheImage: gcr.io/google-containers/busybox
  cacheNodeRestrictions: "false"
  cronScheduleTimezone: UTC
  dbHost: mysql
  dbPort: "3306"
  defaultPipelineRoot: ""
  minioServiceHost: s3.otherendpoint.amazonaws.com      # custom value from minioServiceHost
  minioServiceRegion: ""
  mlmdDb: metadb
  pipelineDb: mlpipeline
  warning: |
    1. Do not use kubectl to edit this configmap, because some values are used
    during kustomize build. Instead, change the configmap and apply the entire
    kustomize manifests again.
    2. After updating the configmap, some deployments may need to be restarted
    until the changes take effect. A quick way to restart all deployments in a
    namespace: `kubectl rollout restart deployment -n <your-namespace>`.
kind: ConfigMap
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: ml-pipeline
    app.kubernetes.io/name: kubeflow-pipelines
    application-crd-id: kubeflow-pipelines
  name: pipeline-install-config
  namespace: kubeflow
---
apiVersion: v1
data:
  config: |
    {
    artifactRepository:
    {
        s3: {
            bucket: ,
            keyPrefix: artifacts,
            endpoint: s3.otherendpoint.amazonaws.com,                             # custom value from minioServiceHost
            insecure: true,
            accessKeySecret: {
                name: mlpipeline-minio-artifact,
                key: accesskey
            },
            secretKeySecret: {
                name: mlpipeline-minio-artifact,
                key: secretkey
            }
        },
        archiveLogs: true
    }
    }
kind: ConfigMap
metadata:
  annotations: {}
  labels:
    application-crd-id: kubeflow-pipelines
  name: workflow-controller-configmap
  namespace: kubeflow
---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.