Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix default eks storage config #907

Merged
merged 3 commits into from
Apr 13, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 24 additions & 30 deletions deployment/eks/flyte_generated.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8080,14 +8080,12 @@ data:
storage:
type: stow
stow:
kind: google
kind: s3
config:
json: ""
# TODO: replace <project-id> with the GCP project ID
project_id: <project-id>
scopes: https://www.googleapis.com/auth/devstorage.read_write
# TODO replace with the container (bucket) in GCS used by Flyte as intermediate store
container: "flyte"
auth_type: iam
region: us-east-2
# TODO replace with the container (bucket) in s3 used by Flyte as intermediate store
container: "flyte-demo"
# NOTE this cache configuration is purely for propeller. But since we are having a common storage
# config, we are configuring this value. In production create a separate storage config for
# propeller and increase the cache size
Expand All @@ -8098,7 +8096,7 @@ data:
maxDownloadMBs: 10
kind: ConfigMap
metadata:
name: datacatalog-config-2bb94tc5b5
name: datacatalog-config-fktmhdgfk8
namespace: flyte
---
apiVersion: v1
Expand Down Expand Up @@ -8178,14 +8176,12 @@ data:
storage:
type: stow
stow:
kind: google
kind: s3
config:
json: ""
# TODO: replace <project-id> with the GCP project ID
project_id: <project-id>
scopes: https://www.googleapis.com/auth/devstorage.read_write
# TODO replace with the container (bucket) in GCS used by Flyte as intermediate store
container: "flyte"
auth_type: iam
region: us-east-2
# TODO replace with the container (bucket) in s3 used by Flyte as intermediate store
container: "flyte-demo"
# NOTE this cache configuration is purely for propeller. But since we are having a common storage
# config, we are configuring this value. In production create a separate storage config for
# propeller and increase the cache size
Expand All @@ -8207,7 +8203,7 @@ data:
gpu: 1
kind: ConfigMap
metadata:
name: flyte-admin-config-t2f9g68k65
name: flyte-admin-config-4ft85ftt4m
namespace: flyte
---
apiVersion: v1
Expand Down Expand Up @@ -8350,14 +8346,12 @@ data:
storage:
type: stow
stow:
kind: google
kind: s3
config:
json: ""
# TODO: replace <project-id> with the GCP project ID
project_id: <project-id>
scopes: https://www.googleapis.com/auth/devstorage.read_write
# TODO replace with the container (bucket) in GCS used by Flyte as intermediate store
container: "flyte"
auth_type: iam
region: us-east-2
# TODO replace with the container (bucket) in s3 used by Flyte as intermediate store
container: "flyte-demo"
# NOTE this cache configuration is purely for propeller. But since we are having a common storage
# config, we are configuring this value. In production create a separate storage config for
# propeller and increase the cache size
Expand All @@ -8379,7 +8373,7 @@ data:
cloudwatch-log-group: <log-group>
kind: ConfigMap
metadata:
name: flyte-propeller-config-727k54546d
name: flyte-propeller-config-8fdc62895h
namespace: flyte
---
apiVersion: v1
Expand Down Expand Up @@ -8623,7 +8617,7 @@ spec:
- emptyDir: {}
name: shared-data
- configMap:
name: datacatalog-config-2bb94tc5b5
name: datacatalog-config-fktmhdgfk8
name: config-volume
- name: db-pass
secret:
Expand Down Expand Up @@ -8680,7 +8674,7 @@ spec:
serviceAccountName: flyte-pod-webhook
volumes:
- configMap:
name: flyte-propeller-config-727k54546d
name: flyte-propeller-config-8fdc62895h
name: config-volume
- name: webhook-certs
secret:
Expand Down Expand Up @@ -8806,7 +8800,7 @@ spec:
- emptyDir: {}
name: shared-data
- configMap:
name: flyte-admin-config-t2f9g68k65
name: flyte-admin-config-4ft85ftt4m
name: config-volume
- configMap:
name: clusterresource-template-tkdkkt4cb5
Expand Down Expand Up @@ -8898,7 +8892,7 @@ spec:
serviceAccountName: flytepropeller
volumes:
- configMap:
name: flyte-propeller-config-727k54546d
name: flyte-propeller-config-8fdc62895h
name: config-volume
---
apiVersion: apps/v1
Expand Down Expand Up @@ -9086,7 +9080,7 @@ spec:
name: clusterresource-template-tkdkkt4cb5
name: resource-templates
- configMap:
name: flyte-admin-config-t2f9g68k65
name: flyte-admin-config-4ft85ftt4m
name: config-volume
- name: db-pass
secret:
Expand Down Expand Up @@ -9129,7 +9123,7 @@ spec:
serviceAccountName: flyte-pod-webhook
volumes:
- configMap:
name: flyte-propeller-config-727k54546d
name: flyte-propeller-config-8fdc62895h
name: config-volume
ttlSecondsAfterFinished: 0
---
Expand Down
12 changes: 5 additions & 7 deletions kustomize/overlays/eks/flyte/config/common/storage.yaml
Original file line number Diff line number Diff line change
@@ -1,14 +1,12 @@
storage:
type: stow
stow:
kind: google
kind: s3
config:
json: ""
# TODO: replace <project-id> with the GCP project ID
project_id: <project-id>
scopes: https://www.googleapis.com/auth/devstorage.read_write
# TODO replace with the container (bucket) in GCS used by Flyte as intermediate store
container: "flyte"
auth_type: iam
region: us-east-2
# TODO replace with the container (bucket) in s3 used by Flyte as intermediate store
container: "flyte-demo"
# NOTE this cache configuration is purely for propeller. But since we are having a common storage
# config, we are configuring this value. In production create a separate storage config for
# propeller and increase the cache size
Expand Down