Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(manifest): add support for Standalone KFP on AWS. Fixes #4337 #4350

Merged
merged 1 commit into from
Aug 13, 2020
Merged

feat(manifest): add support for Standalone KFP on AWS. Fixes #4337 #4350

merged 1 commit into from
Aug 13, 2020

Conversation

PatrickXYS
Copy link
Member

@PatrickXYS PatrickXYS commented Aug 10, 2020 •
edited
Loading

Description of your changes:
Adopt standalone KFP solution on AWS by replacing minio with S3 and mysql with RDS.

Fix #4337

Checklist:

  • The title for your pull request (PR) should follow our title convention. Learn more about the pull request title convention used in this repository.

    PR titles examples:

    • fix(frontend): fixes empty page. Fixes #1234
      Use fix to indicate that this PR fixes a bug.
    • feat(backend): configurable service account. Fixes #1234, fixes #1235
      Use feat to indicate that this PR adds a new feature.
    • chore: set up changelog generation tools
      Use chore to indicate that this PR makes some changes that users don't need to know.
    • test: fix CI failure. Part of #1234
      Use part of to indicate that a PR is working on an issue, but shouldn't close the issue when merged.

  • Do you want this pull request (PR) cherry-picked into the current release branch?

    If yes, use one of the following options:

    • (Recommended.) Ask the PR approver to add the cherrypick-approved label to this PR. The release manager adds this PR to the release branch in a batch update.
    • After this PR is merged, create a cherry-pick PR to add these changes to the release branch. (For more information about creating a cherry-pick PR, see the Kubeflow Pipelines release guide.)

@google-cla google-cla bot added the cla: yes label Aug 10, 2020
@kubeflow-bot
Copy link

This change is Reviewable

@k8s-ci-robot k8s-ci-robot requested review from Bobgy and rmgogogo August 10, 2020 19:53
@PatrickXYS
Copy link
Member Author

/cc @Jeffwan Can you take a look?

Jeffwan
Jeffwan reviewed Aug 10, 2020
View reviewed changes
manifests/kustomize/env/aws/README.md Show resolved Hide resolved
manifests/kustomize/env/aws/aws-configuration-patch.yaml Show resolved Hide resolved
manifests/kustomize/env/aws/config Show resolved Hide resolved
manifests/kustomize/env/aws/params.env Outdated Show resolved Hide resolved
manifests/kustomize/env/aws/params.env Outdated Show resolved Hide resolved
manifests/kustomize/env/aws/README.md Outdated Show resolved Hide resolved
@PatrickXYS PatrickXYS force-pushed the yao_aws_account branch 4 times, most recently from 18271a8 to dcf69da Compare August 12, 2020 01:17
@Bobgy
Copy link
Contributor

Bobgy commented Aug 12, 2020

Would you mind adding an OWNERS file in this folder?

NikeNano
NikeNano reviewed Aug 12, 2020
View reviewed changes
Copy link
Member

@NikeNano NikeNano left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great update :) I have not used KFP standalone on AWS, is it possible to access KFP through the sdk as well? How is auth handled for AWS?

UPDATE: found this PR that answered my questions.

manifests/kustomize/env/aws/README.md Outdated
Follow this [doc](https://www.kubeflow.org/docs/aws/rds/#deploy-amazon-rds-mysql-in-your-environment) to set up AWS RDS instance.

4. Customize your values
- Edit **params.env**, **secret.env** and **minio-artifact-secret-patch.env**
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: maybe you could link the files.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@NikeNano I think this is for standalone. Basically, there's no public endpoint or auth enabled. You can customize it on your on. The PR you linked if for case if user add AWS Application Load balancer with Cognito. It provides a way to authenticate sdk.

Jeffwan
Jeffwan suggested changes Aug 12, 2020
View reviewed changes
manifests/kustomize/README.md Outdated Show resolved Hide resolved
manifests/kustomize/env/aws/README.md Show resolved Hide resolved
manifests/kustomize/env/aws/config Show resolved Hide resolved
@PatrickXYS PatrickXYS force-pushed the yao_aws_account branch 3 times, most recently from 45f44bb to 1299360 Compare August 12, 2020 17:45
@Bobgy
Copy link
Contributor

Bobgy commented Aug 12, 2020

/approve

Double check on release governance.

Does it work if we ping you to verify the manifest before every release?

Or will it be okay we just release as usual, it will be up to you to fix it?

Or will you be able to set up a postsubmit test here?

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Bobgy

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@PatrickXYS
Copy link
Member Author

Does it work if we ping you to verify the manifest before every release?

Or will it be okay we just release as usual, it will be up to you to fix it?

Or will you be able to set up a postsubmit test here?

I think for now, we'll manually test before every release. So feel free to ping me every time when KFP team release new version.

@Jeffwan
Copy link
Member

Jeffwan commented Aug 13, 2020

/lgtm

@k8s-ci-robot k8s-ci-robot merged commit 5ec7ec8 into kubeflow:master Aug 13, 2020
@PatrickXYS PatrickXYS mentioned this pull request Aug 13, 2020
Closed
PatrickXYS pushed a commit to PatrickXYS/pipelines that referenced this pull request Sep 12, 2020
feat(manifest): add support for Standalone KFP on AWS. Fixes kubeflow…
e06743a 
…#4337 (kubeflow#4350)
Bobgy pushed a commit that referenced this pull request Sep 12, 2020
@Bobgy
feat(manifest): add support for Standalone KFP on AWS. Fixes #4337 (#…
668a3ec 
…4350)
@Bobgy Bobgy mentioned this pull request Sep 30, 2020
Merged
2 tasks
Jeffwan pushed a commit to Jeffwan/pipelines that referenced this pull request Dec 9, 2020
@Jeffwan
feat(manifest): add support for Standalone KFP on AWS. Fixes kubeflow…
00c6356 
…#4337 (kubeflow#4350)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@NikeNano NikeNano NikeNano left review comments

@Jeffwan Jeffwan Jeffwan requested changes

@Bobgy Bobgy Awaiting requested review from Bobgy

@rmgogogo rmgogogo Awaiting requested review from rmgogogo

Assignees

@Jeffwan Jeffwan

Labels
approved cla: yes lgtm size/L
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add AWS overlay for production users
6 participants
@PatrickXYS @kubeflow-bot @Bobgy @k8s-ci-robot @Jeffwan @NikeNano