Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create Pytorch Informer/ClientSet #1501

Conversation

migueltol22
Copy link

@migueltol22 migueltol22 commented Dec 6, 2021

What this PR does / why we need it:
This PR runs code generator to create a pytorch client and informer.

Which issue(s) this PR fixes Fixes #1499

Checklist:

  • Docs included if any changes are user facing

@aws-kf-ci-bot
Copy link
Contributor

Hi @migueltol22. Thanks for your PR.

I'm waiting for a kubeflow member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@google-oss-prow
Copy link

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: migueltol22
To complete the pull request process, please assign jeffwan after the PR has been reviewed.
You can assign the PR to them by writing /assign @jeffwan in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@@ -19,6 +19,7 @@ package versioned
import (
"fmt"

kubeflowv1 "github.com/kubeflow/training-operator/pkg/client/clientset/versioned/typed/pytorch/v1"
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@terrytangyuan here is an example of the conflicting import mentioned in this comment

Copy link
Member

@gaocegege gaocegege left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please also add XGBoostJob, MXNetJob, and MPIJob?

@google-oss-prow google-oss-prow bot added size/XXL and removed size/XL labels Dec 7, 2021
@migueltol22
Copy link
Author

Could you please also add XGBoostJob, MXNetJob, and MPIJob?

Added all 3. Although the imports aliases conflict with each other. @gaocegege do you happen to know what's needed to resolve this or is the expectation to manually resolve them?

Comment on lines +22 to +24
kubeflowv1 "github.com/kubeflow/training-operator/pkg/client/clientset/versioned/typed/mpi/v1"
kubeflowv1 "github.com/kubeflow/training-operator/pkg/client/clientset/versioned/typed/mxnet/v1"
kubeflowv1 "github.com/kubeflow/training-operator/pkg/client/clientset/versioned/typed/pytorch/v1"
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gaocegege This is the issue I mentioned with the imports. I'm not sure if I missed something or I should manually fix them?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, get it.

@gaocegege
Copy link
Member

Ref kubernetes/code-generator#135

@cheimu
Copy link
Member

cheimu commented Mar 27, 2022

I'm extremely curious, will this pr continue to update? If possible, I'd like to help : )

@cheimu
Copy link
Member

cheimu commented Mar 27, 2022

Ref kubernetes/code-generator#135

I also think the problem is this groupName. See mpi group name, mxnet group name, pytorch group name, tensorflow group name and xgb group name: they are all kubeflow.org. I think there could be two solutions for it.

  1. Our hierarchy is /pkg/apis/<ml_framework_x>/<version>/ml_framework_x_type.go, where in fact, each ml framework actually is a subgroup, so if we can add one more subgroupName to the groupName, such as tfjob.kubeflow.org, then packing collision could be fixed.
  2. Like other projects, we change the hierarchy to /pkg/apis/<version>/<ml_framework_x>_type.go where they all span in same directory meaning they are in same subgroup, then when generating clientset informer, they will be in the same package.

@gaocegege @zw0610 what do you guys think? 🤔 If so, tons of work will need to adapt it I guess

@zw0610
Copy link
Member

zw0610 commented May 6, 2022

Ref kubernetes/code-generator#135

I also think the problem is this groupName. See mpi group name, mxnet group name, pytorch group name, tensorflow group name and xgb group name: they are all kubeflow.org. I think there could be two solutions for it.

  1. Our hierarchy is /pkg/apis/<ml_framework_x>/<version>/ml_framework_x_type.go, where in fact, each ml framework actually is a subgroup, so if we can add one more subgroupName to the groupName, such as tfjob.kubeflow.org, then packing collision could be fixed.
  2. Like other projects, we change the hierarchy to /pkg/apis/<version>/<ml_framework_x>_type.go where they all span in same directory meaning they are in same subgroup, then when generating clientset informer, they will be in the same package.

@gaocegege @zw0610 what do you guys think? 🤔 If so, tons of work will need to adapt it I guess

The second option looks more authentic to me.

Copy link
Member

@Arhell Arhell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/ok-to-test

@johnugeorge
Copy link
Member

We can close this PR as #1610 is merged

@github-actions
Copy link

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@tenzen-y
Copy link
Member

/close

@google-oss-prow google-oss-prow bot closed this Sep 29, 2023
@google-oss-prow
Copy link

@tenzen-y: Closed this PR.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature] Create a Informer/ClientSet for PyTorch Jobs
8 participants