[v1alpha2] Sync commits with v1alpha1

[gaocegege]

• Use headless services for Training jobs (Use headless services for Training jobs #471)
  • jlewi@ created v1alpha2 should use headless services #574 to track this
• fix field selectors in controller (Fix field selectors in controller #465)
• Fix dashboard + proxy incompatibility (Fix dashboard + proxy incompatibility #441)
• Create PDB of TFReplicaSet for gang scheduling by kube-arbitrator (Create PDB of TFReplicaSet for gang scheduling by kube-arbitrator #452)
  • jlewi@ created [v1alpha2] Add PDB of TFReplicaSet for gang scheduling by kube-arbitrator #575 to track this

There are 4 commits and we should sync them in v1alpha2. #452 #465 #471 needs to be re-implemented and #441 could be cherry-picked.

[ScorpioCPH]

Yes, we can create new PR for this after merge v1alpha2 to master.

[jlewi]

What does sync commits mean? Are we maintaining multiple branches?

I thought per the discussion in #492 the plan was to have 2 different binaries in master 1 for v1alpha2 and 1 for v1alpha1?

So is this issue about keeping both implementations in master in sync? or is this about keeping branches in sync?

I think it will be much better if we keep both implementations on master. Then if there are bits of code that should be the same for both implementations we can just create a function or some other mechanism to allow code reuse.

[gaocegege]

The commits implements some new features in v1alpha1 and I think we could support it in v1alpha2, too. Then we need to re-implement these features in v1alpha2.

[gaocegege]

@jlewi PTAL #492 and I think we could merge it to continue the development.

We keep two implementations in the PR #492 and I think we should deprecate v1alpha1 when downstream updates to the new API.

[gaocegege]

@mitake Could I assign implement gang scheduling option on v1alpha2 to you?

[mitake]

@gaocegege sure, please assign the task to me

[gaocegege]

@mitake Now the v1alpha2 is merged and I think you are not blocked now 🎉

[jlewi]

@gaocegege @ScorpioCPH I'd like to replace this issue with finer grained issues so its easier to track and assign the actual work items.

I created the issues

• v1alpha2 should use headless services #574 to support headless services in v1alpha2
• [v1alpha2] Add PDB of TFReplicaSet for gang scheduling by kube-arbitrator #575 to support gang scheduling

Regarding the PRs mentioned in the first comment

• Create PDB of TFReplicaSet for gang scheduling by kube-arbitrator #452 this is about gang scheduling and covered by [v1alpha2] Add PDB of TFReplicaSet for gang scheduling by kube-arbitrator #575
• Use headless services for Training jobs #471 headless services is covered by v1alpha2 should use headless services #574
• Fix dashboard + proxy incompatibility #441 Cherrypick fixes to dasbhoard
  • I believe this is unnecessary because this change is already on master and development of v1alpha2 is on master now so there's now reason to cherry pick this into the old development branch for v1alpha2.

So I think the only remaining issue is

• Fix field selectors in controller #465 Fix field selectors; is this an issue in v1alpha2? I think we are taking a different approach in v1alpha2?

If its still an issue can we open up a separate issue for it and close this one?

[ScorpioCPH]

@jlewi Thanks, LGTM, we can close this one.