Support Kubeflow Jobs in MultiKueue

[alculquicondor]

What would you like to be added:

Support for Kubeflow Jobs in MultiKueue, in particular, for TFJob and PyTorchJob.
Ideally, the implementation should be mostly common among all job types.

Kubeflow Job doesn't have support for managedBy, so, for now, we can only support the scenario where the manager cluster doesn't have the controller installed.

Why is this needed:

To continue the incremental improvement of MK and satisfy the needs of early adopters.

Completion requirements:

This enhancement requires the following artifacts:

• Design doc
• API change
• Docs update

The artifacts should be linked in subsequent comments.

[alculquicondor]

/assign @mszadkow

[kannon92]

The hope is to implement a new version of Trainingoperator called TrainJob that would use JobSet as the base.

I think if rhat is done then one could use the ManagedField in JobSet for this.

[alculquicondor]

Right, but we have users requesting this feature today.

[kannon92]

Should there be some work done to add managedField to the KubeFlow API? Or are we trying to avoid that to satisfy a solution that bypasses KubeFlow releases?

[alculquicondor]

We don't need it in the current version of MultiKueue. We can just recommend users not to install the operator in the dispatcher cluster.

[mimowo]

/reopen
Let's close it when the ongoing effort of supporting managedBy is complete for the training-operator and MPIJob.

[k8s-ci-robot]

@mimowo: Reopened this issue.

In response to this:

/reopen
Let's close it when the ongoing effort of supporting managedBy is complete for the training-operator and MPIJob.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[tenzen-y]

FYI: kubeflow/training-operator#2203 was merged right now.
We will include the feature in the next training-operator minor release.

RC.0 with the managedBy feature will be released on January 20th, 2025.