-
Notifications
You must be signed in to change notification settings - Fork 14.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add GKEStartKueueInsideClusterOperator to support Kueue inside clusters #37072
Conversation
Why add an operator in airflow to install the installation thanks to kubectl is not enough ? https://github.com/kubernetes-sigs/kueue?tab=readme-ov-file#installation |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We usually use infrastructure as code to install the cluster and additional things together. what is the significance installing Kueue alone?
Hey @dirrao @raphaelauv ! |
Not true, we have an operator to create a GKE cluster where some users create an ephemeral cluster to run their tasks, then they delete it at the end of the run. My first thought was similar to what @raphaelauv said, but since we support creating an ephemeral GKE cluster, why do we refuse to support configuring it? WDYT? |
Yah. You are right. Thanks for answering question. |
Yes. I am strongly included towards this. Use bash operator and run the kubectl command instead creating this operator. |
Okay, but running this in BashOperator will not give us the ability to:
@potiuk Would you suggest something regarding this? |
There is already KubernetesCreateResourceOperator For creating any resource in k8s |
Yes, thank you for the idea, it makes sense to inherit existing logic from this operator. But i would still suggest to keep logic to install Kueue in separate operator, since there is more logic needed then just simply run kubectl command in BashOperator |
Hi @raphaelauv ! As i have checked the KubernetesCreateResourceOperator is not using any specific hook method to call create_from_yaml() from kubernetes client. The logic that we actually need in GKEStartKueueInsideClusterOperator covers more then just this, as i have mentioned before: checking is the cluster accepts installing Kueue inside, waiting for the yaml to be applied on the Cluster method and also outputting errors if there are such. |
7c52afd
to
b587481
Compare
why build operators specific to GKE ? |
cf0e078
to
076feab
Compare
All the operators that we have for operations on cluster (create and delete) were implemented as a part of google provider as GKE operators. So the operator to add support of Kueue inside cluster was decided to add to the same provider without creating base class in Kubernetes. |
create a GKE is calling GCP API deploy a https://github.com/kubernetes-sigs/kueue is calling K8S API not GCP API |
Based on this documentation for Kueue the conditions to meet before we actually can install Kueue are different from the conditions for installing Kueue inside GKE cluster. So the only similar thing we will have between KubernetesInstallKueueInsideOperator and GKEInstallKueueInsideOperator will be only the k8s API call to apply YAML file inside cluster. There will be no need at all to inherit GKE from Kubernetes in this case, because classes and their checks will be different. |
I do not see any reference to GKE in this doc and anything that explicit that deploy Kueue in GKE is different than deploy Kueue to a "normal" K8S |
If you will take a look on the document i have added regarding installation of Kueue inside GKE you will see that conditions for GKE are different from conditions for general K8s cluster. GKEStartKueueInsideClusterOperator was designed specifically to cover GKE clusters and all the checks were added to check needed conditions. |
4271a69
to
ff69bca
Compare
06605cd
to
cdfa800
Compare
cdfa800
to
9529b7e
Compare
This PR adds the ability to install Kueue inside specific types of clusters in GKE.
For more details please check: https://cloud.google.com/kubernetes-engine/docs/tutorials/kueue-intro#create_a_cluster
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rst
or{issue_number}.significant.rst
, in newsfragments.