| Field | Description |
|---|---|
qps [Required]+ float32
+ |
+
+ QPS controls the number of queries per second allowed for K8S api server +connection. + |
+
burst [Required]+ int32
+ |
+
+ Burst allows extra queries to accumulate when a client is exceeding its rate. + |
+
| Field | Description |
|---|---|
maxCount [Required]+ int32
+ |
+
+ MaxCount indicates the maximal number of pending workloads exposed in the +cluster queue status. When the value is set to 0, then ClusterQueue +visibility updates are disabled. +The maximal value is 4000. +Defaults to 10. + |
+
Configuration is the Schema for the kueueconfigurations API
+ + +| Field | Description |
|---|---|
namespace [Required]+ string
+ |
+
+ Namespace is the namespace in which kueue is deployed. It is used as part of DNSName of the webhook Service. +If not set, the value is set from the file /var/run/secrets/kubernetes.io/serviceaccount/namespace +If the file doesn't exist, default value is kueue-system. + |
+
ControllerManager [Required]+ ControllerManager
+ |
+(Members of ControllerManager are embedded into this type.)
+ ControllerManager returns the configurations for controllers + |
+
manageJobsWithoutQueueName [Required]+ bool
+ |
+
+ ManageJobsWithoutQueueName controls whether or not Kueue reconciles +batch/v1.Jobs that don't set the annotation kueue.x-k8s.io/queue-name. +If set to true, then those jobs will be suspended and never started unless +they are assigned a queue and eventually admitted. This also applies to +jobs created before starting the kueue controller. +Defaults to false; therefore, those jobs are not managed and if they are created +unsuspended, they will start immediately. + |
+
internalCertManagement [Required]+ InternalCertManagement
+ |
+
+ InternalCertManagement is configuration for internalCertManagement + |
+
waitForPodsReady [Required]+ WaitForPodsReady
+ |
+
+ WaitForPodsReady is configuration to provide simple all-or-nothing +scheduling semantics for jobs to ensure they get resources assigned. +This is achieved by blocking the start of new jobs until the previously +started job has all pods running (ready). + |
+
clientConnection [Required]+ ClientConnection
+ |
+
+ ClientConnection provides additional configuration options for Kubernetes +API server client. + |
+
integrations [Required]+ Integrations
+ |
+
+ Integrations provide configuration options for AI/ML/Batch frameworks +integrations (including K8S job). + |
+
queueVisibility [Required]+ QueueVisibility
+ |
+
+ QueueVisibility is configuration to expose the information about the top +pending workloads. + |
+
ControllerConfigurationSpec defines the global configuration for +controllers registered with the manager.
+ + +| Field | Description |
|---|---|
groupKindConcurrency+ map[string]int
+ |
+
+ GroupKindConcurrency is a map from a Kind to the number of concurrent reconciliation +allowed for that controller. +When a controller is registered within this manager using the builder utilities, +users have to specify the type the controller reconciles in the For(...) call. +If the object's kind passed matches one of the keys in this map, the concurrency +for that controller is set to the number specified. +The key is expected to be consistent in form with GroupKind.String(),
+e.g. ReplicaSet in apps group (regardless of version) would be |
+
cacheSyncTimeout+ time.Duration
+ |
+
+ CacheSyncTimeout refers to the time limit set to wait for syncing caches. +Defaults to 2 minutes if not set. + |
+
ControllerHealth defines the health configs.
+ + +| Field | Description |
|---|---|
healthProbeBindAddress+ string
+ |
+
+ HealthProbeBindAddress is the TCP address that the controller should bind to +for serving health probes +It can be set to "0" or "" to disable serving the health probe. + |
+
readinessEndpointName+ string
+ |
+
+ ReadinessEndpointName, defaults to "readyz" + |
+
livenessEndpointName+ string
+ |
+
+ LivenessEndpointName, defaults to "healthz" + |
+
| Field | Description |
|---|---|
webhook+ ControllerWebhook
+ |
+
+ Webhook contains the controllers webhook configuration + |
+
leaderElection+ k8s.io/component-base/config/v1alpha1.LeaderElectionConfiguration
+ |
+
+ LeaderElection is the LeaderElection config to be used when configuring +the manager.Manager leader election + |
+
metrics+ ControllerMetrics
+ |
+
+ Metrics contains the controller metrics configuration + |
+
health+ ControllerHealth
+ |
+
+ Health contains the controller health configuration + |
+
pprofBindAddress+ string
+ |
+
+ PprofBindAddress is the TCP address that the controller should bind to +for serving pprof. +It can be set to "" or "0" to disable the pprof serving. +Since pprof may contain sensitive information, make sure to protect it +before exposing it to public. + |
+
controller+ ControllerConfigurationSpec
+ |
+
+ Controller contains global configuration options for controllers +registered within this manager. + |
+
ControllerMetrics defines the metrics configs.
+ + +| Field | Description |
|---|---|
bindAddress+ string
+ |
+
+ BindAddress is the TCP address that the controller should bind to +for serving prometheus metrics. +It can be set to "0" to disable the metrics serving. + |
+
enableClusterQueueResources+ bool
+ |
+
+ EnableClusterQueueResources, if true the cluster queue resource usage and quotas +metrics will be reported. + |
+
ControllerWebhook defines the webhook server for the controller.
+ + +| Field | Description |
|---|---|
port+ int
+ |
+
+ Port is the port that the webhook server serves at. +It is used to set webhook.Server.Port. + |
+
host+ string
+ |
+
+ Host is the hostname that the webhook server binds to. +It is used to set webhook.Server.Host. + |
+
certDir+ string
+ |
+
+ CertDir is the directory that contains the server key and certificate. +if not set, webhook server would look up the server key and certificate in +{TempDir}/k8s-webhook-server/serving-certs. The server key and certificate +must be named tls.key and tls.crt, respectively. + |
+
| Field | Description |
|---|---|
frameworks [Required]+ []string
+ |
+
+ List of framework names to be enabled. +Possible options: +
|
+
podOptions [Required]+ PodIntegrationOptions
+ |
+
+ PodOptions defines kueue controller behaviour for pod objects + |
+
| Field | Description |
|---|---|
enable [Required]+ bool
+ |
+
+ Enable controls whether to enable internal cert management or not. +Defaults to true. If you want to use a third-party management, e.g. cert-manager, +set it to false. See the user guide for more information. + |
+
webhookServiceName [Required]+ string
+ |
+
+ WebhookServiceName is the name of the Service used as part of the DNSName. +Defaults to kueue-webhook-service. + |
+
webhookSecretName [Required]+ string
+ |
+
+ WebhookSecretName is the name of the Secret used to store CA and server certs. +Defaults to kueue-webhook-server-cert. + |
+
| Field | Description |
|---|---|
namespaceSelector [Required]+ k8s.io/apimachinery/pkg/apis/meta/v1.LabelSelector
+ |
+
+ NamespaceSelector can be used to omit some namespaces from pod reconciliation + |
+
podSelector [Required]+ k8s.io/apimachinery/pkg/apis/meta/v1.LabelSelector
+ |
+
+ PodSelector can be used to choose what pods to reconcile + |
+
| Field | Description |
|---|---|
clusterQueues [Required]+ ClusterQueueVisibility
+ |
+
+ ClusterQueues is configuration to expose the information +about the top pending workloads in the cluster queue. + |
+
updateIntervalSeconds [Required]+ int32
+ |
+
+ UpdateIntervalSeconds specifies the time interval for updates to the structure +of the top pending workloads in the queues. +The minimum value is 1. +Defaults to 5. + |
+
| Field | Description |
|---|---|
enable [Required]+ bool
+ |
+
+ Enable when true, indicates that each admitted workload
+blocks the admission of all other workloads from all queues until it is in the
+ |
+
timeout+ k8s.io/apimachinery/pkg/apis/meta/v1.Duration
+ |
+
+ Timeout defines the time for an admitted workload to reach the +PodsReady=true condition. When the timeout is reached, the workload admission +is cancelled and requeued in the same cluster queue. Defaults to 5min. + |
+
blockAdmission [Required]+ bool
+ |
+
+ BlockAdmission when true, cluster queue will block admissions for all subsequent jobs +until the jobs reach the PodsReady=true condition. It defaults to false if Enable is false +and defaults to true otherwise. + |
+
AdmissionCheck is the Schema for the admissionchecks API
+ + +| Field | Description |
|---|---|
apiVersionstring | kueue.x-k8s.io/v1beta1 |
kindstring | AdmissionCheck |
spec [Required]+ AdmissionCheckSpec
+ |
++ No description provided. | +
status [Required]+ AdmissionCheckStatus
+ |
++ No description provided. | +
ClusterQueue is the Schema for the clusterQueue API.
+ + +| Field | Description |
|---|---|
apiVersionstring | kueue.x-k8s.io/v1beta1 |
kindstring | ClusterQueue |
spec [Required]+ ClusterQueueSpec
+ |
++ No description provided. | +
status [Required]+ ClusterQueueStatus
+ |
++ No description provided. | +
LocalQueue is the Schema for the localQueues API
+ + +| Field | Description |
|---|---|
apiVersionstring | kueue.x-k8s.io/v1beta1 |
kindstring | LocalQueue |
spec [Required]+ LocalQueueSpec
+ |
++ No description provided. | +
status [Required]+ LocalQueueStatus
+ |
++ No description provided. | +
ProvisioningRequestConfig is the Schema for the provisioningrequestconfig API
+ + +| Field | Description |
|---|---|
apiVersionstring | kueue.x-k8s.io/v1beta1 |
kindstring | ProvisioningRequestConfig |
spec [Required]+ ProvisioningRequestConfigSpec
+ |
++ No description provided. | +
ResourceFlavor is the Schema for the resourceflavors API.
+ + +| Field | Description |
|---|---|
apiVersionstring | kueue.x-k8s.io/v1beta1 |
kindstring | ResourceFlavor |
spec [Required]+ ResourceFlavorSpec
+ |
++ No description provided. | +
Workload is the Schema for the workloads API
+ + +| Field | Description |
|---|---|
apiVersionstring | kueue.x-k8s.io/v1beta1 |
kindstring | Workload |
spec [Required]+ WorkloadSpec
+ |
++ No description provided. | +
status [Required]+ WorkloadStatus
+ |
++ No description provided. | +
WorkloadPriorityClass is the Schema for the workloadPriorityClass API
+ + +| Field | Description |
|---|---|
apiVersionstring | kueue.x-k8s.io/v1beta1 |
kindstring | WorkloadPriorityClass |
value [Required]+ int32
+ |
+
+ value represents the integer value of this workloadPriorityClass. This is the actual priority that workloads +receive when jobs have the name of this class in their workloadPriorityClass label. +Changing the value of workloadPriorityClass doesn't affect the priority of workloads that were already created. + |
+
description+ string
+ |
+
+ description is an arbitrary string that usually provides guidelines on +when this workloadPriorityClass should be used. + |
+
| Field | Description |
|---|---|
clusterQueue [Required]+ ClusterQueueReference
+ |
+
+ clusterQueue is the name of the ClusterQueue that admitted this workload. + |
+
podSetAssignments [Required]+ []PodSetAssignment
+ |
+
+ PodSetAssignments hold the admission results for each of the .spec.podSets entries. + |
+
| Field | Description |
|---|---|
apiGroup [Required]+ string
+ |
+
+ ApiGroup is the group for the resource being referenced. + |
+
kind [Required]+ string
+ |
+
+ Kind is the type of the resource being referenced. + |
+
name [Required]+ string
+ |
+
+ Name is the name of the resource being referenced. + |
+
AdmissionCheckSpec defines the desired state of AdmissionCheck
+ + +| Field | Description |
|---|---|
controllerName [Required]+ string
+ |
+
+ controllerName is name of the controller which will actually perform +the checks. This is the name with which controller identifies with, +not necessarily a K8S Pod or Deployment name. Cannot be empty. + |
+
retryDelayMinutes+ int64
+ |
+
+ RetryDelayMinutes specifies how long to keep the workload suspended +after a failed check (after it transitioned to False). +After that the check state goes to "Unknown". +The default is 15 min. + |
+
parameters+ AdmissionCheckParametersReference
+ |
+
+ Parameters identifies the resource providing additional check parameters. + |
+
| Field | Description |
|---|---|
name [Required]+ string
+ |
+
+ name identifies the admission check. + |
+
state [Required]+ CheckState
+ |
+
+ status of the condition, one of True, False, Unknown. + |
+
lastTransitionTime [Required]+ k8s.io/apimachinery/pkg/apis/meta/v1.Time
+ |
+
+ lastTransitionTime is the last time the condition transitioned from one status to another. +This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + |
+
message [Required]+ string
+ |
+
+ message is a human readable message indicating details about the transition. +This may be an empty string. + |
+
podSetUpdates+ []PodSetUpdate
+ |
++ No description provided. | +
AdmissionCheckStatus defines the observed state of AdmissionCheck
+ + +| Field | Description |
|---|---|
conditions+ []k8s.io/apimachinery/pkg/apis/meta/v1.Condition
+ |
+
+ conditions hold the latest available observations of the AdmissionCheck +current state. + |
+
ClusterQueuePendingWorkload contains the information identifying a pending workload +in the cluster queue.
+ + +| Field | Description |
|---|---|
name [Required]+ string
+ |
+
+ Name indicates the name of the pending workload. + |
+
namespace [Required]+ string
+ |
+
+ Namespace indicates the name of the pending workload. + |
+
| Field | Description |
|---|---|
clusterQueuePendingWorkload+ []ClusterQueuePendingWorkload
+ |
+
+ Head contains the list of top pending workloads. + |
+
lastChangeTime [Required]+ k8s.io/apimachinery/pkg/apis/meta/v1.Time
+ |
+
+ LastChangeTime indicates the time of the last change of the structure. + |
+
ClusterQueuePreemption contains policies to preempt Workloads from this +ClusterQueue or the ClusterQueue's cohort.
+ + +| Field | Description |
|---|---|
reclaimWithinCohort [Required]+ PreemptionPolicy
+ |
+
+ reclaimWithinCohort determines whether a pending Workload can preempt +Workloads from other ClusterQueues in the cohort that are using more than +their nominal quota. The possible values are: +
|
+
withinClusterQueue [Required]+ PreemptionPolicy
+ |
+
+ withinClusterQueue determines whether a pending Workload that doesn't fit +within the nominal quota for its ClusterQueue, can preempt active Workloads in +the ClusterQueue. The possible values are: +
|
+
ClusterQueueReference is the name of the ClusterQueue.
+ + + + +## `ClusterQueueSpec` {#kueue-x-k8s-io-v1beta1-ClusterQueueSpec} + + +**Appears in:** + +- [ClusterQueue](#kueue-x-k8s-io-v1beta1-ClusterQueue) + + +ClusterQueueSpec defines the desired state of ClusterQueue
+ + +| Field | Description |
|---|---|
resourceGroups [Required]+ []ResourceGroup
+ |
+
+ resourceGroups describes groups of resources. +Each resource group defines the list of resources and a list of flavors +that provide quotas for these resources. +Each resource and each flavor can only form part of one resource group. +resourceGroups can be up to 16. + |
+
cohort [Required]+ string
+ |
+
+ cohort that this ClusterQueue belongs to. CQs that belong to the +same cohort can borrow unused resources from each other. +A CQ can be a member of a single borrowing cohort. A workload submitted +to a queue referencing this CQ can borrow quota from any CQ in the cohort. +Only quota for the [resource, flavor] pairs listed in the CQ can be +borrowed. +If empty, this ClusterQueue cannot borrow from any other ClusterQueue and +vice versa. +A cohort is a name that links CQs together, but it doesn't reference any +object. +Validation of a cohort name is equivalent to that of object names: +subdomain in DNS (RFC 1123). + |
+
queueingStrategy [Required]+ QueueingStrategy
+ |
+
+ QueueingStrategy indicates the queueing strategy of the workloads +across the queues in this ClusterQueue. This field is immutable. +Current Supported Strategies: +
|
+
namespaceSelector [Required]+ k8s.io/apimachinery/pkg/apis/meta/v1.LabelSelector
+ |
+
+ namespaceSelector defines which namespaces are allowed to submit workloads to
+this clusterQueue. Beyond this basic support for policy, an policy agent like
+Gatekeeper should be used to enforce more advanced policies.
+Defaults to null which is a nothing selector (no namespaces eligible).
+If set to an empty selector |
+
flavorFungibility [Required]+ FlavorFungibility
+ |
+
+ flavorFungibility defines whether a workload should try the next flavor +before borrowing or preempting in the flavor being evaluated. + |
+
preemption [Required]+ ClusterQueuePreemption
+ |
+
+ preemption describes policies to preempt Workloads from this ClusterQueue +or the ClusterQueue's cohort. +Preemption can happen in two scenarios: +
The preemption algorithm tries to find a minimal set of Workloads to +preempt to accomomdate the pending Workload, preempting Workloads with +lower priority first. + |
+
admissionChecks+ []string
+ |
+
+ admissionChecks lists the AdmissionChecks required by this ClusterQueue + |
+
ClusterQueueStatus defines the observed state of ClusterQueue
+ + +| Field | Description |
|---|---|
flavorsReservation+ []FlavorUsage
+ |
+
+ flavorsReservation are the reserved quotas, by flavor, currently in use by the +workloads assigned to this ClusterQueue. + |
+
flavorsUsage+ []FlavorUsage
+ |
+
+ flavorsUsage are the used quotas, by flavor, currently in use by the +workloads admitted in this ClusterQueue. + |
+
pendingWorkloads+ int32
+ |
+
+ pendingWorkloads is the number of workloads currently waiting to be +admitted to this clusterQueue. + |
+
reservingWorkloads+ int32
+ |
+
+ reservingWorkloads is the number of workloads currently reserving quota in this +clusterQueue. + |
+
admittedWorkloads+ int32
+ |
+
+ admittedWorkloads is the number of workloads currently admitted to this +clusterQueue and haven't finished yet. + |
+
conditions+ []k8s.io/apimachinery/pkg/apis/meta/v1.Condition
+ |
+
+ conditions hold the latest available observations of the ClusterQueue +current state. + |
+
pendingWorkloadsStatus+ ClusterQueuePendingWorkloadsStatus
+ |
+
+ PendingWorkloadsStatus contains the information exposed about the current +status of the pending workloads in the cluster queue. + |
+
FlavorFungibility determines whether a workload should try the next flavor +before borrowing or preempting in current flavor.
+ + +| Field | Description |
|---|---|
whenCanBorrow [Required]+ FlavorFungibilityPolicy
+ |
+
+ whenCanBorrow determines whether a workload should try the next flavor +before borrowing in current flavor. The possible values are: +
|
+
whenCanPreempt [Required]+ FlavorFungibilityPolicy
+ |
+
+ whenCanPreempt determines whether a workload should try the next flavor +before borrowing in current flavor. The possible values are: +
|
+
| Field | Description |
|---|---|
name [Required]+ ResourceFlavorReference
+ |
+
+ name of this flavor. The name should match the .metadata.name of a +ResourceFlavor. If a matching ResourceFlavor does not exist, the +ClusterQueue will have an Active condition set to False. + |
+
resources [Required]+ []ResourceQuota
+ |
+
+ resources is the list of quotas for this flavor per resource. +There could be up to 16 resources. + |
+
| Field | Description |
|---|---|
name [Required]+ ResourceFlavorReference
+ |
+
+ name of the flavor. + |
+
resources [Required]+ []ResourceUsage
+ |
+
+ resources lists the quota usage for the resources in this flavor. + |
+
| Field | Description |
|---|---|
name [Required]+ ResourceFlavorReference
+ |
+
+ name of the flavor. + |
+
resources [Required]+ []LocalQueueResourceUsage
+ |
+
+ resources lists the quota usage for the resources in this flavor. + |
+
| Field | Description |
|---|---|
name [Required]+ k8s.io/api/core/v1.ResourceName
+ |
+
+ name of the resource. + |
+
total [Required]+ k8s.io/apimachinery/pkg/api/resource.Quantity
+ |
+
+ total is the total quantity of used quota. + |
+
LocalQueueSpec defines the desired state of LocalQueue
+ + +| Field | Description |
|---|---|
clusterQueue [Required]+ ClusterQueueReference
+ |
+
+ clusterQueue is a reference to a clusterQueue that backs this localQueue. + |
+
LocalQueueStatus defines the observed state of LocalQueue
+ + +| Field | Description |
|---|---|
pendingWorkloads+ int32
+ |
+
+ PendingWorkloads is the number of Workloads in the LocalQueue not yet admitted to a ClusterQueue + |
+
reservingWorkloads+ int32
+ |
+
+ reservingWorkloads is the number of workloads in this LocalQueue +reserving quota in a ClusterQueue and that haven't finished yet. + |
+
admittedWorkloads+ int32
+ |
+
+ admittedWorkloads is the number of workloads in this LocalQueue +admitted to a ClusterQueue and that haven't finished yet. + |
+
conditions+ []k8s.io/apimachinery/pkg/apis/meta/v1.Condition
+ |
+
+ Conditions hold the latest available observations of the LocalQueue +current state. + |
+
flavorsReservation+ []LocalQueueFlavorUsage
+ |
+
+ flavorsReservation are the reserved quotas, by flavor currently in use by the +workloads assigned to this LocalQueue. + |
+
flavorUsage+ []LocalQueueFlavorUsage
+ |
+
+ flavorsUsage are the used quotas, by flavor currently in use by the +workloads assigned to this LocalQueue. + |
+
Parameter is limited to 255 characters.
+ + + + +## `PodSet` {#kueue-x-k8s-io-v1beta1-PodSet} + + +**Appears in:** + +- [WorkloadSpec](#kueue-x-k8s-io-v1beta1-WorkloadSpec) + + + +| Field | Description |
|---|---|
name [Required]+ string
+ |
+
+ name is the PodSet name. + |
+
template [Required]+ k8s.io/api/core/v1.PodTemplateSpec
+ |
+
+ template is the Pod template. +The only allowed fields in template.metadata are labels and annotations. +If requests are omitted for a container or initContainer, +they default to the limits if they are explicitly specified for the +container or initContainer. +During admission, the rules in nodeSelector and +nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution that match +the keys in the nodeLabels from the ResourceFlavors considered for this +Workload are used to filter the ResourceFlavors that can be assigned to +this podSet. + |
+
count [Required]+ int32
+ |
+
+ count is the number of pods for the spec. + |
+
minCount+ int32
+ |
+
+ minCount is the minimum number of pods for the spec acceptable +if the workload supports partial admission. +If not provided, partial admission for the current PodSet is not +enabled. +Only one podSet within the workload can use this. +This is an alpha field and requires enabling PartialAdmission feature gate. + |
+
| Field | Description |
|---|---|
name [Required]+ string
+ |
+
+ Name is the name of the podSet. It should match one of the names in .spec.podSets. + |
+
flavors [Required]+ map[k8s.io/api/core/v1.ResourceName]../../apis/kueue/v1beta1.ResourceFlavorReference
+ |
+
+ Flavors are the flavors assigned to the workload for each resource. + |
+
resourceUsage [Required]+ k8s.io/api/core/v1.ResourceList
+ |
+
+ resourceUsage keeps track of the total resources all the pods in the podset need to run. +Beside what is provided in podSet's specs, this calculation takes into account +the LimitRange defaults and RuntimeClass overheads at the moment of admission. +This field will not change in case of quota reclaim. + |
+
count+ int32
+ |
+
+ count is the number of pods taken into account at admission time. +This field will not change in case of quota reclaim. +Value could be missing for Workloads created before this field was added, +in that case spec.podSets[*].count value will be used. + |
+
PodSetUpdate contains a list of pod set modifications suggested by AdmissionChecks. +The modifications should be additive only - modifications of already existing keys +or having the same key provided by multiple AdmissionChecks is not allowed and will +result in failure during workload admission.
+ + +| Field | Description |
|---|---|
name [Required]+ string
+ |
+
+ Name of the PodSet to modify. Should match to one of the Workload's PodSets. + |
+
labels+ map[string]string
+ |
++ No description provided. | +
annotations+ map[string]string
+ |
++ No description provided. | +
nodeSelector+ map[string]string
+ |
++ No description provided. | +
tolerations+ []k8s.io/api/core/v1.Toleration
+ |
++ No description provided. | +
ProvisioningRequestConfigSpec defines the desired state of ProvisioningRequestConfig
+ + +| Field | Description |
|---|---|
provisioningClassName [Required]+ string
+ |
+
+ ProvisioningClassName describes the different modes of provisioning the resources. +Check autoscaling.x-k8s.io ProvisioningRequestSpec.ProvisioningClassName for details. + |
+
parameters+ map[string]../../apis/kueue/v1beta1.Parameter
+ |
+
+ Parameters contains all other parameters classes may require. + |
+
managedResources+ []k8s.io/api/core/v1.ResourceName
+ |
+
+ managedResources contains the list of resources managed by the autoscaling. +If empty, all resources are considered managed. +If not empty, the ProvisioningRequest will contain only the podsets that are +requesting at least one of them. +If none of the workloads podsets is requesting at least a managed resource, +the workload is considered ready. + |
+
| Field | Description |
|---|---|
name [Required]+ string
+ |
+
+ name is the PodSet name. + |
+
count [Required]+ int32
+ |
+
+ count is the number of pods for which the requested resources are no longer needed. + |
+
ResourceFlavorReference is the name of the ResourceFlavor.
+ + + + +## `ResourceFlavorSpec` {#kueue-x-k8s-io-v1beta1-ResourceFlavorSpec} + + +**Appears in:** + +- [ResourceFlavor](#kueue-x-k8s-io-v1beta1-ResourceFlavor) + + +ResourceFlavorSpec defines the desired state of the ResourceFlavor
+ + +| Field | Description |
|---|---|
nodeLabels+ map[string]string
+ |
+
+ nodeLabels are labels that associate the ResourceFlavor with Nodes that +have the same labels. +When a Workload is admitted, its podsets can only get assigned +ResourceFlavors whose nodeLabels match the nodeSelector and nodeAffinity +fields. +Once a ResourceFlavor is assigned to a podSet, the ResourceFlavor's +nodeLabels should be injected into the pods of the Workload by the +controller that integrates with the Workload object. +nodeLabels can be up to 8 elements. + |
+
nodeTaints+ []k8s.io/api/core/v1.Taint
+ |
+
+ nodeTaints are taints that the nodes associated with this ResourceFlavor +have. +Workloads' podsets must have tolerations for these nodeTaints in order to +get assigned this ResourceFlavor during admission. +An example of a nodeTaint is +cloud.provider.com/preemptible="true":NoSchedule +nodeTaints can be up to 8 elements. + |
+
tolerations+ []k8s.io/api/core/v1.Toleration
+ |
+
+ tolerations are extra tolerations that will be added to the pods admitted in +the quota associated with this resource flavor. +An example of a toleration is +cloud.provider.com/preemptible="true":NoSchedule +tolerations can be up to 8 elements. + |
+
| Field | Description |
|---|---|
coveredResources [Required]+ []k8s.io/api/core/v1.ResourceName
+ |
+
+ coveredResources is the list of resources covered by the flavors in this +group. +Examples: cpu, memory, vendor.com/gpu. +The list cannot be empty and it can contain up to 16 resources. + |
+
flavors [Required]+ []FlavorQuotas
+ |
+
+ flavors is the list of flavors that provide the resources of this group. +Typically, different flavors represent different hardware models +(e.g., gpu models, cpu architectures) or pricing models (on-demand vs spot +cpus). +Each flavor MUST list all the resources listed for this group in the same +order as the .resources field. +The list cannot be empty and it can contain up to 16 flavors. + |
+
| Field | Description |
|---|---|
name [Required]+ k8s.io/api/core/v1.ResourceName
+ |
+
+ name of this resource. + |
+
nominalQuota [Required]+ k8s.io/apimachinery/pkg/api/resource.Quantity
+ |
+
+ nominalQuota is the quantity of this resource that is available for +Workloads admitted by this ClusterQueue at a point in time. +The nominalQuota must be non-negative. +nominalQuota should represent the resources in the cluster available for +running jobs (after discounting resources consumed by system components +and pods not managed by kueue). In an autoscaled cluster, nominalQuota +should account for resources that can be provided by a component such as +Kubernetes cluster-autoscaler. +If the ClusterQueue belongs to a cohort, the sum of the quotas for each +(flavor, resource) combination defines the maximum quantity that can be +allocated by a ClusterQueue in the cohort. + |
+
borrowingLimit+ k8s.io/apimachinery/pkg/api/resource.Quantity
+ |
+
+ borrowingLimit is the maximum amount of quota for the [flavor, resource] +combination that this ClusterQueue is allowed to borrow from the unused +quota of other ClusterQueues in the same cohort. +In total, at a given time, Workloads in a ClusterQueue can consume a +quantity of quota equal to nominalQuota+borrowingLimit, assuming the other +ClusterQueues in the cohort have enough unused quota. +If null, it means that there is no borrowing limit. +If not null, it must be non-negative. +borrowingLimit must be null if spec.cohort is empty. + |
+
| Field | Description |
|---|---|
name [Required]+ k8s.io/api/core/v1.ResourceName
+ |
+
+ name of the resource + |
+
total [Required]+ k8s.io/apimachinery/pkg/api/resource.Quantity
+ |
+
+ total is the total quantity of used quota, including the amount borrowed +from the cohort. + |
+
borrowed [Required]+ k8s.io/apimachinery/pkg/api/resource.Quantity
+ |
+
+ Borrowed is quantity of quota that is borrowed from the cohort. In other +words, it's the used quota that is over the nominalQuota. + |
+
WorkloadSpec defines the desired state of Workload
+ + +| Field | Description |
|---|---|
podSets [Required]+ []PodSet
+ |
+
+ podSets is a list of sets of homogeneous pods, each described by a Pod spec +and a count. +There must be at least one element and at most 8. +podSets cannot be changed. + |
+
queueName [Required]+ string
+ |
+
+ queueName is the name of the LocalQueue the Workload is associated with. +queueName cannot be changed while .status.admission is not null. + |
+
priorityClassName [Required]+ string
+ |
+
+ If specified, indicates the workload's priority. +"system-node-critical" and "system-cluster-critical" are two special +keywords which indicate the highest priorities with the former being +the highest priority. Any other name must be defined by creating a +PriorityClass object with that name. If not specified, the workload +priority will be default or zero if there is no default. + |
+
priority [Required]+ int32
+ |
+
+ Priority determines the order of access to the resources managed by the +ClusterQueue where the workload is queued. +The priority value is populated from PriorityClassName. +The higher the value, the higher the priority. +If priorityClassName is specified, priority must not be null. + |
+
priorityClassSource [Required]+ string
+ |
+
+ priorityClassSource determines whether the priorityClass field refers to a pod PriorityClass or kueue.x-k8s.io/workloadpriorityclass. +Workload's PriorityClass can accept the name of a pod priorityClass or a workloadPriorityClass. +When using pod PriorityClass, a priorityClassSource field has the scheduling.k8s.io/priorityclass value. + |
+
WorkloadStatus defines the observed state of Workload
+ + +| Field | Description |
|---|---|
admission [Required]+ Admission
+ |
+
+ admission holds the parameters of the admission of the workload by a +ClusterQueue. admission can be set back to null, but its fields cannot be +changed once set. + |
+
conditions+ []k8s.io/apimachinery/pkg/apis/meta/v1.Condition
+ |
+
+ conditions hold the latest available observations of the Workload +current state. +The type of the condition could be: +
|
+
reclaimablePods+ []ReclaimablePod
+ |
+
+ reclaimablePods keeps track of the number pods within a podset for which +the resource reservation is no longer needed. + |
+
admissionChecks+ []AdmissionCheckState
+ |
+
+ admissionChecks list all the admission checks required by the workload and the current status + |
+
{{ .FieldName }}
+ {{- if not .IsOptional }} [Required]{{- end -}}
+{{ .DisplayName }}
+ {{- else -}}
+{{ .DisplayName }}
+ {{- end -}}
+ {{- end }}
+{{ .FieldName }} are embedded into this type.)
+ {{- end }}
+ {{ if .GetComment -}}
+ {{ .GetComment }}
+ {{- else -}}
+ No description provided.
+ {{- end }}
+ {{- if and (eq (.GetType.Name.Name) "ObjectMeta") -}}
+Refer to the Kubernetes API documentation for the fields of the metadata field.
+ {{- end -}}
+| Field | Description |
|---|---|
apiVersionstring | {{- .APIGroup -}} |
kindstring | {{- .Name.Name -}} |