-
Notifications
You must be signed in to change notification settings - Fork 16.3k
Description
Apache Airflow Provider(s)
cncf-kubernetes
Versions of Apache Airflow Providers
apache-airflow-providers-cncf-kubernetes==10.9.0
Apache Airflow version
2.10.5
Operating System
linux
Deployment
Google Cloud Composer
Deployment details
composer-2.16.1-airflow-2.10.5
What happened
When using GKEStartJobOperator the job name that is generated doesn't strip _ characters, leading to a kubernetes server error response:
[2026-01-07, 11:54:30 EST] {kubernetes.py:568} ERROR - Exception when attempting to create Namespaced Job:
{
"apiVersion": "batch/v1",
"kind": "Job",
"metadata": {
"annotations": {},
"labels": {},
"name": "job-cloud_composer_task_export_answers_stats-3gxv44tg",
"namespace": "data-science"
},
}
// actual error
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {},
"status": "Failure",
"message": "Job.batch \"job-cloud_composer_task_export_answers_stats-3gxv44tg\" is invalid: metadata.name: Invalid value: \"job-cloud_composer_task_export_answers_stats-3gxv44tg\": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')",
"reason": "Invalid",
"details": {
"name": "job-cloud_composer_task_export_answers_stats-3gxv44tg",
"group": "batch",
"kind": "Job",
"causes": [
{
"reason": "FieldValueInvalid",
"message": "Invalid value: \"job-cloud_composer_task_export_answers_stats-3gxv44tg\": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')",
"field": "metadata.name"
}
]
},
"code": 422
}
The regex provided in the stacktrace:
[a-z0-9]([-a-z0-9]*[a-z0-9])?(\\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')
Doesn't validate the job name:
job-cloud_composer_task_export_answers_stats-3gxv44tg
Our task names use underscores as it's the python convetion
What you think should happen instead
the pod operator
airflow/providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/operators/pod.py
Line 671 in 2daadf4
| self.name = self._set_name(self.name) |
calls _set_name which replaces invalid chars by -
but the job operator
airflow/providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/operators/job.py
Line 190 in 2daadf4
| def execute(self, context: Context): |
doesn't call set name, leading to invalid names being preserved.
The actual names come from this function:
Line 125 in 2daadf4
| def create_unique_id( |
How to reproduce
Use GKEStartJobOperator or KubernetesStartJobOperator with a task id containing a _
Anything else
I'm willing to submit a fix and I'm already prepping one
Are you willing to submit PR?
- Yes I am willing to submit a PR! Fix job_name normalization in KubernetesJobOperator #60231
Code of Conduct
- I agree to follow this project's Code of Conduct