Fix ModuleNotFoundError in kubernetes cleanup cronjob#61673
Merged
shahar1 merged 3 commits intoapache:mainfrom Feb 10, 2026
Merged
Fix ModuleNotFoundError in kubernetes cleanup cronjob#61673shahar1 merged 3 commits intoapache:mainfrom
shahar1 merged 3 commits intoapache:mainfrom
Conversation
The cleanup cronjob in apache-airflow-providers-cncf-kubernetes >= 10.12.2 fails in production because it imports from tests_common.test_utils.taskinstance, which is not available outside the dev repository. Solution: Remove test utility import and use TaskInstance directly, manually setting dag_version_id for Airflow 3.0+ compatibility.
Eason09053360
requested review from
hussein-awala,
jedcunningham and
jscheffl
as code owners
boring-cyborg
bot
added
area:providers
provider:cncf-kubernetes
The kubernetes cleanup cronjob command was importing from tests_common which is not available in production environments, causing ModuleNotFoundError when running the cronjob. Changes: - Import create_scheduler_operator conditionally within AIRFLOW_V_3_0_PLUS block to avoid import errors in older versions where it may not exist - Use create_scheduler_operator to serialize tasks before creating TaskInstance, ensuring compatibility with Airflow 3.0+ which requires serialized operators for proper TaskInstance initialization - Pass dag_version_id as constructor parameter (required in Airflow 3.0+) This ensures the cleanup cronjob works correctly in production Kubernetes environments running Airflow 3.0+.
Contributor
|
Looks good, just one back-compat test is failing, so 3.0.x-line probably needs another entry. Missing expertise... @amoghrajesh Do you have a hint what is also working in 3.0.0? Would be great fixing it before next provider wave tomorrow. |
Moved the try-except import of create_scheduler_operator from inside the for loop to the top-level import section. This follows Python best practices and improves performance by: - Executing the import only once instead of on every loop iteration - Making dependencies more visible at the top of the file - Matching the pattern used in tests_common.test_utils.taskinstance The fallback lambda ensures compatibility with Airflow versions where create_scheduler_operator doesn't exist (e.g., 3.0.6).
shahar1
added
the
priority:critical
1 task
amoghrajesh
reviewed
Feb 10, 2026
Contributor
amoghrajesh
left a comment
amoghrajesh
left a comment
There was a problem hiding this comment.
Thanks this looks good.
Contributor
|
Created #61713 with a pre-commit to avoid such cases for the next times |
81 tasks
Ratasa143
pushed a commit
to Ratasa143/airflow
that referenced
this pull request
Feb 15, 2026
|
Which release this fix will get into? |
Contributor
See https://airflow.apache.org/docs/apache-airflow-providers-cncf-kubernetes/stable/changelog.html - is released with 10.12.4 |
choo121600
pushed a commit
to choo121600/airflow
that referenced
this pull request
Feb 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The cleanup cronjob in apache-airflow-providers-cncf-kubernetes >= 10.12.2 fails in production because it imports from tests_common.test_utils.taskinstance, which is not available outside the dev repository.
Solution: Remove test utility import and use TaskInstance directly, manually setting dag_version_id for Airflow 3.0+ compatibility.
related: #60112
closed: #61360
Was generative AI tooling used to co-author this PR?
{pr_number}.significant.rstor{issue_number}.significant.rst, in airflow-core/newsfragments.