Skip to content

Conversation

@MaksYermak
Copy link
Contributor

In this PR I have prepared DAG for showing how to enable Ray on GKE cluster


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@boring-cyborg boring-cyborg bot added area:providers kind:documentation provider:google Google (including GCP) related issues labels May 27, 2025
@MaksYermak MaksYermak force-pushed the enable-ray-on-gke branch from 4fc0e65 to 141aacf Compare May 27, 2025 12:20
@eladkal eladkal force-pushed the enable-ray-on-gke branch from 141aacf to 6065bba Compare June 15, 2025 06:40
@eladkal eladkal changed the title Create DAG for showing how to enable Ray on GKE cluster docs: Add example how to enable Ray on GKE cluster Jun 15, 2025
@eladkal
Copy link
Contributor

eladkal commented Jun 15, 2025

Tests are failing:

FAILED airflow-core/tests/unit/always/test_example_dags.py::test_should_be_importable[providers/google/tests/system/google/cloud/kubernetes_engine/example_kubernetes_engine_ray.py] - AssertionError: import_errors={'/opt/airflow/providers/google/tests/system/google/cloud/kubernetes_engine/example_kubernetes_engine_ray.py': 'Traceback (most recent call last):\n  File "/opt/airflow/task-sdk/src/airflow/sdk/bases/operator.py", line 502, in apply_defaults\n    result = func(self, **kwargs, default_args=default_args)\n  File "/opt/airflow/task-sdk/src/airflow/sdk/bases/operator.py", line 1026, in __init__\n    raise TypeError(\nTypeError: Invalid arguments were passed to GKEDeleteClusterOperator (task_id: delete_cluster). Invalid arguments were:\n**kwargs: {\'name\': \'cluster-kubernetes-engine-ray-\'}\n'}
assert 1 == 0
 +  where 1 = len({'/opt/airflow/providers/google/tests/system/google/cloud/kubernetes_engine/example_kubernetes_engine_ray.py': 'Traceb...erator (task_id: delete_cluster). Invalid arguments were:\n**kwargs: {\'name\': \'cluster-kubernetes-engine-ray-\'}\n'})
 +    where {'/opt/airflow/providers/google/tests/system/google/cloud/kubernetes_engine/example_kubernetes_engine_ray.py': 'Traceb...erator (task_id: delete_cluster). Invalid arguments were:\n**kwargs: {\'name\': \'cluster-kubernetes-engine-ray-\'}\n'} = <airflow.models.dagbag.DagBag object at 0x7fa1eecf72b0>.import_errors

@MaksYermak
Copy link
Contributor Author

Tests are failing:

FAILED airflow-core/tests/unit/always/test_example_dags.py::test_should_be_importable[providers/google/tests/system/google/cloud/kubernetes_engine/example_kubernetes_engine_ray.py] - AssertionError: import_errors={'/opt/airflow/providers/google/tests/system/google/cloud/kubernetes_engine/example_kubernetes_engine_ray.py': 'Traceback (most recent call last):\n  File "/opt/airflow/task-sdk/src/airflow/sdk/bases/operator.py", line 502, in apply_defaults\n    result = func(self, **kwargs, default_args=default_args)\n  File "/opt/airflow/task-sdk/src/airflow/sdk/bases/operator.py", line 1026, in __init__\n    raise TypeError(\nTypeError: Invalid arguments were passed to GKEDeleteClusterOperator (task_id: delete_cluster). Invalid arguments were:\n**kwargs: {\'name\': \'cluster-kubernetes-engine-ray-\'}\n'}
assert 1 == 0
 +  where 1 = len({'/opt/airflow/providers/google/tests/system/google/cloud/kubernetes_engine/example_kubernetes_engine_ray.py': 'Traceb...erator (task_id: delete_cluster). Invalid arguments were:\n**kwargs: {\'name\': \'cluster-kubernetes-engine-ray-\'}\n'})
 +    where {'/opt/airflow/providers/google/tests/system/google/cloud/kubernetes_engine/example_kubernetes_engine_ray.py': 'Traceb...erator (task_id: delete_cluster). Invalid arguments were:\n**kwargs: {\'name\': \'cluster-kubernetes-engine-ray-\'}\n'} = <airflow.models.dagbag.DagBag object at 0x7fa1eecf72b0>.import_errors

@eladkal I have fixed the problem

@eladkal eladkal merged commit ee2235a into apache:main Jun 18, 2025
71 checks passed
RoyLee1224 pushed a commit to RoyLee1224/airflow that referenced this pull request Jun 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers kind:documentation provider:google Google (including GCP) related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants