-
Notifications
You must be signed in to change notification settings - Fork 16.3k
Adjust ray operators in google provider. Add ray package as a dependency. #53182
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Is not that contradicitng that ray package is >= 2.42 and that supported ray versions are 2.9.3, 2.33, 2.42 ? |
@potiuk I tested it with ray version 2.47(the latest) in breeze and it works ok. The only warning from vertex ai that it is better to have the same library. |
10dc080 to
d77b636
Compare
|
@potiuk |
providers/google/src/airflow/providers/google/cloud/operators/vertex_ai/ray.py
Show resolved
Hide resolved
providers/google/src/airflow/providers/google/cloud/operators/vertex_ai/ray.py
Show resolved
Hide resolved
…ncy. (apache#53182) Co-authored-by: Anton Nitochkin <nitochkin@google.com>
| Please note that you need to specify python_version and ray_version in :class:`~airflow.providers.google.cloud.operators.vertex_ai.ray.CreateRayClusterOperator`. | ||
| Currently supported versions of ray package in ray cluster are: 2.9.3, 2.33, 2.42. | ||
| For more information you can check: https://github.com/googleapis/python-aiplatform/blob/main/setup.py#L101 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't like that our docs reference to internal details of other packages - especially when we are referencing main branch where any commit can make this link not accurate. doesn't the library serve a user facing doc that we can reference instead?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As I understand, this is specifically for users to see the limitations of the package they are trying to use, because without this it will be hard to determine what is wrong if the user will use the operator with unsupported versions of Ray.
For the link to the comment I agree, maybe better to refer to it in another way, more flexible one
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The package has doc it looks like they just don't document which ray version they support. Might worth asking them to do so?
The sentence of
Currently supported versions of ray package in ray cluster are: 2.9.3, 2.33, 2.42.
For more information you can check: https://github.com/googleapis/python-aiplatform/blob/main/setup.py#L101
Should be something like:
To read more about compatible versions of Ray check the Google docs in <Link to upstream doc>
This avoids the need to update our docs for every change upstream library makes.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in airflow-core/newsfragments.