Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Google Cloud Generative Model Hooks/Operators to bring parity with Vertex AI API #40484

Merged
merged 8 commits into from
Jul 4, 2024
Merged

Update Google Cloud Generative Model Hooks/Operators to bring parity with Vertex AI API #40484

merged 8 commits into from
Jul 4, 2024

Conversation

CYarros10
Copy link
Contributor

With the constantly evolving Vertex AI APIs for generative models, this PR updates the generative_model hooks/operators in the following ways:

  • naming conventions in line with Vertex AI APIs
  • consolidation of multimodal operators into a single GenerativeModelGenerateContentOperator
  • addition of contents and tools parameters for greater flexibility interacting with Vertex AI APIs
  • docs updates
  • provider dependency upgrade for aiplatform package

for more information visit: https://cloud.google.com/vertex-ai/generative-ai/docs/start/quickstarts/quickstart-multimodal


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

Copy link
Contributor

@shahar1 shahar1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First of all, thank you for your contribution!
Overall - it looks promising, as adapting parity with the latest API and naming conventions is important.
However, I'm quite worried that the proposed changes in their current form are too overwhelming, in terms of breaking changes. If someone updates their GCP provider after having these changes in the new version, they will have no clue where operators and hooks are suddenly gone without prior notice.
I suggest making the transition more gradual by keeping all the existing hooks/operators and parameters and adding deprecation warnings where applicable (there are examples within Airflow's source code, even within the GCP provider itself).
Also, I'd like @VladaZakharova to take a look, as her team has been working on Vertex AI operators recently.

@CYarros10
Copy link
Contributor Author

@shahar1 Thank you for your feedback. I agree with your points. I'll add a commit that retains the old operators/hooks and include deprecation warnings.

@potiuk
Copy link
Member

potiuk commented Jul 2, 2024

conflicts

@potiuk
Copy link
Member

potiuk commented Jul 4, 2024

And again :( conflicts

@potiuk potiuk merged commit 9c97067 into apache:main Jul 4, 2024
107 of 108 checks passed
romsharon98 pushed a commit to romsharon98/airflow that referenced this pull request Jul 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants