Deprecate old VertexAI generative model operators and introduce new Generative AI on Vertex AI operators

airflow-core/tests/unit/always/test_project_structure.py

@@ -467,6 +467,12 @@ class TestGoogleProviderProjectStructure(ExampleCoverageTest, AssetsCoverageTest
         "airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogUpdateTagTemplateOperator",
         "airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogCreateEntryOperator",
         "airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogUpdateTagTemplateFieldOperator",
+        "airflow.providers.google.cloud.operators.vertex_ai.generative_model.GenerateFromCachedContentOperator",
+        "airflow.providers.google.cloud.operators.vertex_ai.generative_model.CreateCachedContentOperator",
+        "airflow.providers.google.cloud.operators.vertex_ai.generative_model.CountTokensOperator",
+        "airflow.providers.google.cloud.operators.vertex_ai.generative_model.SupervisedFineTuningTrainOperator",
+        "airflow.providers.google.cloud.operators.vertex_ai.generative_model.GenerativeModelGenerateContentOperator",
+        "airflow.providers.google.cloud.operators.vertex_ai.generative_model.TextEmbeddingModelGetEmbeddingsOperator",
     }
 
     BASE_CLASSES = {

providers/google/docs/operators/cloud/gen_ai.rst

@@ -0,0 +1,112 @@
+ .. Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+ ..   http://www.apache.org/licenses/LICENSE-2.0
+
+ .. Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+Google Cloud Generative AI on Vertex AI Operators
+=================================================
+
+The `Google Cloud VertexAI <https://cloud.google.com/vertex-ai/generative-ai/docs/>`__
+extends Vertex AI with powerful foundation models capable of generating text, images, and other modalities.
+It provides access to Google Gemini family of multimodal models and other pre-trained generative models through
+a unified API, SDK, and console. Developers can prompt, tune, and ground these models using their own data to build
+applications such as chat bots, content creation tools, code assistants, and summarization systems.
+With Vertex AI, you can securely integrate generative capabilities into enterprise workflows, monitor usage,
+evaluate model quality, and deploy models at scale — all within the same managed ML platform.
+
+
+Interacting with Generative AI
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+To generate text embeddings you can use
+:class:`~airflow.providers.google.cloud.operators.gen_ai.GenAIGenerateEmbeddingsOperator`.
+The operator returns the model's response in :ref:`XCom <concepts:xcom>` under ``model_response`` key.
+
+.. exampleinclude:: /../../google/tests/system/google/cloud/gen_ai/example_gen_ai_generative_model.py
+    :language: python
+    :dedent: 4
+    :start-after: [START how_to_cloud_gen_ai_generate_embeddings_task]
+    :end-before: [END how_to_cloud_gen_ai_generate_embeddings_task]
+
+
+To generate content with a generative model you can use
+:class:`~airflow.providers.google.cloud.operators.gen_ai.GenAIGenerateContentOperator`.
+The operator returns the model's response in :ref:`XCom <concepts:xcom>` under ``model_response`` key.
+
+.. exampleinclude:: /../../google/tests/system/google/cloud/gen_ai/example_gen_ai_generative_model.py
+    :language: python
+    :dedent: 4
+    :start-after: [START how_to_cloud_gen_ai_generate_content_operator]
+    :end-before: [END how_to_cloud_gen_ai_generate_content_operator]
+
+
+To run a supervised fine tuning job you can use
+:class:`~airflow.providers.google.cloud.operators.gen_ai.GenAISupervisedFineTuningTrainOperator`.
+The operator returns the tuned model's endpoint name in :ref:`XCom <concepts:xcom>` under ``tuned_model_endpoint_name`` key.
+
+.. exampleinclude:: /../../google/tests/system/google/cloud/gen_ai/example_gen_ai_generative_model_tuning.py
+    :language: python
+    :dedent: 4
+    :start-after: [START how_to_cloud_gen_ai_supervised_fine_tuning_train_operator]
+    :end-before: [END how_to_cloud_gen_ai_supervised_fine_tuning_train_operator]
+
+You can also use supervised fine tuning job for video tasks (training and tracking):
+
+.. exampleinclude:: /../../google/tests/system/google/cloud/gen_ai/example_gen_ai_generative_model_tuning.py
+    :language: python
+    :dedent: 4
+    :start-after: [START how_to_cloud_gen_ai_supervised_fine_tuning_train_operator_for_video]
+    :end-before: [END how_to_cloud_gen_ai_supervised_fine_tuning_train_operator_for_video]
+
+
+To calculates the number of input tokens before sending a request to the Gemini API you can use:
+:class:`~airflow.providers.google.cloud.operators.gen_ai.GenAICountTokensOperator`.
+The operator returns the total tokens in :ref:`XCom <concepts:xcom>` under ``total_tokens`` key.
+
+.. exampleinclude:: /../../google/tests/system/google/cloud/gen_ai/example_gen_ai_generative_model.py
+    :language: python
+    :dedent: 4
+    :start-after: [START how_to_cloud_gen_ai_count_tokens_operator]
+    :end-before: [END how_to_cloud_gen_ai_count_tokens_operator]
+
+
+To create cached content you can use
+:class:`~airflow.providers.google.cloud.operators.gen_ai.GenAICreateCachedContentOperator`.
+The operator returns the cached content resource name in :ref:`XCom <concepts:xcom>` under ``cached_content`` key.
+
+.. exampleinclude:: /../../google/tests/system/google/cloud/gen_ai/example_gen_ai_generative_model.py
+    :language: python
+    :dedent: 4
+    :start-after: [START how_to_cloud_gen_ai_create_cached_content_operator]
+    :end-before: [END how_to_cloud_gen_ai_create_cached_content_operator]
+
+
+To generate a response from cached content you can use
+:class:`~airflow.providers.google.cloud.operators.gen_ai.GenAIGenerateContentOperator`.
+The operator returns the cached content response in :ref:`XCom <concepts:xcom>` under ``model_response`` key.
+
+.. exampleinclude:: /../../google/tests/system/google/cloud/gen_ai/example_gen_ai_generative_model.py
+    :language: python
+    :dedent: 4
+    :start-after: [START how_to_cloud_gen_ai_generate_from_cached_content_operator]
+    :end-before: [END how_to_cloud_gen_ai_generate_from_cached_content_operator]
+
+
+Reference
+^^^^^^^^^
+
+For further information, look at:
+
+* `Client Library Documentation <https://cloud.google.com/vertex-ai/generative-ai/docs/sdks/overview>`__

providers/google/docs/operators/cloud/vertex_ai.rst

@@ -576,83 +576,107 @@ To get a pipeline job list you can use
 Interacting with Generative AI
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
+.. warning::
+    This operator is deprecated and will be removed after January 3, 2026. Please use
+    :class:`~airflow.providers.google.cloud.operators.gen_ai.generative_model.GenAIGenerateEmbeddingsOperator`.
+
 To generate text embeddings you can use
 :class:`~airflow.providers.google.cloud.operators.vertex_ai.generative_model.TextEmbeddingModelGetEmbeddingsOperator`.
 The operator returns the model's response in :ref:`XCom <concepts:xcom>` under ``model_response`` key.
 
-.. exampleinclude:: /../../google/tests/system/google/cloud/vertex_ai/example_vertex_ai_generative_model.py
+.. exampleinclude:: /../../google/tests/system/google/cloud/gen_ai/example_gen_ai_generative_model.py
     :language: python
     :dedent: 4
-    :start-after: [START how_to_cloud_vertex_ai_text_embedding_model_get_embeddings_operator]
-    :end-before: [END how_to_cloud_vertex_ai_text_embedding_model_get_embeddings_operator]
+    :start-after: [START how_to_cloud_gen_ai_generate_embeddings_task]
+    :end-before: [END how_to_cloud_gen_ai_generate_embeddings_task]
+
+.. warning::
+    This operator is deprecated and will be removed after January 3, 2026. Please use
+    :class:`~airflow.providers.google.cloud.operators.gen_ai.generative_model.GenAIGenerateContentOperator`.
 
 To generate content with a generative model you can use
 :class:`~airflow.providers.google.cloud.operators.vertex_ai.generative_model.GenerativeModelGenerateContentOperator`.
 The operator returns the model's response in :ref:`XCom <concepts:xcom>` under ``model_response`` key.
 
-.. exampleinclude:: /../../google/tests/system/google/cloud/vertex_ai/example_vertex_ai_generative_model.py
+.. exampleinclude:: /../../google/tests/system/google/cloud/gen_ai/example_gen_ai_generative_model.py
     :language: python
     :dedent: 4
-    :start-after: [START how_to_cloud_vertex_ai_generative_model_generate_content_operator]
-    :end-before: [END how_to_cloud_vertex_ai_generative_model_generate_content_operator]
+    :start-after: [START how_to_cloud_gen_ai_generate_content_operator]
+    :end-before: [END how_to_cloud_gen_ai_generate_content_operator]
+
+.. warning::
+    This operator is deprecated and will be removed after January 3, 2026. Please use
+    :class:`~airflow.providers.google.cloud.operators.gen_ai.generative_model.GenAISupervisedFineTuningTrainOperator`.
 
 To run a supervised fine tuning job you can use
 :class:`~airflow.providers.google.cloud.operators.vertex_ai.generative_model.SupervisedFineTuningTrainOperator`.
 The operator returns the tuned model's endpoint name in :ref:`XCom <concepts:xcom>` under ``tuned_model_endpoint_name`` key.
 
-.. exampleinclude:: /../../google/tests/system/google/cloud/vertex_ai/example_vertex_ai_generative_model_tuning.py
+.. exampleinclude:: /../../google/tests/system/google/cloud/gen_ai/example_gen_ai_generative_model_tuning.py
     :language: python
     :dedent: 4
-    :start-after: [START how_to_cloud_vertex_ai_supervised_fine_tuning_train_operator]
-    :end-before: [END how_to_cloud_vertex_ai_supervised_fine_tuning_train_operator]
+    :start-after: [START how_to_cloud_gen_ai_supervised_fine_tuning_train_operator]
+    :end-before: [END how_to_cloud_gen_ai_supervised_fine_tuning_train_operator]
 
 You can also use supervised fine tuning job for video tasks: training and tracking
 
-.. exampleinclude:: /../../google/tests/system/google/cloud/vertex_ai/example_vertex_ai_generative_model_tuning.py
+.. exampleinclude:: /../../google/tests/system/google/cloud/gen_ai/example_gen_ai_generative_model_tuning.py
     :language: python
     :dedent: 4
-    :start-after: [START how_to_cloud_vertex_ai_supervised_fine_tuning_train_operator_for_video]
-    :end-before: [END how_to_cloud_vertex_ai_supervised_fine_tuning_train_operator_for_video]
+    :start-after: [START how_to_cloud_gen_ai_supervised_fine_tuning_train_operator_for_video]
+    :end-before: [END how_to_cloud_gen_ai_supervised_fine_tuning_train_operator_for_video]
+
+.. warning::
+    This operator is deprecated and will be removed after January 3, 2026. Please use
+    :class:`~airflow.providers.google.cloud.operators.gen_ai.generative_model.GenAICountTokensOperator`.
 
 To calculates the number of input tokens before sending a request to the Gemini API you can use:
 :class:`~airflow.providers.google.cloud.operators.vertex_ai.generative_model.CountTokensOperator`.
 The operator returns the total tokens in :ref:`XCom <concepts:xcom>` under ``total_tokens`` key.
 
-.. exampleinclude:: /../../google/tests/system/google/cloud/vertex_ai/example_vertex_ai_generative_model.py
+.. exampleinclude:: /../../google/tests/system/google/cloud/gen_ai/example_gen_ai_generative_model.py
     :language: python
     :dedent: 4
-    :start-after: [START how_to_cloud_vertex_ai_count_tokens_operator]
-    :end-before: [END how_to_cloud_vertex_ai_count_tokens_operator]
+    :start-after: [START how_to_cloud_gen_ai_count_tokens_operator]
+    :end-before: [END how_to_cloud_gen_ai_count_tokens_operator]
 
 To evaluate a model you can use
 :class:`~airflow.providers.google.cloud.operators.vertex_ai.generative_model.RunEvaluationOperator`.
 The operator returns the evaluation summary metrics in :ref:`XCom <concepts:xcom>` under ``summary_metrics`` key.
 
-.. exampleinclude:: /../../google/tests/system/google/cloud/vertex_ai/example_vertex_ai_generative_model.py
+.. exampleinclude:: /../../google/tests/system/google/cloud/gen_ai/example_gen_ai_generative_model.py
     :language: python
     :dedent: 4
     :start-after: [START how_to_cloud_vertex_ai_run_evaluation_operator]
     :end-before: [END how_to_cloud_vertex_ai_run_evaluation_operator]
 
+.. warning::
+    This operator is deprecated and will be removed after January 3, 2026. Please use
+    :class:`~airflow.providers.google.cloud.operators.gen_ai.generative_model.GenAICreateCachedContentOperator`.
+
 To create cached content you can use
 :class:`~airflow.providers.google.cloud.operators.vertex_ai.generative_model.CreateCachedContentOperator`.
 The operator returns the cached content resource name in :ref:`XCom <concepts:xcom>` under ``return_value`` key.
 
-.. exampleinclude:: /../../google/tests/system/google/cloud/vertex_ai/example_vertex_ai_generative_model.py
+.. exampleinclude:: /../../google/tests/system/google/cloud/gen_ai/example_gen_ai_generative_model.py
     :language: python
     :dedent: 4
-    :start-after: [START how_to_cloud_vertex_ai_create_cached_content_operator]
-    :end-before: [END how_to_cloud_vertex_ai_create_cached_content_operator]
+    :start-after: [START how_to_cloud_gen_ai_create_cached_content_operator]
+    :end-before: [END how_to_cloud_gen_ai_create_cached_content_operator]
+
+.. warning::
+    This operator is deprecated and will be removed after January 3, 2026. Please use
+    :class:`~airflow.providers.google.cloud.operators.gen_ai.generative_model.GenAIGenerateContentOperator`.
 
 To generate a response from cached content you can use
 :class:`~airflow.providers.google.cloud.operators.vertex_ai.generative_model.GenerateFromCachedContentOperator`.
 The operator returns the cached content response in :ref:`XCom <concepts:xcom>` under ``return_value`` key.
 
-.. exampleinclude:: /../../google/tests/system/google/cloud/vertex_ai/example_vertex_ai_generative_model.py
+.. exampleinclude:: /../../google/tests/system/google/cloud/gen_ai/example_gen_ai_generative_model.py
     :language: python
     :dedent: 4
-    :start-after: [START how_to_cloud_vertex_ai_generate_from_cached_content_operator]
-    :end-before: [END how_to_cloud_vertex_ai_generate_from_cached_content_operator]
+    :start-after: [START how_to_cloud_gen_ai_generate_from_cached_content_operator]
+    :end-before: [END how_to_cloud_gen_ai_generate_from_cached_content_operator]
 
 Interacting with Vertex AI Feature Store
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

providers/google/provider.yaml

@@ -455,6 +455,11 @@ integrations:
     how-to-guide:
       - /docs/apache-airflow-providers-google/operators/cloud/cloud_logging_sink.rst
     tags: [gcp]
+  - integration-name: Google Cloud Generative AI
+    external-doc-url: https://cloud.google.com/generative-ai-studio
+    how-to-guide:
+      - /docs/apache-airflow-providers-google/operators/cloud/gen_ai.rst
+    tags: [gcp]
 
 operators:
   - integration-name: Google Ads
@@ -620,6 +625,9 @@ operators:
   - integration-name: Google Cloud Logging Sink
     python-modules:
       - airflow.providers.google.cloud.operators.cloud_logging_sink
+  - integration-name: Google Cloud Generative AI
+    python-modules:
+      - airflow.providers.google.cloud.operators.gen_ai
 
 sensors:
   - integration-name: Google BigQuery
@@ -901,6 +909,9 @@ hooks:
   - integration-name: Google Cloud Logging
     python-modules:
       - airflow.providers.google.cloud.hooks.cloud_logging
+  - integration-name: Google Cloud Generative AI
+    python-modules:
+      - airflow.providers.google.cloud.hooks.gen_ai
 
 
 triggers: