diff --git a/task-sdk/docs/api.rst b/task-sdk/docs/api.rst index 40ea8e17287a4..b75d19647e0ca 100644 --- a/task-sdk/docs/api.rst +++ b/task-sdk/docs/api.rst @@ -23,7 +23,7 @@ This page documents the full public API exposed in Airflow 3.0+ via the Task SDK If something is not on this page it is best to assume that it is not part of the public API and use of it is entirely at your own risk -- we won't go out of our way break usage of them, but we make no promises either. -Defining DAGs +Defining Dags ------------- .. autoapiclass:: airflow.sdk.DAG diff --git a/task-sdk/docs/concepts.rst b/task-sdk/docs/concepts.rst index 25b45ba305acf..ef96d834f23a3 100644 --- a/task-sdk/docs/concepts.rst +++ b/task-sdk/docs/concepts.rst @@ -26,7 +26,7 @@ This section covers the fundamental concepts that Dag authors need to understand Terminology ----------- -- **Task**: a Python function (decorated with ``@task``) or Operator invocation representing a unit of work in a DAG. +- **Task**: a Python function (decorated with ``@task``) or Operator invocation representing a unit of work in a Dag. - **Task Execution**: the runtime machinery that executes user tasks in isolated subprocesses, managed via the Supervisor and Execution API. Task Lifecycle @@ -39,7 +39,7 @@ Understanding the task lifecycle helps Dag authors write more effective tasks an - **Subprocess Launch**: The worker's Supervisor process spawns a dedicated subprocess (Task Runner) for the task instance, isolating its execution. - **Run API Call**: The Supervisor sends a ``POST /run`` call to the Execution API to mark the task as running; the API server responds with a ``TIRunContext`` containing essential runtime information including: - - **``dag_run``**: Complete DAG run information (logical date, data intervals, configuration, etc.) + - **``dag_run``**: Complete Dag run information (logical date, data intervals, configuration, etc.) - **``max_tries``**: Maximum number of retry attempts allowed for this task instance - **``should_retry``**: Boolean flag indicating whether the task should enter retry state or fail immediately on error - **``task_reschedule_count``**: Number of times this task has been rescheduled diff --git a/task-sdk/docs/examples.rst b/task-sdk/docs/examples.rst index 9f8ff0124f4f3..4cef5b28c7a83 100644 --- a/task-sdk/docs/examples.rst +++ b/task-sdk/docs/examples.rst @@ -22,12 +22,13 @@ Examples Key Concepts ------------ -Defining DAGs + +Defining Dags ~~~~~~~~~~~~~ -Example: Defining a DAG +Example: Defining a Dag -Use the :func:`airflow.sdk.dag` decorator to convert a Python function into an Airflow DAG. All nested calls to :func:`airflow.sdk.task` within the function will become tasks in the DAG. For full parameters and usage, see the API reference for :func:`airflow.sdk.dag`. +Use the :func:`airflow.sdk.dag` decorator to convert a Python function into an Airflow Dag. All nested calls to :func:`airflow.sdk.task` within the function will become tasks in the Dag. For full parameters and usage, see the API reference for :func:`airflow.sdk.dag`. .. exampleinclude:: ../../airflow-core/src/airflow/example_dags/example_dag_decorator.py :language: python @@ -40,10 +41,10 @@ Decorators Example: Using Task SDK decorators -The Task SDK provides decorators to simplify DAG definitions: +The Task SDK provides decorators to simplify Dag definitions: - :func:`airflow.sdk.task_group` groups related tasks into logical TaskGroups. -- :func:`airflow.sdk.setup` and :func:`airflow.sdk.teardown` define setup and teardown hooks for DAGs or TaskGroups. +- :func:`airflow.sdk.setup` and :func:`airflow.sdk.teardown` define setup and teardown hooks for Dags or TaskGroups. .. exampleinclude:: ../../airflow-core/src/airflow/example_dags/example_task_group_decorator.py :language: python @@ -101,16 +102,16 @@ TaskFlow API Tutorial This section provides a concise, code-first view. For the full tutorial and context, see the `core TaskFlow tutorial <../../airflow-core/docs/tutorial/taskflow.rst>`_. -Step 1: Define the DAG +Step 1: Define the Dag ---------------------- -In this step, define your DAG by applying the :func:`airflow.sdk.dag` decorator to a Python function. This registers the DAG with its schedule and default arguments. For more details, see :func:`airflow.sdk.dag`. +In this step, define your Dag by applying the :func:`airflow.sdk.dag` decorator to a Python function. This registers the Dag with its schedule and default arguments. For more details, see :func:`airflow.sdk.dag`. .. exampleinclude:: ../../airflow-core/src/airflow/example_dags/tutorial_taskflow_api.py :language: python :start-after: [START instantiate_dag] :end-before: [END instantiate_dag] - :caption: Defining the DAG with the :func:`@dag ` decorator + :caption: Defining the Dag with the :func:`@dag ` decorator Step 2: Write your Tasks ------------------------ @@ -146,11 +147,11 @@ Step 3: Build the Flow :end-before: [END main_flow] :caption: Connecting tasks by invoking them like normal Python functions -Step 4: Invoke the DAG +Step 4: Invoke the Dag ---------------------- .. exampleinclude:: ../../airflow-core/src/airflow/example_dags/tutorial_taskflow_api.py :language: python :start-after: [START dag_invocation] :end-before: [END dag_invocation] - :caption: Registering the DAG by calling the decorated function + :caption: Registering the Dag by calling the decorated function diff --git a/task-sdk/docs/index.rst b/task-sdk/docs/index.rst index 6c4621dd12241..ba91ea48bebdf 100644 --- a/task-sdk/docs/index.rst +++ b/task-sdk/docs/index.rst @@ -18,13 +18,13 @@ Apache Airflow Task SDK ================================= -The Apache Airflow Task SDK provides python-native interfaces for defining DAGs, +The Apache Airflow Task SDK provides python-native interfaces for defining Dags, executing tasks in isolated subprocesses and interacting with Airflow resources (e.g., Connections, Variables, XComs, Metrics, Logs, and OpenLineage events) at runtime. It also includes core execution-time components to manage communication between the worker and the Airflow scheduler/backend. -The goal of task-sdk is to decouple Dag authoring from Airflow internals (Scheduler, API Server, etc.), providing a forward-compatible, stable interface for writing and maintaining DAGs across Airflow versions. This approach reduces boilerplate and keeps your DAG definitions concise and readable. +The goal of task-sdk is to decouple Dag authoring from Airflow internals (Scheduler, API Server, etc.), providing a forward-compatible, stable interface for writing and maintaining Dags across Airflow versions. This approach reduces boilerplate and keeps your Dag definitions concise and readable. 1. Introduction and Getting Started ----------------------------------- @@ -42,13 +42,13 @@ To install the Task SDK, run: Getting Started ^^^^^^^^^^^^^^^ -Define a basic DAG and task in just a few lines of Python: +Define a basic Dag and task in just a few lines of Python: .. exampleinclude:: ../../airflow-core/src/airflow/example_dags/example_simplest_dag.py :language: python :start-after: [START simplest_dag] :end-before: [END simplest_dag] - :caption: Simplest DAG with :func:`@dag ` and :func:`@task ` + :caption: Simplest Dag with :func:`@dag ` and :func:`@task ` 2. Public Interface ------------------- @@ -59,16 +59,17 @@ Airflow now supports a service-oriented architecture, enabling tasks to be execu To support remote execution, Airflow provides the Task SDK — a lightweight runtime environment for running Airflow tasks in external systems such as containers, edge environments, or other runtimes. This lays the groundwork for language-agnostic task execution and brings improved isolation, portability, and extensibility to Airflow-based workflows. -Airflow 3.0 also introduces a new ``airflow.sdk`` namespace that exposes the core authoring interfaces for defining DAGs and tasks. Dag authors should now import objects like :class:`airflow.sdk.DAG`, :func:`airflow.sdk.dag`, and :func:`airflow.sdk.task` from ``airflow.sdk`` rather than internal modules. This new namespace provides a stable, forward-compatible interface for Dag authoring across future versions of Airflow. +Airflow 3.0 also introduces a new ``airflow.sdk`` namespace that exposes the core authoring interfaces for defining Dags and tasks. Dag authors should now import objects like :class:`airflow.sdk.DAG`, :func:`airflow.sdk.dag`, and :func:`airflow.sdk.task` from ``airflow.sdk`` rather than internal modules. This new namespace provides a stable, forward-compatible interface for Dag authoring across future versions of Airflow. 3. Dag authoring Enhancements ----------------------------- -Writing your DAGs is now more consistent in Airflow 3.0. Use the stable :mod:`airflow.sdk` interface to define your workflows and tasks. +Writing your Dags is now more consistent in Airflow 3.0. Use the stable :mod:`airflow.sdk` interface to define your workflows and tasks. Why use ``airflow.sdk``? ^^^^^^^^^^^^^^^^^^^^^^^^ -- Decouple your DAG definitions from Airflow internals (Scheduler, API Server, etc.) + +- Decouple your Dag definitions from Airflow internals (Scheduler, API Server, etc.) - Enjoy a consistent API that won't break across Airflow upgrades - Import only the classes and decorators you need, without installing the full Airflow core @@ -106,7 +107,7 @@ Why use ``airflow.sdk``? - :func:`airflow.sdk.get_current_context` - :func:`airflow.sdk.get_parsing_context` -All DAGs must update their imports to refer to ``airflow.sdk`` instead of using internal Airflow modules directly. Deprecated legacy import paths, such as ``airflow.models.dag.DAG`` and ``airflow.decorator.task``, will be removed in a future version of Airflow. Some utilities and helper functions currently used from ``airflow.utils.*`` and other modules will gradually be migrated to the Task SDK over the next minor releases. These upcoming updates aim to completely separate DAG creation from internal Airflow services. Dag authors can look forward to continuous improvements to airflow.sdk, with no backwards-incompatible changes to their existing code. +All Dags must update their imports to refer to ``airflow.sdk`` instead of using internal Airflow modules directly. Deprecated legacy import paths, such as ``airflow.models.dag.DAG`` and ``airflow.decorator.task``, will be removed in a future version of Airflow. Some utilities and helper functions currently used from ``airflow.utils.*`` and other modules will gradually be migrated to the Task SDK over the next minor releases. These upcoming updates aim to completely separate Dag creation from internal Airflow services. Dag authors can look forward to continuous improvements to airflow.sdk, with no backwards-incompatible changes to their existing code. Legacy imports (deprecated): @@ -123,10 +124,10 @@ Use instead: # Airflow 3.x from airflow.sdk import DAG, task -4. Example DAG References +4. Example Dag References ------------------------- -Explore a variety of DAG examples and patterns in the :doc:`examples` page. +Explore a variety of Dag examples and patterns in the :doc:`examples` page. 5. Concepts -----------