Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion task-sdk/docs/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ This page documents the full public API exposed in Airflow 3.0+ via the Task SDK
If something is not on this page it is best to assume that it is not part of the public API and use of it is entirely at your own risk
-- we won't go out of our way break usage of them, but we make no promises either.

Defining DAGs
Defining Dags
-------------
.. autoapiclass:: airflow.sdk.DAG

Expand Down
4 changes: 2 additions & 2 deletions task-sdk/docs/concepts.rst
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ This section covers the fundamental concepts that Dag authors need to understand

Terminology
-----------
- **Task**: a Python function (decorated with ``@task``) or Operator invocation representing a unit of work in a DAG.
- **Task**: a Python function (decorated with ``@task``) or Operator invocation representing a unit of work in a Dag.
- **Task Execution**: the runtime machinery that executes user tasks in isolated subprocesses, managed via the Supervisor and Execution API.

Task Lifecycle
Expand All @@ -39,7 +39,7 @@ Understanding the task lifecycle helps Dag authors write more effective tasks an
- **Subprocess Launch**: The worker's Supervisor process spawns a dedicated subprocess (Task Runner) for the task instance, isolating its execution.
- **Run API Call**: The Supervisor sends a ``POST /run`` call to the Execution API to mark the task as running; the API server responds with a ``TIRunContext`` containing essential runtime information including:

- **``dag_run``**: Complete DAG run information (logical date, data intervals, configuration, etc.)
- **``dag_run``**: Complete Dag run information (logical date, data intervals, configuration, etc.)
- **``max_tries``**: Maximum number of retry attempts allowed for this task instance
- **``should_retry``**: Boolean flag indicating whether the task should enter retry state or fail immediately on error
- **``task_reschedule_count``**: Number of times this task has been rescheduled
Expand Down
21 changes: 11 additions & 10 deletions task-sdk/docs/examples.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,13 @@ Examples

Key Concepts
------------
Defining DAGs

Defining Dags
~~~~~~~~~~~~~

Example: Defining a DAG
Example: Defining a Dag

Use the :func:`airflow.sdk.dag` decorator to convert a Python function into an Airflow DAG. All nested calls to :func:`airflow.sdk.task` within the function will become tasks in the DAG. For full parameters and usage, see the API reference for :func:`airflow.sdk.dag`.
Use the :func:`airflow.sdk.dag` decorator to convert a Python function into an Airflow Dag. All nested calls to :func:`airflow.sdk.task` within the function will become tasks in the Dag. For full parameters and usage, see the API reference for :func:`airflow.sdk.dag`.

.. exampleinclude:: ../../airflow-core/src/airflow/example_dags/example_dag_decorator.py
:language: python
Expand All @@ -40,10 +41,10 @@ Decorators

Example: Using Task SDK decorators

The Task SDK provides decorators to simplify DAG definitions:
The Task SDK provides decorators to simplify Dag definitions:

- :func:`airflow.sdk.task_group` groups related tasks into logical TaskGroups.
- :func:`airflow.sdk.setup` and :func:`airflow.sdk.teardown` define setup and teardown hooks for DAGs or TaskGroups.
- :func:`airflow.sdk.setup` and :func:`airflow.sdk.teardown` define setup and teardown hooks for Dags or TaskGroups.

.. exampleinclude:: ../../airflow-core/src/airflow/example_dags/example_task_group_decorator.py
:language: python
Expand Down Expand Up @@ -101,16 +102,16 @@ TaskFlow API Tutorial
This section provides a concise, code-first view. For the full tutorial and context,
see the `core TaskFlow tutorial <../../airflow-core/docs/tutorial/taskflow.rst>`_.

Step 1: Define the DAG
Step 1: Define the Dag
----------------------

In this step, define your DAG by applying the :func:`airflow.sdk.dag` decorator to a Python function. This registers the DAG with its schedule and default arguments. For more details, see :func:`airflow.sdk.dag`.
In this step, define your Dag by applying the :func:`airflow.sdk.dag` decorator to a Python function. This registers the Dag with its schedule and default arguments. For more details, see :func:`airflow.sdk.dag`.

.. exampleinclude:: ../../airflow-core/src/airflow/example_dags/tutorial_taskflow_api.py
:language: python
:start-after: [START instantiate_dag]
:end-before: [END instantiate_dag]
:caption: Defining the DAG with the :func:`@dag <airflow.sdk.dag>` decorator
:caption: Defining the Dag with the :func:`@dag <airflow.sdk.dag>` decorator

Step 2: Write your Tasks
------------------------
Expand Down Expand Up @@ -146,11 +147,11 @@ Step 3: Build the Flow
:end-before: [END main_flow]
:caption: Connecting tasks by invoking them like normal Python functions

Step 4: Invoke the DAG
Step 4: Invoke the Dag
----------------------

.. exampleinclude:: ../../airflow-core/src/airflow/example_dags/tutorial_taskflow_api.py
:language: python
:start-after: [START dag_invocation]
:end-before: [END dag_invocation]
:caption: Registering the DAG by calling the decorated function
:caption: Registering the Dag by calling the decorated function
21 changes: 11 additions & 10 deletions task-sdk/docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,13 @@
Apache Airflow Task SDK
=================================

The Apache Airflow Task SDK provides python-native interfaces for defining DAGs,
The Apache Airflow Task SDK provides python-native interfaces for defining Dags,
executing tasks in isolated subprocesses and interacting with Airflow resources
(e.g., Connections, Variables, XComs, Metrics, Logs, and OpenLineage events) at runtime.
It also includes core execution-time components to manage communication between the worker
and the Airflow scheduler/backend.

The goal of task-sdk is to decouple Dag authoring from Airflow internals (Scheduler, API Server, etc.), providing a forward-compatible, stable interface for writing and maintaining DAGs across Airflow versions. This approach reduces boilerplate and keeps your DAG definitions concise and readable.
The goal of task-sdk is to decouple Dag authoring from Airflow internals (Scheduler, API Server, etc.), providing a forward-compatible, stable interface for writing and maintaining Dags across Airflow versions. This approach reduces boilerplate and keeps your Dag definitions concise and readable.

1. Introduction and Getting Started
-----------------------------------
Expand All @@ -42,13 +42,13 @@ To install the Task SDK, run:

Getting Started
^^^^^^^^^^^^^^^
Define a basic DAG and task in just a few lines of Python:
Define a basic Dag and task in just a few lines of Python:

.. exampleinclude:: ../../airflow-core/src/airflow/example_dags/example_simplest_dag.py
:language: python
:start-after: [START simplest_dag]
:end-before: [END simplest_dag]
:caption: Simplest DAG with :func:`@dag <airflow.sdk.dag>` and :func:`@task <airflow.sdk.task>`
:caption: Simplest Dag with :func:`@dag <airflow.sdk.dag>` and :func:`@task <airflow.sdk.task>`

2. Public Interface
-------------------
Expand All @@ -59,16 +59,17 @@ Airflow now supports a service-oriented architecture, enabling tasks to be execu

To support remote execution, Airflow provides the Task SDK — a lightweight runtime environment for running Airflow tasks in external systems such as containers, edge environments, or other runtimes. This lays the groundwork for language-agnostic task execution and brings improved isolation, portability, and extensibility to Airflow-based workflows.

Airflow 3.0 also introduces a new ``airflow.sdk`` namespace that exposes the core authoring interfaces for defining DAGs and tasks. Dag authors should now import objects like :class:`airflow.sdk.DAG`, :func:`airflow.sdk.dag`, and :func:`airflow.sdk.task` from ``airflow.sdk`` rather than internal modules. This new namespace provides a stable, forward-compatible interface for Dag authoring across future versions of Airflow.
Airflow 3.0 also introduces a new ``airflow.sdk`` namespace that exposes the core authoring interfaces for defining Dags and tasks. Dag authors should now import objects like :class:`airflow.sdk.DAG`, :func:`airflow.sdk.dag`, and :func:`airflow.sdk.task` from ``airflow.sdk`` rather than internal modules. This new namespace provides a stable, forward-compatible interface for Dag authoring across future versions of Airflow.

3. Dag authoring Enhancements
-----------------------------

Writing your DAGs is now more consistent in Airflow 3.0. Use the stable :mod:`airflow.sdk` interface to define your workflows and tasks.
Writing your Dags is now more consistent in Airflow 3.0. Use the stable :mod:`airflow.sdk` interface to define your workflows and tasks.

Why use ``airflow.sdk``?
^^^^^^^^^^^^^^^^^^^^^^^^
- Decouple your DAG definitions from Airflow internals (Scheduler, API Server, etc.)

- Decouple your Dag definitions from Airflow internals (Scheduler, API Server, etc.)
- Enjoy a consistent API that won't break across Airflow upgrades
- Import only the classes and decorators you need, without installing the full Airflow core

Expand Down Expand Up @@ -106,7 +107,7 @@ Why use ``airflow.sdk``?
- :func:`airflow.sdk.get_current_context`
- :func:`airflow.sdk.get_parsing_context`

All DAGs must update their imports to refer to ``airflow.sdk`` instead of using internal Airflow modules directly. Deprecated legacy import paths, such as ``airflow.models.dag.DAG`` and ``airflow.decorator.task``, will be removed in a future version of Airflow. Some utilities and helper functions currently used from ``airflow.utils.*`` and other modules will gradually be migrated to the Task SDK over the next minor releases. These upcoming updates aim to completely separate DAG creation from internal Airflow services. Dag authors can look forward to continuous improvements to airflow.sdk, with no backwards-incompatible changes to their existing code.
All Dags must update their imports to refer to ``airflow.sdk`` instead of using internal Airflow modules directly. Deprecated legacy import paths, such as ``airflow.models.dag.DAG`` and ``airflow.decorator.task``, will be removed in a future version of Airflow. Some utilities and helper functions currently used from ``airflow.utils.*`` and other modules will gradually be migrated to the Task SDK over the next minor releases. These upcoming updates aim to completely separate Dag creation from internal Airflow services. Dag authors can look forward to continuous improvements to airflow.sdk, with no backwards-incompatible changes to their existing code.

Legacy imports (deprecated):

Expand All @@ -123,10 +124,10 @@ Use instead:
# Airflow 3.x
from airflow.sdk import DAG, task

4. Example DAG References
4. Example Dag References
-------------------------

Explore a variety of DAG examples and patterns in the :doc:`examples` page.
Explore a variety of Dag examples and patterns in the :doc:`examples` page.

5. Concepts
-----------
Expand Down