Move over serde library to task sdk

[amoghrajesh]

Body

Serde is a library responsible for serialisation and deserialisation of objects in Airflow (not same as DAG serialisation but generally for ser + communication / transport of python objects).

Now, this library is mainly used on the task execution front: like for xcoms: https://github.com/apache/airflow/blob/main/task-sdk/src/airflow/sdk/bases/xcom.py#L325-L346

This is more of a utility helpful on execution front and having it in airflow-core creates unwanted dependency on core in task sdk. It would better be housed in task sdk.

Doing this task has 2 blockers though:

a) Usage in migration: https://github.com/apache/airflow/blob/main/airflow-core/src/airflow/migrations/versions/0092_3_2_0_replace_deadline_inline_callback_with_fkey.py -> this is a badly written migration and something needs to be done about it.
b) [Important] Usage in core API to render xcoms beautifully in the UI: https://github.com/apache/airflow/blob/main/airflow-core/src/airflow/api_fastapi/core_api/routes/public/xcom.py#L102-L116 (driven by deserialize flag)

Committer

• I acknowledge that I am a maintainer/committer of the Apache Airflow project.