Fix XCom.delete error in Airflow 2.2.0

[jordanjeremy]

In Airflow 2.2.0 XCom.delete causes error, by trying to update dag_run table dag_id and execution_date columns to NULLs.

sqlalchemy.exc.IntegrityError: (psycopg2.errors.NotNullViolation) null value in column "dag_id" violates not-null constraint
[SQL: UPDATE dag_run SET dag_id=%(dag_id)s, execution_date=%(execution_date)s WHERE dag_run.id = %(dag_run_id)s]
[parameters: {'dag_id': None, 'execution_date': None, 'dag_run_id': 2409}]

Setting passive_deletes to the string value 'all' will disable the "nulling out" behavior.

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
Here are some useful points:

• Pay attention to the quality of your code (flake8, mypy and type annotations). Our pre-commits will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
• Consider using Breeze environment for testing locally, it’s a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: dev@airflow.apache.org
  Slack: https://s.apache.org/airflow-slack

[github-actions]

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

[boring-cyborg]

Awesome work, congrats on your first merged pull request!

[potiuk]

One doubt that came to me afer merging. Should we have a migration updated after that change @uranusjr @jordanjeremy ?

[jordanjeremy]

@potiuk I do not believe that any migration changes are required. In the Airflow 2.2.0 update changes were made to add a foreign key between the task_instance and dag_run tables. As part of that the dag_run table columns for dag_id and execution_date had the not null constraint added (e6c56c4).

This change doesn't really change the relationship between the xcom and dag_run tables. It is changing what sqlalchemy tries to do when an xcom is deleted. From the sqlalchemy documentation, the default behavior:
Normally, when a parent item is deleted, all child items are loaded so that they can either be marked as deleted, or have their foreign key to the parent set to NULL

So when deleting the xcom, sqlalchemy tried to set the dag_id and execution_date in the dag_run table to null, which is when the error happened due to the recently added not null constraint.

The change in this request stops sqlalchemy from trying to also update the dag_run table when deleting an xcom record.
setting the flag to the string value ‘all’ will disable the “nulling out” of the child foreign keys

[uranusjr]

Correct, the relationship behaviour is handled entirely in Python and stores nothing in the database.

[potiuk]

Cool! Great to confirm that - I thought so, but I was not entirely sure :)