-
Notifications
You must be signed in to change notification settings - Fork 16.3k
Description
Apache Airflow version
3.0.2
If "Other Airflow 2 version" selected, which one?
No response
What happened?
As others have reported, our Slack Notifications broke after upgrading to 3.0.2 and there was nothing in the release notes about this breaking change. After many hours of hunting this down, the root cause is that the provided context no longer contains task_instance (TaskInstance) and the provided dictionary does not contain enough to be useful for providing meaningful notifications.
After some debugging, this is all we're getting in the context for our on_failure_callback function for example:
context:{'dag': <DAG: test_slack_failure>, 'run_id': 'manual__2025-07-03T17:26:12.877086+00:00', 'reason': 'task_failure'}
As you can see, there is barely any information being provided now, so it makes it practically useless to use for alert notifications, etc.
I found a few issues where others have noticed this for other reasons, but I'm opening this one up more generally to highlight the problems that this change to the callback context in 3.0 has caused for notifications specifically.
See also: #50754, #52630, #51402
This PR is a good start, but is not complete enough for notifications: #51949
What you think should happen instead?
For the context to be useful for notifications and similar purposes, I propose that the following should be present at minimum, which should already be in the TaskInstance, but since we don't have that any more, we need:
dag- We already have this, which at least provides some basic details.task_id- Just the plain unformatted ID string that represents the actual ID.run_id- Just the plain unformatted ID string that represents the actual ID.tsortimestamp- This was once useful as a timestamp for the callback.next_retry_datetime- For the purposes of knowing if the event is going to be retried and when. Some notifications are not done unless it's permanently failed (with no retries scheduled).log_url- For the purposes of being able to provide a convenient link to investigate when receiving a notification, etc.
Of course, this could all be solved if the TaskInstance was available in the Context again, because this information is all there (and more), but it looks like there may have been other reasons this was removed from in v3.
How to reproduce
Specify a callback on a DAG or Task (e.g., on_failure_callback) and trigger it. You'll see that the context is practically useless because it only contains a dag string, a run_id string , and a reason string.
Operating System
Debian GNU/Linux 12 (bookworm)
Versions of Apache Airflow Providers
N/A
Deployment
Virtualenv installation
Deployment details
Simple self-hosted LocalExecutor installation with postgres and systemd services.
Anything else?
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct