Skip to content

Callback context is missing the most useful attributes for notifications #52824

@cardilloscreations

Description

@cardilloscreations

Apache Airflow version

3.0.2

If "Other Airflow 2 version" selected, which one?

No response

What happened?

As others have reported, our Slack Notifications broke after upgrading to 3.0.2 and there was nothing in the release notes about this breaking change. After many hours of hunting this down, the root cause is that the provided context no longer contains task_instance (TaskInstance) and the provided dictionary does not contain enough to be useful for providing meaningful notifications.

After some debugging, this is all we're getting in the context for our on_failure_callback function for example:

context:{'dag': <DAG: test_slack_failure>, 'run_id': 'manual__2025-07-03T17:26:12.877086+00:00', 'reason': 'task_failure'}

As you can see, there is barely any information being provided now, so it makes it practically useless to use for alert notifications, etc.

I found a few issues where others have noticed this for other reasons, but I'm opening this one up more generally to highlight the problems that this change to the callback context in 3.0 has caused for notifications specifically.

See also: #50754, #52630, #51402
This PR is a good start, but is not complete enough for notifications: #51949

What you think should happen instead?

For the context to be useful for notifications and similar purposes, I propose that the following should be present at minimum, which should already be in the TaskInstance, but since we don't have that any more, we need:

  • dag - We already have this, which at least provides some basic details.
  • task_id - Just the plain unformatted ID string that represents the actual ID.
  • run_id - Just the plain unformatted ID string that represents the actual ID.
  • ts or timestamp - This was once useful as a timestamp for the callback.
  • next_retry_datetime - For the purposes of knowing if the event is going to be retried and when. Some notifications are not done unless it's permanently failed (with no retries scheduled).
  • log_url - For the purposes of being able to provide a convenient link to investigate when receiving a notification, etc.

Of course, this could all be solved if the TaskInstance was available in the Context again, because this information is all there (and more), but it looks like there may have been other reasons this was removed from in v3.

How to reproduce

Specify a callback on a DAG or Task (e.g., on_failure_callback) and trigger it. You'll see that the context is practically useless because it only contains a dag string, a run_id string , and a reason string.

Operating System

Debian GNU/Linux 12 (bookworm)

Versions of Apache Airflow Providers

N/A

Deployment

Virtualenv installation

Deployment details

Simple self-hosted LocalExecutor installation with postgres and systemd services.

Anything else?

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions