Skip to content

Conversation

@ferruzzi
Copy link
Contributor

@ferruzzi ferruzzi commented Sep 26, 2025

Adds async support to the existing SNS Notifier so it can be used as an AsyncCallback for DeadlineAlerts, among other things

dags used for manual testing:

from datetime import datetime, timedelta

from airflow.providers.amazon.aws.hooks.sns import SnsHook
from airflow.providers.amazon.aws.notifications.sns import SnsNotifier
from airflow.sdk import task, DAG
from airflow.sdk.definitions.deadline import DeadlineAlert, DeadlineReference, AsyncCallback

PAST_DATE = datetime(1980, 8, 10, 2)
MSG_PREFIX = "SNS Testing Rnd 1:"

DEFAULTS = {"target_arn": << REDACTED >>, "message": "testing via Airflow"}


def _sns_callback(context):
    SnsHook().publish_to_target(**DEFAULTS, subject=f"{MSG_PREFIX} Hook as Callback -- {context['dag_run'].dag_id}")


@task
def send_sns_message(subject):
    SnsHook().publish_to_target(**DEFAULTS, subject=subject)


@task.bash(task_id='sleep_task')
def sleep_n_secs(seconds):
    return f'sleep {seconds}'


with DAG(
    "sns_hook_via_taskflow",
    tags=["sns"]
):
    send_sns_message(subject=f"{MSG_PREFIX} Taskflow -- {{{{ dag_run.dag_id }}}} ")

with DAG(
    "sns_callback_on_success",
    on_success_callback=_sns_callback,
    tags=["sns"],
):
    sleep_n_secs(1)

with DAG(
    "sns_notifier_as_callback",
    on_success_callback=SnsNotifier(**DEFAULTS, subject=f"{MSG_PREFIX} Sync Notifier as callback -- {{{{ dag_run.dag_id }}}} "),
    tags=["sns"],
):
    sleep_n_secs(1)


with DAG(
    "sns_notifier_as_deadline",
    deadline=DeadlineAlert(
        reference=DeadlineReference.FIXED_DATETIME(PAST_DATE),
        interval=timedelta(0),
        callback=AsyncCallback(
            SnsNotifier,
            kwargs={**DEFAULTS, "subject": f"{MSG_PREFIX} Async Notifier as Deadline -- {{{{ dag_run.dag_id }}}} "}
        ),
    ),
    tags=["sns"],
):
    sleep_n_secs(1)

and proof of delivery with working templating

image

CC: @ramitkataria @seanghaeli


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@ferruzzi ferruzzi requested a review from vincbeck September 26, 2025 03:55
@boring-cyborg boring-cyborg bot added area:providers provider:amazon AWS/Amazon - related issues labels Sep 26, 2025
@ferruzzi ferruzzi mentioned this pull request Sep 26, 2025
1 task
@ferruzzi ferruzzi merged commit b47586e into apache:main Sep 26, 2025
80 checks passed
@ferruzzi ferruzzi deleted the ferruzzi/notifiers/sns-async branch September 26, 2025 23:34
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Sep 30, 2025
* Add async support for Amazon SNS Notifier
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 1, 2025
* Add async support for Amazon SNS Notifier
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 2, 2025
* Add async support for Amazon SNS Notifier
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 3, 2025
* Add async support for Amazon SNS Notifier
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 4, 2025
* Add async support for Amazon SNS Notifier
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 5, 2025
* Add async support for Amazon SNS Notifier
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 5, 2025
* Add async support for Amazon SNS Notifier
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 7, 2025
* Add async support for Amazon SNS Notifier
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 8, 2025
* Add async support for Amazon SNS Notifier
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 9, 2025
* Add async support for Amazon SNS Notifier
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 10, 2025
* Add async support for Amazon SNS Notifier
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 11, 2025
* Add async support for Amazon SNS Notifier
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 12, 2025
* Add async support for Amazon SNS Notifier
dabla pushed a commit to dabla/airflow that referenced this pull request Oct 12, 2025
* Add async support for Amazon SNS Notifier
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 14, 2025
* Add async support for Amazon SNS Notifier
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 15, 2025
* Add async support for Amazon SNS Notifier
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 17, 2025
* Add async support for Amazon SNS Notifier
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 19, 2025
* Add async support for Amazon SNS Notifier
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:amazon AWS/Amazon - related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants