prev_execution_date_success deprication warning suggests using prev_data_interval_start_success which is not equivalent

[KeeganCarter11]

Apache Airflow version

2.7.3

What happened

While using this variable from context ex: last_dag_success=context.get('prev_execution_date_success') I get the below warning. This suggests using prev_data_interval_start_success however this is NOT an equivalent date_time. The first is the date_time for which the DAG most recently had a successful run including manual runs. The latter is the assumed last successful start of a run based on the data interval.

{{logging_mixin.py:151}} WARNING - /usr/local/***/.local/lib/python3.11/site-packages/***/utils/context.py:314 AirflowContextDeprecationWarning: Accessing 'prev_execution_date_success' from the template is deprecated and will be removed in a future version. Please use 'prev_data_interval_start_success' instead.

What you think should happen instead

Reading documentation it seems like this should direct users to prev_start_date_success since that is defined as "Start date from prior successful DagRun (if available)." however that variable also doesn't seem to behave as expected. In my example dag below running the dag multiple times manually even though it succeeds this variable never updates from None

How to reproduce

Below sample dag that shows this behavior. Run it a couple of times and watch the date-times in the logs. Something is weird here.

from datetime import datetime, timedelta
from airflow.decorators import task, dag
default_args = {
    'start_date': datetime(2023, 9, 12),
    'retries': 0,
    'retry_delay': timedelta(minutes=15)
}
@dag(dag_id="AAA_Context_Example",
     tags=["example"],
     catchup=False,
     default_args=default_args,
     schedule_interval="01 0 * * *"
     )
def main():
    @task()
    def example(**context):
        print(f"prev_execution_date_success: {context.get('prev_execution_date_success')}")
        print(f"prev_start_date_success : {context.get('prev_start_date_success ')}")
        print(f"prev_data_interval_start_success: {context.get('prev_data_interval_start_success')}")
    example()
dag = main()

Operating System

MWAA

Versions of Apache Airflow Providers

NA

Deployment

Official Apache Airflow Helm Chart

Deployment details

MWAA / Local runner testing 7.3

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[jscheffl]

Thanks for reporting this issue. In detail I agree that there might be a small difference between the two context variables but it is mainly caused by the fact that Airflow tries to move away from execution_date to data_interval. If there is any difference you see it might be caused by the TimeTable you use.

I tried to run the posted example multiple times and could not find a problem there. Can you maybe retry directly pointing with more details on the case how to reproduce the problem? How much often did you run? Any more details on how much often until you had the error? Was it a concurrency when both tasks of two runs were executed in parallel the first time?

Besides the error report, let me have a small PR adjusting the documentation.

Note: There is a typo in your example, there is a space too much when accessing the context variable. Correct is: print(f"prev_start_date_success : {context.get('prev_start_date_success')}")

[github-actions]

This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.

[github-actions]

This issue has been closed because it has not received response from the issue author.