Branching is broken when callable function returns None or falsy value

[yershovkiril]

Apache Airflow version

3.0.4

If "Other Airflow 2 version" selected, which one?

No response

What happened?

Previously (before 3.0.4) when BranchPythonOperator callable returned None (or any falsy value) downstream task(-s) would be marked as skipped, but this behavior is broken in 3.0.4 and fail pipeline with:
AirflowException: 'branch_task_ids' expected all task IDs are strings. Invalid tasks found: {(None, 'NoneType')}.

https://github.com/apache/airflow/pull/53431/files - I believe this change has brought an issue

What you think should happen instead?

We should skip downstream tasks as it used to be, even in tutorial None-s are part of the example - https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/dags.html#branching

How to reproduce

Use callable with return None for branching task

Operating System

Ubuntu 20.04.6 LTS

Versions of Apache Airflow Providers

No response

Deployment

Docker-Compose

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[yershovkiril]

@eladkal it is not a duplication, at the very least previous one should be reopened then... workflow should NOT fail in the branching flow of you pass None, isn't it right?

[eladkal]

@eladkal it is not a duplication, at the very least previous one should be reopened then... workflow should NOT fail in the branching flow of you pass None, isn't it right?

See #53984 (comment) for how it's going to be avoided.
Branch is expected to always follow something. The python callable of the branch should account for cases where logic fails and in that case should you want to not follow anything then you need to explicitly raise AirflowSkipException.

to my perspective following nothing = hiding a problem in the pipeline logic thus we should not silently skip it. Branch is not like ShortCircutOperator. If you believe otherwise please open a new feature request explain the use case and we'd be happy to look into it and improve it. If there is a valid use case then we will support it.

[yershovkiril]

@eladkal but that's exactly how it's recommended to be used in the documentation (return None scenario) - https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/dags.html#branching

if you believe that following nothing = hiding, let's then you this issue to fix documentation, otherwise it's not consistent

[eladkal]

@eladkal but that's exactly how it's recommended to be used in the documentation (return None scenario) - https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/dags.html#branching

if you believe that following nothing = hiding, let's then you this issue to fix documentation, otherwise it's not consistent

I believe this doc is wrong we can fix it, I looked it up and this code entry is from before 2022 so hard to see what the author had in mind.

I will reopen this as feature request cc @amoghrajesh @atul-astronomer