SequentialExecutor is not compatible with task-sdk

[tirkarthi]

Apache Airflow version

main (development)

If "Other Airflow 2 version" selected, which one?

No response

What happened?

SequentialExecutor which is the default executor fails in the main branch. It seems #47453 which removed the run command is still used with SequentialExecutor and the executor might need to be ported to task-sdk I guess

[2025-03-11T16:08:34.960+0530] {base_executor.py:301} DEBUG - 32 open slots for executor SequentialExecutor
[2025-03-11T16:08:34.961+0530] {base_executor.py:253} DEBUG - Calling the <class 'airflow.executors.sequential_executor.SequentialExecutor'> sync method
[2025-03-11T16:08:34.961+0530] {sequential_executor.py:85} INFO - Executing command: ['airflow', 'tasks', 'run', 'example_branch_operator', 'branching', 'scheduled__2025-03-11T00:00:00+00:00', '--local', '--subdir', 'DAGS_FOLDER/example_branch_operator.py']
/home/karthikeyan/stuff/python/airflow/airflow/cli/cli_config.py:602 DeprecationWarning: The web_server_port option in [webserver] has been moved to the port option in [api] - the old setting has been used, but please update your config.
/home/karthikeyan/stuff/python/airflow/airflow/cli/cli_config.py:608 DeprecationWarning: The workers option in [webserver] has been moved to the workers option in [api] - the old setting has been used, but please update your config.
/home/karthikeyan/stuff/python/airflow/airflow/cli/cli_config.py:620 DeprecationWarning: The web_server_host option in [webserver] has been moved to the host option in [api] - the old setting has been used, but please update your config.
/home/karthikeyan/stuff/python/airflow/airflow/cli/cli_config.py:625 DeprecationWarning: The access_logfile option in [webserver] has been moved to the access_logfile option in [api] - the old setting has been used, but please update your config.
/home/karthikeyan/stuff/python/airflow/airflow/cli/cli_config.py:640 DeprecationWarning: The web_server_ssl_cert option in [webserver] has been moved to the ssl_cert option in [api] - the old setting has been used, but please update your config.
/home/karthikeyan/stuff/python/airflow/airflow/cli/cli_config.py:645 DeprecationWarning: The web_server_ssl_key option in [webserver] has been moved to the ssl_key option in [api] - the old setting has been used, but please update your config.
Usage: airflow tasks [-h] COMMAND ...

Manage tasks

Positional Arguments:
  COMMAND
    clear             Clear a set of task instance, as if they never ran
    failed-deps       Returns the unmet dependencies for a task instance
    list              List the tasks within a DAG
    render            Render a task instance's template(s)
    state             Get the status of a task instance
    states-for-dag-run
                      Get the status of all task instances in a dag run
    test              Test a task instance

Options:
  -h, --help          show this help message and exit

airflow tasks command error: argument COMMAND: invalid choice: 'run' (choose from 'clear', 'failed-deps', 'list', 'render', 'state', 'states-for-dag-run', 'test'), see help above.
[2025-03-11T16:08:37.977+0530] {base_executor.py:453} DEBUG - Changing state: TaskInstanceKey(dag_id='example_branch_operator', task_id='branching', run_id='scheduled__2025-03-11T00:00:00+00:00', try_number=1, map_index=-1)

What you think should happen instead?

No response

How to reproduce

1. Set executor as SequentialExecutor in airflow.cfg
2. Run a sample example dag

Operating System

Ubuntu 20.04

Versions of Apache Airflow Providers

No response

Deployment

Virtualenv installation

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[tirkarthi]

cc: @ashb @amoghrajesh @kaxil . I am not sure if this is planned for Airflow 3 or to change the default executor value. Please feel free to remove task-sdk if not relevant. Thanks.

[amoghrajesh]

Yeah, we didnt port over SequentialExecutor. It is not even in the roadmap as there were mail discussions to remove ut iirc.

LocalExecutor runs fine even now and can be used as a Sequential one with certain parameter values

[tirkarthi]

Thanks, if it's not intended to be fixed then it might be better to switch the default for Airflow 3.0 . It was meant as a good to have or 3.1+ change but #47453 makes it required now.

Mailing list discussion : https://lists.apache.org/thread/cc7wfz9w8d1npopj72nlxnh52dpzbls9

[potiuk]

Yes. We should remove it and add newsfragment about it.

[amoghrajesh]

Two tasks here:

1. Changing the default to local executor
2. Removal of sequential executor, what was stopping us from doing it? I remember there was a slack thread soemwhere

[vikramkoka]

Yes, I believe there was an email thread regarding this started by @potiuk