Skip to content

Conversation

@ashb
Copy link
Member

@ashb ashb commented Mar 6, 2025

With the switch to Task SDK the hacky way we've had for 10 years of
re-invoking our CLI processor can finally go the way of the Dodo.


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@ashb ashb added the full tests needed We need to run full set of tests for this PR to merge label Mar 6, 2025
@ashb ashb requested a review from jedcunningham March 6, 2025 21:20
@ashb ashb force-pushed the remove-task-run-commands branch from 210d1ce to 04b9dcf Compare March 6, 2025 21:20
@ashb ashb requested a review from vincbeck March 6, 2025 21:20
@ashb ashb requested a review from potiuk March 6, 2025 21:20
@ashb
Copy link
Member Author

ashb commented Mar 6, 2025

I deleted LocalTaskJob too since it's tests were failing (and it was only used by airflow task run anyway) so I've requested a re-review

With the switch to Task SDK the hacky way we've had for 10 years of
re-invoking our CLI processor can finally go the way of the Dodo.

And since LocalTaskJob only existed to drive/monitor/heartbeat tasks from the
CLI, that can go too.
@ashb ashb force-pushed the remove-task-run-commands branch from 04b9dcf to 3753490 Compare March 6, 2025 22:03
@ashb ashb changed the title Remove the old task run commands Remove the old task run commands and LocalTaskJob Mar 6, 2025
@ashb ashb merged commit 58c6a49 into main Mar 7, 2025
89 checks passed
@ashb ashb deleted the remove-task-run-commands branch March 7, 2025 09:07
@tirkarthi
Copy link
Contributor

This seems to break running example dags on the main branch in my laptop. Is this some issue with SequentialExecutor not using task-sdk ?

[2025-03-08T10:15:41.699+0530] {base_executor.py:295} DEBUG - 0 running task instances for executor SequentialExecutor
[2025-03-08T10:15:41.700+0530] {base_executor.py:296} DEBUG - 1 in queue for executor SequentialExecutor
[2025-03-08T10:15:41.700+0530] {base_executor.py:301} DEBUG - 32 open slots for executor SequentialExecutor
[2025-03-08T10:15:41.701+0530] {base_executor.py:253} DEBUG - Calling the <class 'airflow.executors.sequential_executor.SequentialExecutor'> sync method
[2025-03-08T10:15:41.701+0530] {sequential_executor.py:85} INFO - Executing command: ['airflow', 'tasks', 'run', 'tutorial', 'print_date', 'scheduled__2025-03-08T04:45:32.805274+00:00', '--local', '--subdir', 'DAGS_FOLDER/tutorial.py']
Usage: airflow tasks [-h] COMMAND ...

Manage tasks

Positional Arguments:
  COMMAND
    clear             Clear a set of task instance, as if they never ran
    failed-deps       Returns the unmet dependencies for a task instance
    list              List the tasks within a DAG
    render            Render a task instance's template(s)
    state             Get the status of a task instance
    states-for-dag-run
                      Get the status of all task instances in a dag run
    test              Test a task instance

Options:
  -h, --help          show this help message and exit

airflow tasks command error: argument COMMAND: invalid choice: 'run' (choose from 'clear', 'failed-deps', 'list', 'render', 'state', 'states-for-dag-run', 'test'), see help above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:CLI full tests needed We need to run full set of tests for this PR to merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants