Add row selection + bulk action to table views in Airflow UI

[paolaperaza]

Description

When users are writing and testing new DAGs, migrating from one Airflow service to another, or managing many of dynamically created DAGs -- it's common to need to pause or unpause all DAGs or a subset fo DAGs at once.

For those with 100s of DAGs in a single Airflow environment, this is almost impossible to do -- and not being able to take this action can result in loss of data freshness, duplicate DAG runs, and production-grade issues.

Use case/motivation

A few things:

• Add a way to "Multi-Select" DAGs in the DAG view of the Airflow UI
• Add a way to "Select All" DAGs in the DAG view of the Airflow UI
• With the DAGs that you select, be able to take the following bulk actions:
  • Pause
  • Unpause
  • Delete
  • Trigger

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[collinmcnulty]

If you want this for pausing all DAGs at once, I think use_job_schedule will achieve that goal.

[dstandish]

what do y'all think of a switch that just disables the scheduler?

[dstandish]

So one approach would be you add a button in top right (or somewhere) that is essentially a kill switch. My thought is it would do the exact same thing as the existing use_job_schedule param but the difference is it would be controllable from the UI and rest API.

The cost though is ... there's now a kill switch button on the home page ... and it would require another round trip to the DB in each scheduler loop (because we'd have to stick the status in a table and query it in each loop).

Do we think that it's worth it? Do we like having a button on home page? Is this hijacking the original request?

[jscheffl]

I feel "turning all off" is might be very simple but if you have a selective set enabled, then re-enabling still is a tedious job. If I would need to make this I probably would create a script that calls the API in a loop. Having a single "kill switch" might be dangerous if somebody clicks on this by accident.
Still is a valid feature having a "simple" administrative "halt" of scheduler. If somebody wants to contribute this is valid...

[dstandish]

Still is a valid feature having a "simple" administrative "halt" of scheduler. If somebody wants to contribute this is valid...

You write Having a single "kill switch" might be dangerous if somebody clicks on this by accident but then also Still is a valid feature having a "simple" administrative "halt" of scheduler. If somebody wants to contribute this is valid... -- they seem contradictory so I must misunderstand something. Are you saying that we should not have a UI kill switch but should have an API kill switch? Or perhaps you are saying "kill switch is dangerous but we should have it anyway"?

[jscheffl]

Or perhaps you are saying "kill switch is dangerous but we should have it anyway"?

I was referring to the "danger" that if suh kill switch would be implemented on the UI the button should not be sooo present that people click by accident and then take a production environment down.
So the kill switch need to have an extra security layer like https://images.carid.com/dorman/electrical-parts/84815.jpg :-D

[utkarsharma2]

IMO this feature is useful, I personally have faced this need earlier and it's tedious to do any action one by one. Also, when it comes to the danger of accidentally clicking a kill all button that is certainly risky.
However, we can have a flow instead of a single action like below:

Step 1: User selects the dag(s) to act on

Step 2: Then once when the user selects what to do. we can confirm with the user.

Not really good with the diagrams :)

Also we are already making a periodic call(as part of auto-refresh) to http://localhost:28080/dag_stats maybe we can extent the API to also get additional states?

[bbovenzi]

Yes, when we rewrite the dags list in React this would be great to add along with the more advanced tag filters.

"Filter to all dags that are tag X or Y, select all, and then unpause all of them"