AIP-38 | Migrate to new UI

[bbovenzi]

Meta epic to track all legacy UI pages we need to migrate over to the new UI

See here for designs: #42371

I'll probably update the main issue description. But here's the priority for the month:

December core user stories

• Dags List
  • Dag Card to show last and next runs and a chart of recent runs
  • Can pause/unpause a Dag from the list
  • Can trigger a dag from the list AIP-38 | Connect Trigger form to rest API #44857
  • Filter dags by last dag run state, paused, and tags
  • Sort dag list by display name, latest run, next run,
  • Search for a dag by dag_id or dag_display_name
  • Show dag import errors on dag list level
• Dag Details
  • Click on a row in the Dags list to open up Dag Details
  • Show header that has the same information as the Dag Card and includes actions to:
    • Trigger basic Dag AIP-38 | Connect Trigger form to rest API #44857
    • Use dag params to generate forms for triggering a dag AIP-38 | Use Dag params to build an advanced Trigger Dag Run form #44858
  • List all dag runs and link to a specific Dag Run
  • List all tasks
    • Include task group information
  • Render Dag Code
• Dag Visualizations
  • Graph
    • Show the nodes and edges of a graph
    • Toggle showing the external dependencies of a graph (sensors, triggers, data assets with conditions)
    • Show a specific dag run in the graph with each node filled with task instance or asset event info
    • Click on a task to select it as a task or task instance
  • Grid
    • Explore use of icons for each state to be more accessible. And how to balance that with run_type, try_number and has a note functionality
  • Gantt
  • Dropdown selector on each viz to change between dag runs
  • Side drawer on each viz to see details about a task instance or dag run and the ability to quickly clear or mark as a failure
  • Auto refresh grid data to see dags running
• Task Details
  • List all task instances
• Dag Run Details
  • List all task instances
  • Manually mark dag run as failed or success
  • Clear a dag run
• Task Instance Details
  • Manually mark a task failed or success
  • Clear a task instance AIP-38 | Clear Task Instance #44860
  • See task instance logs across tries AIP-38 | Add Task Instance Logs #44663
  • See task instance details across tries
  • List all mapped task instances
    • Allow selecting a single mapped instance with all task instance functionality
  • See task instance xcoms AIP-38 | Add Task Instance XCom #44667

January

• Backfills (theoretically optional - ui support is net new from AF2)
  • Trigger a backfill for a dag and see how many runs will be created before finally submitting
  • See if a dag has active backfills and see the state of all the backfilled runs
• Versioning
  • Show version on dag/dagrun/task/task instance
  • When viewing an older dag run, fetch the dag code for that version
  • Fetch the dag structure for an older version or give an alert that the structure has changed
  • Rework "clear" to be bundle/version aware

Dashboard

Import Errors #43711
Pool Summary #43328
Historic Dag Run metrics #42700
Quick links to DAGs List #44673
Add an event timeline to help with date range selection
Summary of components ( scheduler(s) , celery-worker(s) , triggerer(s) , dag-processor(s) )

Dags List

Search by display_name #42714
Filter by tags #43712
Filter by last run state #42715
Update sort options #43043

Dag Details

Trigger
Versioning (to scope)
Overview
List Runs
List Tasks
Grid View #44671
Graph View #44200
Gantt Chart #44672
Code #43713
Events per Dag #43705

Backfills

List backfills #43967
Active backfills banner #43968
Backfill details
Create a new backfill #43969
Pause/cancel backfill

Dag Run Details

Header Card
Clear Run
Mark run as...
Change Dag Run Note

Task Details #44669

Header Card
Overview
List Task Instances

Task Instance Details

Header Card
Logs #44663
XCom #44667
Rendered templates
Clear TI
Mark TI as...
Change TI Note

Assets

To be scoped

Admin

Connections

List Connections #43703
Edit Connection
Delete Connection
Test Connection

Events

List Events #43704
Filter events

Pools

List Pools #43706
Edit Pool
Delete Pool
Create Pool

Providers

List Providers #43708

Variables

List Variables #43709
Add Variable
Edit Variable
Delete Variable

User

Light/Dark toggle #42823 #11334
Timezone selection #42817
Color customization #43054

Plugins

New React UI Plugins #42702
Iframe Plugins #42708

Config

Remove unnneeded config #43519
Render UI with config values

[hterik]

Filter by tags #43715

This appears to link to something else.

[bbovenzi]

Filter by tags #43715

This appears to link to something else.

Good catch. Fixed.

[pierrejeambrun]

I feel like we have everything from the backend perspective. (or planned such as asset dependencies).

The only points I'm not so sure are:

• Include task group information (I'm not sure we have this)
• Show dag import errors on dag list level (I am not sure how we want to display them, as in the legacy we can put them at the top of the dag list, but I don't think we can group them by dag_id there's not really a relation there in the db model)

[bbovenzi]

I feel like we have everything from the backend perspective. (or planned such as asset dependencies).

The only points I'm not so sure are:

• Include task group information (I'm not sure we have this)
• Show dag import errors on dag list level (I am not sure how we want to display them, as in the legacy we can put them at the top of the dag list, but I don't think we can group them by dag_id there's not really a relation there in the db model)

• I think almost all Task Group information is handled by grid_data and structure endpoints
• We literally just need to copy the component from the home page and also show it at the top of the page on the dags list