Add filters to Browse XComs #54049

RoyLee1224 · 2025-08-02T13:11:06Z

Related Issue

Changes Made

Added search filters for Key, Dag display name, Task ID,Run ID, logical date, run after .
~~- Note: The current implementation requires an exact match for the filter values.~~

Screenshot

xcom_filters.mp4

TO-DO

Add collapsible filter panel(Advanced filters) for better UX
Propose backend API extensions for pattern matching
Implement remaining filters (Logical Date Range, Run After Range)
Implement map_index filter with NumberInput
Hide some filters in the TI Xcom tab depending on the context

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

jscheffl · 2025-08-03T16:59:29Z

Looks good on the first view. When I test I realize that an exact match is needed, also no wildcard is supported. Can this be added?

RoyLee1224 · 2025-08-03T19:35:13Z

Thanks for the review @jscheffl!

Yes, I'll modify the backend to use _SearchParam with paginated_select(). If I understand correctly, this will apply the ilike for partial matching as seen in other APIs.

I'll add this once the UI work is finalized.

RoyLee1224 · 2025-08-03T19:50:58Z

UI Update

Here’s a quick demo of the new Advanced filter. Hope this help us to push the main issue forward.

filter.mp4

I've avoided abstracting these components for now to prevent premature optimization, as the UI design isn't final yet. I'll make them reusable for other pages once the design stabilizes.

pierrejeambrun

Nice.

Do not incorporate the collapsing of filters etc... in this PR. (FilterManager.tsx) It is a separate one because we will probably need to discuss about the design. We should keep the same structure that other page filters, for instance in your example there is no 'reset filter' button, etc.

RoyLee1224 · 2025-08-04T19:04:58Z

@pierrejeambrun Sure! I'll scope this PR down to focus on the core filtering functionality.

My initial reason for adding the collapsible panel was based on the "UI Impact" section in the related issue #53056, but I completely agree that a separate PR is better for discussing and implementing.

I'll make a separate PR for those UI changes after finishing the basic filter. 💪

RoyLee1224 · 2025-08-04T19:15:28Z

While researching, I found that the filter pattern in Airflow 2 looks like this:

For the temporary UI in this PR, should I use the simple multi-input bar design in Airflow 2 as a baseline?

fix(api): Refactor XCom entries filter fix(api): Correct XCom endpoint filtering

RoyLee1224 · 2025-08-08T20:24:32Z

I noticed that filtering by map_index with fuzzy string search would led to incorrect results:
searching for "1" also show "10", "11", or "-1".

So I changed map_index filter to exact match.

jason810496

Nice! Thanks for the PR!

Non-blocking:
It would be great to add some test cases for new filters in airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_xcom.py, thanks.

jscheffl

I also think this is looking good now!

One thing that jumps into my view now - but is something that might be a follow-up PR: The view in the tag of a task instance is fitlering on the task instance. But (1) filters as well as (2) table columns are not needed for these pre-selections as dag, run, task, map-index are filtered. That might make the screen a bit cleaner inside the tab of the grid view:

could be

RoyLee1224 · 2025-08-11T02:44:56Z

@jason810496 Sure, I'll give it a try!

RoyLee1224 · 2025-08-11T02:52:37Z

@jscheffl I'm happy to open a follow-up PR to hide the filter bars and clean up the UI for task instance Xcom tab.

Before I start, I want to make sure we're all aligned. @pierrejeambrun mentioned previously in this PR that he'd prefer to keep the filtering options on detail pages.

To make sure I build the right thing, could we get a consensus on the final direction?

pierrejeambrun

A few things to address but looking good overall!

airflow-core/src/airflow/api_fastapi/core_api/routes/public/xcom.py

airflow-core/src/airflow/ui/src/pages/XCom/XCom.tsx

airflow-core/src/airflow/ui/src/pages/XCom/XComFilters.tsx

jscheffl · 2025-08-11T16:02:23Z

@jscheffl I'm happy to open a follow-up PR to hide the filter bars and clean up the UI for task instance Xcom tab.

Before I start, I want to make sure we're all aligned. @pierrejeambrun mentioned previously in this PR that he'd prefer to keep the filtering options on detail pages.

To make sure I build the right thing, could we get a consensus on the final direction?

Yeah, follow-up PR is totally fine!

pierrejeambrun · 2025-08-11T22:29:13Z

I'm happy to open a follow-up PR to hide the filter bars and clean up the UI for task instance Xcom tab.

Yes Jens is right, we need to hide some filters in the details tab depending on the context. (those that are set in the URL should not appear).

I tend to recommend to do it in this PR because this PR is not self sufficient and the follow up is required. If for some reason we can't work on the follow up, we forget to do it or it gets delayed, the UI is in a blocking state. Unless you're sure you can follow up in a short term, then it doesn't really matter, and as you prefer.

RoyLee1224 · 2025-08-11T23:16:26Z

Update

1. Hide some filters in the details tab depending on the context

2. Use NumberInput for num filters

NumberInput.mp4

bbovenzi · 2025-08-12T16:45:34Z

UX-wise. The intention of Advanced Search was to open a dropdown to show all of these search fields instead of them taking up so much screenspace at all times

Gmail advanced search for example:

RoyLee1224 · 2025-08-12T23:25:05Z

@bbovenzi

Agreed, collapsing the advanced search is a good idea. Notion's filter design is also a great example. It lets users combine multiple conditions with direct results, the UI is clean, and the functionality is flexible. I personally think it's an even better pattern than the traditional dropdown form.

Notion filters example

notion_filter.mp4

RoyLee1224 · 2025-08-12T23:48:02Z

It could look like this:

filterPill_demo.mp4

bbovenzi · 2025-08-14T21:01:57Z

@RoyLee1224 Oh yeah that's much better. Let's do that in a separate PR

RoyLee1224 · 2025-08-15T03:11:31Z

@bbovenzi Sure, I'm happy to open follow-up PR for new filter design. And I'll try to make it reusable for other pages.

(from https://github.com/apache/airflow/tree/python-client/3.1.0rc1) ## New Features: - Add `map_index` filter to TaskInstance API queries ([#55614](apache/airflow#55614)) - Add `has_import_errors` filter to Core API GET /dags endpoint ([#54563](apache/airflow#54563)) - Add `dag_version` filter to get_dag_runs endpoint ([#54882](apache/airflow#54882)) - Implement pattern search for event log endpoint ([#55114](apache/airflow#55114)) - Add asset-based filtering support to DAG API endpoint ([#54263](apache/airflow#54263)) - Add Greater Than and Less Than range filters to DagRuns and Task Instance list ([#54302](apache/airflow#54302)) - Add `try_number` as filter to task instances ([#54695](apache/airflow#54695)) - Add filters to Browse XComs endpoint ([#54049](apache/airflow#54049)) - Add Filtering by DAG Bundle Name and Version to API routes ([#54004](apache/airflow#54004)) - Add search filter for DAG runs by triggering user name ([#53652](apache/airflow#53652)) - Enable multi sorting (AIP-84) ([#53408](apache/airflow#53408)) - Add `run_on_latest_version` support for backfill and clear operations ([#52177](apache/airflow#52177)) - Add `run_id_pattern` search for Dag Run API ([#52437](apache/airflow#52437)) - Add tracking of triggering user to Dag runs ([#51738](apache/airflow#51738)) - Expose DAG parsing duration in the API ([#54752](apache/airflow#54752)) ## New API Endpoints: - Add Human-in-the-Loop (HITL) endpoints for approval workflows ([#52868](apache/airflow#52868), [#53373](apache/airflow#53373), [#53376](apache/airflow#53376), [#53885](apache/airflow#53885), [#53923](apache/airflow#53923), [#54308](apache/airflow#54308), [#54310](apache/airflow#54310), [#54723](apache/airflow#54723), [#54773](apache/airflow#54773), [#55019](apache/airflow#55019), [#55463](apache/airflow#55463), [#55525](apache/airflow#55525), [#55535](apache/airflow#55535), [#55603](apache/airflow#55603), [#55776](apache/airflow#55776)) - Add endpoint to watch dag run until finish ([#51920](apache/airflow#51920)) - Add TI bulk actions endpoint ([#50443](apache/airflow#50443)) - Add Keycloak Refresh Token Endpoint ([#51657](apache/airflow#51657)) ## Deprecations: - Mark `DagDetailsResponse.concurrency` as deprecated ([#55150](apache/airflow#55150)) ## Bug Fixes: - Fix dag import error modal pagination ([#55719](apache/airflow#55719))

feat(ui): Add filters for Key, DAG, Task, and Run ID to XCom page

0dbf93f

RoyLee1224 requested review from bbovenzi, jscheffl, pierrejeambrun, ryanahamilton and shubhamraj-git as code owners August 2, 2025 13:11

boring-cyborg bot added area:translations area:UI Related to UI/UX. For Frontend Developers. translation:default labels Aug 2, 2025

RoyLee1224 marked this pull request as draft August 2, 2025 13:11

feat: implement basic search with filter pill

48c3255

chore(ui): modify filterpill appearance

b45e340

style(ui): Polish appearance of filter components

92dec9f

pierrejeambrun reviewed Aug 4, 2025

View reviewed changes

RoyLee1224 and others added 5 commits August 5, 2025 22:44

feat(api, ui): Add wildcard search to XComs and simplify filter UI

806126f

feat(api): Add dag, run, task wildcard search to XComs

93a4064

fix(api): Refactor XCom entries filter fix(api): Correct XCom endpoint filtering

refactor(ui): Simplify XCom filters

37472f9

feat(api, ui): Add Logical Date Range, Run After Range filter to Xcoms

e119a12

Merge branch 'main' into filters

e483522

RoyLee1224 marked this pull request as ready for review August 6, 2025 11:24

RoyLee1224 requested review from Lee-W, bugraoz93, ephraimbuddy, jason810496 and rawwar as code owners August 6, 2025 11:24

jason810496 reviewed Aug 9, 2025

View reviewed changes

jscheffl reviewed Aug 9, 2025

View reviewed changes

pierrejeambrun mentioned this pull request Aug 11, 2025

Add filtering support to Audit Log view #54210

Merged

pierrejeambrun reviewed Aug 11, 2025

View reviewed changes

fix(api): keep xcom_key and map_index

19534e8

fix(ui): improve XCom filter UX with NumberInput

fa320ed

RoyLee1224 added 2 commits August 12, 2025 06:58

fix(ui): showing only relevant filters in TI Xcom tab

d5a4ab6

fix(ui): replace unicode placeholder with CSS layout

c68729e

RoyLee1224 added 2 commits August 12, 2025 07:28

Merge branch 'main' into filters

ca2f0a8

Merge branch 'main' into filters

115f386

bbovenzi approved these changes Aug 14, 2025

View reviewed changes

bbovenzi added this to the Airflow 3.1.0 milestone Aug 14, 2025

bbovenzi merged commit e00777a into apache:main Aug 14, 2025
199 of 200 checks passed

RoyLee1224 mentioned this pull request Aug 17, 2025

chore(ui): remove redundant enabled check #54579

Merged

RoyLee1224 deleted the filters branch August 17, 2025 22:32

RoyLee1224 mentioned this pull request Aug 25, 2025

Add reusable FilterBar component for date, number, text input #54895

Merged

1 task

kaxil mentioned this pull request Oct 22, 2025

Update Python Client to 3.1.0 apache/airflow-client-python#138

Merged

Add filters to Browse XComs #54049

Add filters to Browse XComs #54049

Uh oh!

Conversation

RoyLee1224 commented Aug 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Related Issue

Changes Made

Screenshot

TO-DO

Uh oh!

jscheffl commented Aug 3, 2025

Uh oh!

RoyLee1224 commented Aug 3, 2025

Uh oh!

RoyLee1224 commented Aug 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

UI Update

Uh oh!

pierrejeambrun left a comment

Choose a reason for hiding this comment

Uh oh!

RoyLee1224 commented Aug 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

RoyLee1224 commented Aug 4, 2025

Uh oh!

RoyLee1224 commented Aug 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jason810496 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jscheffl left a comment

Choose a reason for hiding this comment

Uh oh!

RoyLee1224 commented Aug 11, 2025

Uh oh!

RoyLee1224 commented Aug 11, 2025

Uh oh!

pierrejeambrun left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jscheffl commented Aug 11, 2025

Uh oh!

pierrejeambrun commented Aug 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

RoyLee1224 commented Aug 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Update

1. Hide some filters in the details tab depending on the context

2. Use NumberInput for num filters

Uh oh!

bbovenzi commented Aug 12, 2025

Uh oh!

RoyLee1224 commented Aug 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Notion filters example

Uh oh!

RoyLee1224 commented Aug 12, 2025

Uh oh!

bbovenzi commented Aug 14, 2025

Uh oh!

Uh oh!

RoyLee1224 commented Aug 15, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

RoyLee1224 commented Aug 2, 2025 •

edited

Loading

RoyLee1224 commented Aug 3, 2025 •

edited

Loading

RoyLee1224 commented Aug 4, 2025 •

edited

Loading

RoyLee1224 commented Aug 8, 2025 •

edited

Loading

jason810496 left a comment •

edited

Loading

pierrejeambrun left a comment •

edited

Loading

pierrejeambrun commented Aug 11, 2025 •

edited

Loading

RoyLee1224 commented Aug 11, 2025 •

edited

Loading

RoyLee1224 commented Aug 12, 2025 •

edited

Loading