Repository navigation
[Filters] Add filtering support to Search – DAG Runs #53043
Description
Activity
- added a parent issue
on Jul 8, 2025 - changed the title
[-][Filters] Add filtering support to DAG Runs View (Search – DAG Runs)[/-][+][Filters] Add filtering support to Search – DAG Runs[/+]on Jul 8, 2025 I would like to work on this
Go head @babyitachi, you're assigned
Reacted by Abhishrut Vaidya@babyitachi , how are we progressing on this?
Hey @vatsrahul1001, I could not proceed with this one with the pace I wanted to. Give me till weekend, I'll post an update. Thanks!
Reacted by Pierre Jeambrun@babyitachi , do you have an update?
I'm assigning myself to this to help with the API part. It would be great if you could help with the UI part of the issue.
Edit: I'm also looking into the UI part, just in case @babyitachi can't finish this one.
I'm not sure if we should support Duration range. We don't have this data readily available and DagRun needs to be computed on the go. Also, our search filters are currently triggered by change in the field. If someone wants to type "523" seconds. I am not sure if its going to hit queries for 5 and then 52 and then 523 by the time user completes typing into the search box. And, that I think is going to be compute intensive on DB. Maybe not too much when user is already on a Dag Page. As we do filter by Dag Id. But, if on the page Dags-> DagRuns, its going to run this on all of the data in DagRun table.
If my concern sounds legit, we can update DagRun Model to have duration column(its currently a hybrid property). Then have duration_<lte|lt|gte|gt> params
If someone wants to type "523" seconds. I am not sure if its going to hit queries for 5 and then 52 and then 523 by the time user completes typing into the search box. And, that I think is going to be compute intensive on DB.
That part is handled by a debounce for the search box, and we should probably do the same for other inputs that might be query sensitive.
data readily available and DagRun needs to be computed on the go
We already have a hybrid_property for that. We should be able to query and filter transparently on that virtual column. Yes the computation will add to the query time but it's a simple one and we shouldn't notice it.(I would say, it's worth to give it a try and see how much that adds up on a table with a fair number of dag runs. >10k maybe more).
More than happy to create a new PR carrying this forward. I will link the new one here shortly. 💪
Reacted by Kalyan RReacted by Kalyan RAh! I wasn't aware folks were already working on this. Some of the AIs here I have already implemented (and merged)
see: #55691 , #55648 , #53652Thanks for that. I have updated the ticket. (filters for triggering_user_name_pattern, and hyperlinks are nice addition but were not part of this ticket, so we're fine)
Reacted by Dheeraj Turaga and LI,JHE-CHENClosed by #55735
Special remaining items such as consuming or producing assets can come later as follow up improvement, those are more efforts
Reacted by LI,JHE-CHEN
View
Search – DAG Runs
Filters to Add
Filter by DAG ID
↪ Useful when triaging DAGs across environments or viewing all runs for a specific DAG without switching views
Filter by Run ID
↪ Useful for debugging across views and linking from external systems or audit logs
Filter by Start Date
↪ Helps find DAG runs that started within a specific time window (e.g., peak load periods)
Filter by End Date
↪ Useful for spotting recently completed runs or identifying DAGs that finished during outages or incidents
Filter by Duration Range
↪ Identify slow or long-running DAGs to tune performance or spot issues with stuck tasks
Filter by DAG Version and/or Bundle Version
↪ Correlate failures or behavioral changes with code changes over time
Filter by Conf Content
↪ Search for runs where conf contains specific keys or values (e.g., for dynamic configuration or user-specific logic)
Filter by Triggered By
↪ Distinguish manual runs, asset-triggered runs, CLI-triggered runs, or other automated triggers; important in hybrid scheduling
Filter by Producing Asset
↪ Show runs created as a result of an upstream asset event, useful for event-driven DAGs and lineage debugging
Filter by **Logical Date ** or Run After (Range)
↪ Logical Date is the canonical “execution date” and is often used for backfill or partitioned logic; “Run After” helps with when something is scheduled to run, independent of the logical date
API Support
/dagRunssupports query params:dag_id,execution_date,state,run_typeWould require API enhancements to support full filter set.
UI Impact