Repository navigation
UI - N+1 Calls in the Tasks Tab #59598
Description
Apache Airflow version
Other Airflow 3 version (please specify below)
If "Other Airflow 3 version" selected, which one?
3.1.3
What happened?
When browsing the Airflow UI and looking at the tasks tab, it triggers n+1 queries for every task in your dag. If your dag has 200+ tasks, it does 200+ queries which should not be the case. The first 30 might succeed but afterwards the calls fail due to a sqlalchemy timeout.
problematic UI path: <airflowBaseUrl>/airflow/dags/<some-dag>/tasks
UI:
Logs of the api-server where only 1 user is using it:
What you think should happen instead?
Visualizing and working with dags that have many tasks should work. The UI should not do a call for every task in the dag as that can never scale. Redesign of parts of the UI might be needed.
I saw in the issues that work has been done to fix n+1 queries (#57561), but it seems this path has been missed.
How to reproduce
Create a dag with 200 tasks in it and make sure there are a couple of dagruns (5 or so).
Go to the dag (gridview) and then click on tasks tab on the right panel. See previous screenshot.
Check the logs and you will see that 200 calls are triggered to fetch task instances for every task in the dag.
The 200 calls done against the backend look as follows:
<airflowBaseUrl>/airflow/api/v2/dags/<some-dag>/dagRuns/~/taskInstances?task_id=<some-task>&limit=14&order_by=-run_after
Operating System
ubuntu
Versions of Apache Airflow Providers
apache-airflow==3.1.3
apache-airflow-core==3.1.3
apache-airflow-providers-cncf-kubernetes==10.6.0
apache-airflow-providers-common-compat==1.9.0
apache-airflow-providers-common-io==1.6.5
apache-airflow-providers-common-sql==1.29.0
apache-airflow-providers-openlineage==2.9.0
apache-airflow-providers-postgres==6.4.1
apache-airflow-providers-smtp==2.3.2
apache-airflow-providers-standard==1.9.2
apache-airflow-task-sdk==1.1.3
Deployment
Other Docker-based deployment
Deployment details
We deploy airflow on kubernetes similarly to the officical Airflow helm chart but than using our own operator.
Anything else?
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
Activity
[-]n+1 queries still exists in the Airflow UI[/-][+]n+1 calls problem still occuring in the Airflow UI[/+]sania-2912 commentedon Dec 18, 2025
Hi, I’d be happy to take a look at this issue.
Since it’s currently unassigned, I can start investigating it. From an initial look, it seems that the Tasks tab in the UI may be triggering one API call per task, which can lead to N+1 calls for larger DAGs and cause scalability issues.
I’ll begin by profiling the UI/API interaction and share my findings here before proposing or opening a PR.
Thanks!
potiuk commentedon Dec 18, 2025
I think it's worth checking if it is the same for 3.1.5 ? This is the latest version released - and the report is about 3.1.3 which is quite "old news" already and a number of things in UI has been fixed.
potiuk commentedon Dec 18, 2025
Can you confirm @nclaeys that it's the same in 3.1.5 ?
tirkarthi commentedon Dec 18, 2025
I think this is an intentional design in #46296 . In my initial implementation in #44604 I used the all task instances for a dag with dag run id as "~" which returned only 100 task instances. Perhaps this page can use a UI specific API since all the fields from public API for a task instance are not used here.
cc: @bbovenzi
nclaeys commentedon Dec 22, 2025
@potiuk I can confirm the same logic still exists on Airflow 3.1.5. I have just tested it.
Shally-Katariya commentedon Dec 27, 2025
Hi! I recently contributed a fix related to unnecessary polling in the Airflow UI
(HITL polling on the DAG page).
I’m interested in helping further with UI performance issues, and I’ve been
looking into the N+1 calls behavior on the Tasks tab.
I plan to first understand the current data-fetching flow and the linked PR (#59814),
and I can help with testing, review, or follow-up improvements if needed.
Shally-Katariya commentedon Dec 27, 2025
Summarizing the issue:
1️ The issue affects the DAG Tasks tab in the Airflow UI (
/dags/<dag_id>/tasks).2️ The problem is an N+1 query pattern where the UI makes one
taskInstancesAPI call per task, leading to hundreds of requests for large DAGs.
3️ A fix is already in progress and linked via PR #59814 by @Arunodoy18,
which batch-fetches task instances in
Tasks.tsxand avoids per-task requests.I tested the changes from PR #59814 locally using Breeze, and the N+1 calls
are no longer present on the Tasks tab. Task cards render correctly with
batched data.
Once the PR is merged, this issue should be resolved.
2 remaining items
github-actions commentedon Jan 19, 2026
This issue has been closed because it has not received response from the issue author.
nclaeys commentedon Jan 19, 2026
@tirkarthi not stale, this issue still needs resolving. The github actions configuration is very strict here, there was no open question for me to answer on.
tirkarthi commentedon Jan 19, 2026
There were recent changes to close issues with pending response label. Please check on latest Airflow version. Please feel free to propose a PR if you have a solution if exists on latest Airflow so that it can be reviewed or we need to wait for someone to take a look.
nclaeys commentedon Jan 19, 2026
Well it feels a bit unfair to the people spending the effort in writing a detailed bug report to just close it after 2 weeks without resolving the issue.
I liked the proposal of @Shally-Katariya as it will fix this issue structurally but the PR got closed due to issues. As a result this ticket had no work done, so it is not fixed.
I think the pending-response label is wrong here. I think the problem is pretty clearly described for someone to pick it up.
[-]n+1 calls problem still occuring in the Airflow UI[/-][+]UI - N+1 Calls in the Grid[/+][-]UI - N+1 Calls in the Grid[/-][+]UI - N+1 Calls in the Tasks Tab[/+]pierrejeambrun commentedon Jan 20, 2026
@nclaeys Yes, we forgot to remove the
pending-responselabel, hence the auto closing. Sorry for that.Indeed the issue you mention still occurs, that's caused by the Tasks cards trying to load past TI execution. Compared to the Dags card that uses a specific endpoint fetching all at once and that is paginated.
First of all the
get_tasksendpoint should probably be paginated. We don't want 1 api call returning 5000 tasks if the dag is huge. THat would limit the number of parallel call in the UI.I think we need to convert the UI table into a simple 'table' not 'card'. If we really want card, we need an extra ui endpoint similar to get_dags
uithat will return 'recent_runs' as well in the endpoint. So 1 call -> 1 page of tasks with 14 recent execution.I also just want to mention that this is about another problem (hence the confusion). It fixed how the backend would perform N+1 db queries for a single API endpoint call. The issue reported here is how the UI would make multiple calls to the backend to retrieve information.
pierrejeambrun commentedon Jan 20, 2026
Opened #60830 to fix the problem.
nclaeys commentedon Jan 21, 2026
@pierrejeambrun Thanks for the clarification. I like your plan, looking forward to the solution as it will help our users a lot.