Repository navigation
Links to specific dagruns don't work when older than 25 runs #34723
Description
Activity
I cannot reproduce with main:
Could you provide some screenshots for your problem?
Sure, hope this is enough
The url was given by clicking a run in the List dag runs page.
Something interesting to observe is that after clicking the url, it redirects 2 times, first adding thedag_run_idparameter (notrun_id), which then quickly gets removed again.This date is the max date for the X dag runs, when you choose 2023-01-01 as start date, and 25 as a number of runs, the UI will show you the previous 25 runs before 2023-01-01. To select a dag run, you need to click on the red/green bar in the grid view.
This was not the case in Airflow 2.5, i could click on links referring to any dagrun, no matter how old, and it would correctly select it in the grid-view.
The selection of date and number of runs shouldn't have to be provided manually if run_id is part of the url.To clarify, i am browsing the dagrun list attached below and clicking an item in the picture 1 here, then it results in the picture 2 (same as above) where no run is selected:
I did a test with an example DAG, triggered this manually >25 times. As the GRID view shows 25 items per default all is fine if you click on one of the recent 25 items.
If you click on an item >25 runs, GRID view will not adjust the selection of the 25runs to include in the column section your run. In my setup it shows the grid view, but clicking on the "Details" tab it shows the DAG overview, not details of the (previously) selected DAG runs. So in total it is hard to navigate from a DAG search result to the GRID view t check for details. (Especially because in GRID view the search options are limited compared to the DAG search screen)
Reacted by HTRafal and Joseph MinI've done some debugging:
Problem 1.
Dag list is linking to the legacy/graphendpoint. This endpoint is only for backwards compatibility purposes and redirects to the new/gridendpoint.
This old function does actually calculate thebase_datebased onrun_id, insidedt_nr_dr_data = get_date_time_num_runs_dag_runs_form_data(request, session, dag)
https://github.com/apache/airflow/blob/0c8e30e43b70e9d033e1686b327eb00aab82479c/airflow/www/views.py#L216-230
But then it forgets to forward thedt_nr_dr_data["base_date"]in the returned kwargs:
https://github.com/apache/airflow/blob/0c8e30e43b70e9d033e1686b327eb00aab82479c/airflow/www/views.py#L2964-2969
SolutionA: Forward base_date
SolutionB: Change dag list to use new urlProblem 2
The new grid endpoint, receives therun_idparameter correctly, after the redirection described above.
However, it then performs a Javascript fetch call to thegrid_dataAPI. In this API call, therun_idis NOT included.
Thegrid_datafunction that serves this request has not implemented handling of run_id parameter, in the same way that the oldget_date_time_num_runs_dag_runs_form_datafunction did.
https://github.com/apache/airflow/blob/0c8e30e43b70e9d033e1686b327eb00aab82479c/airflow/www/views.py#L3475-3478
Solution C: Implement run_id handling in grid_data and forward the parameter from the frontend.I think both solution A and C are must haves here. B is optional, more nice to have to get rid of legacy in the future.
12 remaining items
I found a workaround to this problem in v2.7.3. If you add
&base_date=<same as execution date>to the URL, it will locate runs older than the latest 25. Here's an example link that is working for me:<base_url>/dags/<dag_id>/grid?execution_date=2023-12-14+19%3A10%3A24.218280%2B00%3A00&base_date=2023-12-14+19%3A10%3A24.218280%2B00%3A00&tab=graph&dag_run_id=<run_id>Reacted by Quinten BruynseraedeThanks @jermaine151, that seems to work indeed. It doesn't visually highlight a dagrun in the grid view (see Screenshot), but if you navigate to the Graph tab, then click on a Task it does lead you to the dagrun from the URL.
That last bit is important because in our case multiple runs are triggered on the same second, the gridview doesn't distinguish between those.
Reacted by Jermaine L.Thanks @jermaine151, that seems to work indeed. It doesn't visually highlight a dagrun in the grid view (see Screenshot), but if you navigate to the Graph tab, then click on a Task it does lead you to the dagrun from the URL.
That last bit is important because in our case multiple runs are triggered on the same second, the gridview doesn't distinguish between those.
Are you adding the
&dag_run_id=to the URL? That should land you on the right page, with the run selected.cc: @bbovenzi @pierrejeambrun - would be cool to take a look and fix that one as it seems to be an important feature for those who complain about loosing the old grid view (see #36884) and it seems a bit of an architectural decision on the grid view behaviour.
Reacted by Jens Scheffler- added a commit that references this issue
on Feb 20, 2024 - added a commit that references this issue
on Feb 20, 2024 - added a commit that references this issue
on Feb 22, 2024 This still happens in 2.9.1 - very frustrating experience troubleshooting a failed fag
Reacted by elivakninkUnlikely Anything will be done about it - Airflow 3 is behind the corner with completely new UI,
Similar problems persists in Airflow 3.2.2. I think it doesn't show the incorrect details any more but the selection of the grid view is difficult to understand and easy to misclick when the link goes beyond the most recent runs. I can file a new issue about this.
Apache Airflow version
2.7.1
What happened
AIRFLOW/dagrun/list/?_flt_3_dag_id=my_dagIt should take you to
AIRFLOW/dags/mydag/graph?run_id=scheduled__2023-09-21T00%3A00%3A00%2B00%3A00&execution_date=2023-09-21+00%3A00%3A00%2B00%3A00and it does so correctlyWhen clicking on any item in the graph, the LATEST run is selected in the grid. If the user is not careful to observe this, they are mislead to thinking the results they are browsing in the graph is the ones they selected in step 2.
As workaround, one can append
&num_runs=365to the url, this will take you to the correct run from step 2. But eventually you will exhaust 365 runs as well.This problem isn't only applicable to the dagrun list. Any links to the /dags/ graph are affected. We have external systems linking to individual dagruns that now get unusable after 25 runs.
What you think should happen instead
When navigating to a run via link, it should always present the run that was provided in run_id querystring.
How to reproduce
See above
Operating System
Debian GNU/Linux 11 (bullseye)
Versions of Apache Airflow Providers
No response
Deployment
Other Docker-based deployment
Deployment details
No response
Anything else
This was not a problem in Airflow 2.5, regression in 2.7, probably introduced by new embedded graph view (which otherwise is great :))
Are you willing to submit PR?
Code of Conduct