Repository navigation
Links to dagruns beyond last 25 runs don't work (take 3) #39642
Description
Activity
There's some additional bizarre behavior that might be related. When clicking on the logs icon from Browse -> Task Instances, it only shows a single DAG run (the only one that was scheduled -- the other ~40 were triggered by button mashing), and it says "No XCom" where the logs should be:
Hi @hterik, we faced same issue too. Problem is that
task_instance.log_urlgenerates log url to page that has this error. So, I resolved issue by generating custom execution log url with below function and use in operator callbacks for our Slack notifications. Basically, function generate url to see full execution log, not routes traffic to grid view 😅:from datetime import datetime from urllib.parse import quote def __generate_execution_log_url(context: dict): dag_id = context["dag_run"].dag_id task_instance = context["ti"] task_id = task_instance.task_id encoded_execution_date = quote(task_instance.execution_date.strftime("%Y-%m-%dT%H:%M:%S.%f%z")) return f"https://airflow.example.com/log?dag_id={dag_id}&task_id={task_id}&execution_date={encoded_execution_date}"
I have built the main branch of the project (at commit c5c50cc to be precise) and still see the problem. Should we reopen this ticket or should I create a new one? Or is there something I'm missing here?
Reacted by HTErikThis problem partially persists in Airflow 3.1.8. Linking to a dagrun older than the 10 runs shown in the grid view will not show the grid for the selection, instead keep it at the most recent 10. The Gantt view is not possible to use for older dags any more, that used to be possible before.
The ambiguity and selection of incorrect task instance seem to be fixed now though. When the dag is older, no task instance is selected at all in the grid view. This is a better fallback as it avoids misrepresentation.
Apache Airflow version
2.9.1
If "Other Airflow 2 version" selected, which one?
No response
What happened?
This is a continuation of #34723 that was fixed in #34887, later fixed again in #37018, reported as problematic for other reasons in #38939, then later fixed yet again in #38941.
Unfortunatelly the issue still remains.
In short, the problem is that the grid view by default only shows 25 dagruns, with
base_datestarting at current time. Ifdag_run_idquery parameter points to a dag run that is older than today - 25 runs, it will not be pre-selected. This makes it impossible to permalink to specific runs, without knowing in advance thebase_datefor that run.1.Start 50 instances of dag mydag and wait for them to complete
2. Visit Browse -> Task Instances list:
airflow/taskinstance/list/?_flt_3_dag_id=mydag3. Scroll to the bottom and click on the oldest run, in the Task Id column, this link will look something like
/dags/mydag/grid?task_id=mytask&dag_run_id=123456&tab=graph4. Dag grid view opens, but no dagrun is selected. 🙁
What you think should happen instead?
Dag grid should open and have dag_run_id=123456 selected.
It appears like there are two different endpoints, one called
/graph, that does handle redirecting the base_date to include older runs, and another called/gridthat doesn't.When browsing from the Browse -> DAG runs list instead, it uses the
/graphendpoint and there the links work, even for older runs. 👍@bbovenzi, do you know if the
/graphis the one to use for all permalink purposes and this solves these base_date issues?How to reproduce
Operating System
Debian Bookworm
Versions of Apache Airflow Providers
No response
Deployment
Other Docker-based deployment
Deployment details
No response
Anything else?
No response
Are you willing to submit PR?
Code of Conduct