Repository navigation
UI DAG counts including deleted DAGs #33698
Description
Apache Airflow version
2.7.0
What happened
On the DAGs page, the All, Active, and Paused counts include deleted DAGs. This is different from <= 2.6.1 (at least), where they were not included in the totals. Specifically this is for DAGs for which the DAG files have been removed, not DAGs that have been deleted via the UI.
What you think should happen instead
Including deleted DAGs in those counts is confusing, and this behavior should revert to the previous behavior.
How to reproduce
Create a DAG.
Wait for totals to increment.
Remove the DAG file.
The totals will not change.
Operating System
Debian GNU/Linux 11 (bullseye)
Versions of Apache Airflow Providers
apache-airflow-providers-celery==3.3.2
apache-airflow-providers-cncf-kubernetes==7.4.2
apache-airflow-providers-common-sql==1.7.0
apache-airflow-providers-datadog==3.3.1
apache-airflow-providers-elasticsearch==5.0.0
apache-airflow-providers-ftp==3.5.0
apache-airflow-providers-google==10.6.0
apache-airflow-providers-http==4.5.0
apache-airflow-providers-imap==3.3.0
apache-airflow-providers-microsoft-azure==6.2.4
apache-airflow-providers-postgres==5.6.0
apache-airflow-providers-redis==3.3.1
apache-airflow-providers-salesforce==5.4.1
apache-airflow-providers-slack==7.3.2
apache-airflow-providers-snowflake==4.4.2
apache-airflow-providers-sqlite==3.4.3Deployment
Astronomer
Deployment details
No response
Anything else
I suspect the issue is with DagModel.deactivate_deleted_dags, but I'm unable to verify.
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
Activity
eladkal commentedon Aug 24, 2023
Remove dag file does not remove the records from the DB.
the DAG entries are left. If you want to remove the history you need to remove the records
joeknize-bc commentedon Aug 24, 2023
This isn't about history, this is about how many DAGs are in my Airflow instance right now -- that's what I expect to see on the DAGs page.
Also, there is no way that I'm aware of to delete a DAG via the UI once the files are gone. This is new behavior that the DagModel.deactivate_deleted_dags function seems designed to deal with.
boushphong commentedon Aug 26, 2023
I created a PR for this issue at: #33778
@potiuk @eladkal Could you take a look? 🙇
potiuk commentedon Aug 27, 2023
Merged and marked for 2.7.1