Skip to content

Downgrade DagBundlesManager 'DAG bundles loaded' log from INFO to DEBUG - #69960

Open
nagasrisai wants to merge 5 commits into
apache:mainfrom
nagasrisai:fix/dag-bundle-manager-log-noise-to-debug
Open

nagasrisai wants to merge 5 commits into
apache:mainfrom
nagasrisai:fix/dag-bundle-manager-log-noise-to-debug

Conversation

@nagasrisai

@nagasrisai nagasrisai commented Jul 16, 2026 •

Copy link
Copy Markdown
Contributor

Closes #69934

Summary

Every time the DagBundlesManager is initialised (which happens at task startup for each task run) it emits an INFO-level log line listing every configured bundle:

INFO - DAG bundles loaded: dags-folder, bundle-a, bundle-b, ...

For deployments with many bundles this generates tens or hundreds of redundant lines in every task log. The message is useful for debugging configuration but has no operational value during normal task execution.

This commit downgrades the log call from self.log.info to self.log.debug. The message is still available when debug logging is enabled.

Changes

  • airflow-core/src/airflow/dag_processing/bundles/manager.py: log.info → log.debug for the bundle-loaded message.

Important

🛠️ Maintainer triage note for @nagasrisai · by @potiuk · 2026-07-28 16:21 UTC

Helpful heads-up from the maintainers — please address before this PR can be reviewed:

  • ❌ Pre-commit / static checks. See docs.

The ball is in your court — you've been assigned to this PR. Fix the above, then mark it Ready for review.

See the Pull Request quality criteria for how to fix each item. There is no rush.

Note: your branch is 2835 commits behind main — please rebase and push again to get up-to-date CI results.

Automated triage — may be imperfect; a maintainer takes the next look. We use this two-stage triage process so maintainers' limited time goes to the conversation with you.

nagasrisai added a commit to nagasrisai/airflow that referenced this pull request Jul 16, 2026
@potiuk

potiuk commented Jul 20, 2026

Copy link
Copy Markdown
Member

@nagasrisai This PR has a few issues that need to be addressed before it can be reviewed — please see our Pull Request quality criteria.

Issues found:

  • ❌ Pre-commit / static checks: Failing: CI image checks / Static checks. Run prek run --from-ref main --stage pre-commit locally to reproduce and fix.

Note: Your branch is 2616 commits behind main. Some check failures may be caused by changes in the base branch rather than by your PR. Please rebase your branch and push again to get up-to-date CI results.

What to do next:

  • Fix each issue listed above.
  • Make sure static checks pass locally (prek run --from-ref main --stage pre-commit).
  • Mark the PR as "Ready for review" when you're done.

There is no rush — take your time and work at your own pace. We appreciate your contribution and are happy to wait for updates. If you have questions, feel free to ask on the Airflow Slack.


Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.

@stephen-bracken

Copy link
Copy Markdown
Contributor

I don't think this is the right approach, we should try to work out why the dag bundle manager is producing duplicate logs instead of suppressing them altogether.

@nagasrisai

Copy link
Copy Markdown
Contributor Author

Hey @stephen-bracken, that's a fair point — lowering the log level is a surface-level fix rather than addressing the root cause. I was seeing the 'DAG bundles loaded' message repeat on every scheduler heartbeat in my setup and went for the quick win, but you're right that we should understand why it's firing repeatedly before just hiding it.

Do you have any pointers on where to look? My suspicion is that the bundle refresh check isn't short-circuiting when nothing has changed, but I haven't tracked down where the repeated call originates yet. Happy to dig into it properly if you can point me in the right direction.

@stephen-bracken

Copy link
Copy Markdown
Contributor

@nagasrisai sorry I don't, its just something I noticed in my logs and raised it as an issue.

…o DEBUG

The 'DAG bundles loaded' message was emitted at INFO level on every call
to load_bundles_from_config(), which gets called on each scheduler heartbeat.
Downgrade to DEBUG so it doesn't flood the task/worker logs during normal
operation.
@nagasrisai
nagasrisai force-pushed the fix/dag-bundle-manager-log-noise-to-debug branch from d1fc5d9 to 3609eaa Compare August 5, 2026 03:40
@nagasrisai nagasrisai changed the title fix: downgrade DagBundlesManager 'DAG bundles loaded' log from INFO to DEBUG Downgrade DagBundlesManager 'DAG bundles loaded' log from INFO to DEBUG Aug 8, 2026
@potiuk potiuk added the ready for maintainer review Set after triaging when all criteria pass. label Aug 13, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:dag-processor ready for maintainer review Set after triaging when all criteria pass.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Parsing dag bundles creates lots of log noise on task startup with GitDagBundle

3 participants