Skip to content

Cannot send custom statsd metrics in a plugin #69172

Open

Description

@Jeoffreybauvin

Under which category would you file this issue?

Airflow Core

Apache Airflow version

3.2.1

What happened and how to reproduce it?

A while back, I was using a plugin to send custom statsd metrics after a fail or success dag run :

from airflow.stats import Stats
metric_success = f"brs_dag_last_5_runs_states.success.{dag_id}"
metric_failed = f"brs_dag_last_5_runs_states.failed.{dag_id}"
Stats.gauge(metric_success, success_count)
Stats.gauge(metric_failed, failed_count)

And it was working fine. But, it's not working anymore : no metric is sent to my statsd_exporter.

Here is my configuration :

      AIRFLOW__METRICS__STATSD_HOST: 10.2.218.29
      AIRFLOW__METRICS__STATSD_INFLUXDB_ENABLED: 'True'
      AIRFLOW__METRICS__STATSD_ON: 'True'
      AIRFLOW__METRICS__STATSD_PORT: '9125'
      AIRFLOW__METRICS__STATSD_PREFIX: prod

Please note : I don't have any issue with the "native" statsd metrics from Airflow : my stats_exporter is collecting them.

After some debug of the statsd object inside my plugin file :

  log.info("Stats object: %r", Stats)

  for attr in ("_instance", "_logger", "factory"):
      log.info("Stats.%s = %r", attr, getattr(Stats, attr, None))
  2026-06-30T09:03:49.079501Z [info     ] Stats object: <class 'airflow.sdk._shared.observability.metrics.stats.Stats'> [airflow.utils.log.logging_mixin.LoggingMixin] loc=brs_tools.py:90
  2026-06-30T09:03:49.079666Z [info     ] Stats._instance = None         [airflow.utils.log.logging_mixin.LoggingMixin] loc=brs_tools.py:92
  2026-06-30T09:03:49.079812Z [info     ] Stats._logger = None           [airflow.utils.log.logging_mixin.LoggingMixin] loc=brs_tools.py:92
  2026-06-30T09:03:49.079946Z [info     ] Stats.factory = <class 'airflow.sdk._shared.observability.metrics.base_stats_logger.NoStatsLogger'> [airflow.utils.log.logging_mixin.LoggingMixin] loc=brs_tools.py:92
  2026-06-30T09:03:49.080074Z [info     ] Stats.gauge returned: None     [airflow.utils.log.logging_mixin.LoggingMixin] loc=brs_tools.py:96

What you think should happen instead?

The metrics inside my plugin should be sent to statsd.

Operating System

Ubuntu 24.04 + Docker

Deployment

Docker-Compose

Apache Airflow Provider(s)

No response

Versions of Apache Airflow Providers

apache-airflow-providers-amazon==9.6.1
apache-airflow-providers-apache-cassandra==3.9.4
apache-airflow-providers-apache-hdfs==4.11.5
apache-airflow-providers-apache-kafka==1.13.3
apache-airflow-providers-apache-spark==6.0.1
apache-airflow-providers-celery==3.18.0
apache-airflow-providers-cncf-kubernetes==10.16.1
apache-airflow-providers-common-compat==1.14.3
apache-airflow-providers-common-io==1.7.2
apache-airflow-providers-common-messaging==2.0.3
apache-airflow-providers-common-sql==1.35.0
apache-airflow-providers-docker==4.5.5
apache-airflow-providers-elasticsearch==6.5.3
apache-airflow-providers-fab==3.6.2
apache-airflow-providers-ftp==3.14.3
apache-airflow-providers-git==0.3.1
apache-airflow-providers-google==21.2.0
apache-airflow-providers-grpc==3.9.4
apache-airflow-providers-hashicorp==4.6.0
apache-airflow-providers-http==6.0.2
apache-airflow-providers-microsoft-azure==13.1.2
apache-airflow-providers-microsoft-mssql==4.5.2
apache-airflow-providers-mysql==6.5.2
apache-airflow-providers-odbc==4.12.2
apache-airflow-providers-openlineage==2.15.0
apache-airflow-providers-oracle==4.5.3
apache-airflow-providers-postgres==6.6.3
apache-airflow-providers-redis==4.4.4
apache-airflow-providers-sendgrid==4.2.3
apache-airflow-providers-sftp==5.7.4
apache-airflow-providers-slack==9.10.0
apache-airflow-providers-smtp==3.0.0
apache-airflow-providers-snowflake==6.12.2
apache-airflow-providers-ssh==5.0.1
apache-airflow-providers-standard==1.12.3

Official Helm Chart version

Not Applicable

Kubernetes Version

No response

Helm Chart configuration

No response

Docker Image customizations

No response

Anything else?

For the time being, I'm sending "manually" the metrics :

from functools import lru_cache

 from airflow.configuration import conf
 from statsd import StatsClient


 @lru_cache
 def get_statsd_client() -> StatsClient:
     return StatsClient(
         host=conf.get("metrics", "statsd_host"),
         port=conf.getint("metrics", "statsd_port"),
         prefix=conf.get("metrics", "statsd_prefix", fallback=None),
     )


 def statsd_gauge(metric_name: str, value: int) -> None:
     get_statsd_client().gauge(metric_name, value)

 statsd_gauge(metric_success, success_count)
 statsd_gauge(metric_failed, failed_count)

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Activity

  1. added a commit that references this issue on Jul 24, 2026
    3daa850
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions