Apache Airflow version
3.0.2
If "Other Airflow 2 version" selected, which one?
No response
What happened?
After upgrading an Airflow instance from 2.10.5 to 3.0.2, we noticed that errors generated by our tasks weren't sent to Sentry, even when the AIRFLOW__SENTRY__SENTRY_ON and AIRFLOW__SENTRY__SENTRY_DSN are set.
The same Sentry configuration worked on 2.10.5.
What you think should happen instead?
Task errors should be sent to Sentry when AIRFLOW__SENTRY__SENTRY_ON is activated and AIRFLOW__SENTRY__SENTRY_DSN is set correctly.
How to reproduce
- Start an Airflow 3.0.2 project using the docker-compose.yml file provided in the documentation. Add
apache-airflow[sentry] in the requirements with _PIP_ADDITIONAL_REQUIREMENTS and add the AIRFLOW__SENTRY__SENTRY_ON: "true" environment variable to the docker-compose.yml.
- Set up a Sentry server and create a project to get a DSN.
- Add the Sentry project's DSN to the
AIRFLOW__SENTRY__SENTRY_DSN environment variable of the containers in docker-compose.yml
- Create a DAG with a task only raising an exception
- Check Sentry issues. No issue is created when the task fails.
Operating System
Fedora Linux 42
Versions of Apache Airflow Providers
package_name | description | version
==========================================+==============================================================================================+========
apache-airflow-providers-amazon | Amazon integration (including Amazon Web Services (AWS) https://aws.amazon.com/) | 9.8.0
apache-airflow-providers-celery | Celery https://docs.celeryq.dev/en/stable/ | 3.11.0
apache-airflow-providers-cncf-kubernetes | Kubernetes https://kubernetes.io/ | 10.5.0
apache-airflow-providers-common-compat | Common Compatibility Provider - providing compatibility code for previous Airflow versions | 1.7.0
apache-airflow-providers-common-io | Common IO Provider | 1.6.0
apache-airflow-providers-common-messaging | Common Messaging Provider | 1.0.3
apache-airflow-providers-common-sql | Common SQL Provider https://en.wikipedia.org/wiki/SQL | 1.27.1
apache-airflow-providers-docker | Docker https://www.docker.com/ | 4.4.0
apache-airflow-providers-elasticsearch | Elasticsearch https://www.elastic.co/elasticsearch | 6.3.0
apache-airflow-providers-fab | Flask App Builder https://flask-appbuilder.readthedocs.io/ | 2.2.1
apache-airflow-providers-ftp | File Transfer Protocol (FTP) https://tools.ietf.org/html/rfc114 | 3.13.0
apache-airflow-providers-git | Distributed version control system (GIT) https://git-scm.com/ | 0.0.2
apache-airflow-providers-google | Google services including: | 15.1.0
| |
| - Google Ads https://ads.google.com/ |
| - Google Cloud (GCP) https://cloud.google.com/ |
| - Google Firebase https://firebase.google.com/ |
| - Google LevelDB https://github.com/google/leveldb/ |
| - Google Marketing Platform https://marketingplatform.google.com/ |
| - Google Workspace https://workspace.google.com/ (formerly Google Suite) |
apache-airflow-providers-grpc | gRPC https://grpc.io/ | 3.8.0
apache-airflow-providers-hashicorp | Hashicorp including Hashicorp Vault https://www.vaultproject.io/ | 4.2.0
apache-airflow-providers-http | Hypertext Transfer Protocol (HTTP) https://www.w3.org/Protocols/ | 5.3.0
apache-airflow-providers-microsoft-azure | Microsoft Azure https://azure.microsoft.com/ | 12.4.0
apache-airflow-providers-mysql | MySQL https://www.mysql.com/ | 6.3.0
apache-airflow-providers-odbc | ODBC https://github.com/mkleehammer/pyodbc/wiki | 4.10.0
apache-airflow-providers-openlineage | OpenLineage https://openlineage.io/ | 2.3.0
apache-airflow-providers-postgres | PostgreSQL https://www.postgresql.org/ | 6.2.0
apache-airflow-providers-redis | Redis https://redis.io/ | 4.1.0
apache-airflow-providers-sendgrid | Sendgrid https://sendgrid.com/ | 4.1.0
apache-airflow-providers-sftp | SSH File Transfer Protocol (SFTP) https://tools.ietf.org/wg/secsh/draft-ietf-secsh-filexfer/ | 5.3.0
apache-airflow-providers-slack | Slack https://slack.com/ services integration including: | 9.1.0
| |
| - Slack API https://api.slack.com/ |
| - Slack Incoming Webhook https://api.slack.com/messaging/webhooks |
apache-airflow-providers-smtp | Simple Mail Transfer Protocol (SMTP) https://tools.ietf.org/html/rfc5321 | 2.1.0
apache-airflow-providers-snowflake | Snowflake https://www.snowflake.com/ | 6.3.1
apache-airflow-providers-ssh | Secure Shell (SSH) https://tools.ietf.org/html/rfc4251 | 4.1.0
apache-airflow-providers-standard | Airflow Standard Provider | 1.2.0
Deployment
Docker-Compose
Deployment details
No response
Anything else?
I also tried to use the AIRFLOW__SENTRY__DSN environment variable to set the DSN, but it didn't work.
I checked in the code of Airflow 3.0.2, and it seems the only place where airflow.sentry.Sentry is used is in the _execute_in_fork fonction in https://github.com/apache/airflow/blob/main/providers/celery/src/airflow/providers/celery/executors/celery_executor_utils.py#L211 , but this function doesn't seem to be called in Airflow 3.0.2?
Are you willing to submit PR?
Code of Conduct
Apache Airflow version
3.0.2
If "Other Airflow 2 version" selected, which one?
No response
What happened?
After upgrading an Airflow instance from 2.10.5 to 3.0.2, we noticed that errors generated by our tasks weren't sent to Sentry, even when the
AIRFLOW__SENTRY__SENTRY_ONandAIRFLOW__SENTRY__SENTRY_DSNare set.The same Sentry configuration worked on 2.10.5.
What you think should happen instead?
Task errors should be sent to Sentry when
AIRFLOW__SENTRY__SENTRY_ONis activated andAIRFLOW__SENTRY__SENTRY_DSNis set correctly.How to reproduce
apache-airflow[sentry]in the requirements with_PIP_ADDITIONAL_REQUIREMENTSand add theAIRFLOW__SENTRY__SENTRY_ON: "true"environment variable to the docker-compose.yml.AIRFLOW__SENTRY__SENTRY_DSNenvironment variable of the containers in docker-compose.ymlOperating System
Fedora Linux 42
Versions of Apache Airflow Providers
Deployment
Docker-Compose
Deployment details
No response
Anything else?
I also tried to use the
AIRFLOW__SENTRY__DSNenvironment variable to set the DSN, but it didn't work.I checked in the code of Airflow 3.0.2, and it seems the only place where
airflow.sentry.Sentryis used is in the_execute_in_forkfonction in https://github.com/apache/airflow/blob/main/providers/celery/src/airflow/providers/celery/executors/celery_executor_utils.py#L211 , but this function doesn't seem to be called in Airflow 3.0.2?Are you willing to submit PR?
Code of Conduct