Deprecate airflow.utils.email in favor of the SMTP provider by shivaam · Pull Request #75414 · apache/airflow

shivaam · 2026-10-22T14:55:36Z

This deprecates airflow.utils.email. Its functions live in airflow.providers.smtp.utils.emailer (#74623), and the old names now forward to them with a DeprecatedImportWarning. Core now requires apache-airflow-providers-smtp>=3.2.0, the first release with the emailer, so the floor change needs the allow provider dependency bump label.

Existing imports and email_backend settings keep working. Keep email_backend at its default airflow.utils.email.send_email_smtp: Airflow treats that value as "send with SmtpNotifier". Only the Python imports moved.

What changes for code that calls these functions directly:

The SMTP Connection is looked up with BaseHook.get_connection. Tasks and Airflow's server processes still find a Connection stored only in the metadata database. A standalone script outside Airflow's processes needs it from an environment variable or a secrets backend.
Settings are read through the Task SDK configuration, so values changed at runtime with airflow.configuration.conf.set() are not seen. airflow.cfg and environment variables work as before.
Tests that patch airflow.utils.email.send_mime_email no longer intercept the call made by send_email_smtp, which now runs in the provider. Patch airflow.providers.smtp.utils.emailer.send_mime_email instead.

These are also in the significant newsfragment and the deprecation warning.

Testing

New tests: the old names resolve to the provider functions, including through from airflow.utils.email import *, the warning names the Connection lookup, and a server process finds a database-only Connection. They fail against the current module.
With SMTP 3.2.0rc1 installed, a real SMTP server (Mailpit) receives task retry and failure alerts for the default backend and for a custom backend that imports airflow.utils.email, and the messages match the original core code. The new tests also pass with SMTP 3.2.0.

What's next

Separate provider PRs so the SES and SendGrid providers no longer import airflow.utils.email.
The SMTP emailer gets its own defaults for the [smtp] settings; after that release, core deprecates [smtp].
In Airflow 4: remove airflow.utils.email.

related: #30530, #74623

Was generative AI tooling used to co-author this PR?

Yes — Claude Code (Opus 5.5)

Generated-by: Claude Code (Opus 5.5) following the guidelines

shivaam · 2026-10-22T15:23:23Z

This PR raises core's minimum apache-airflow-providers-smtp to 3.2.0, the first release with the emailer. The Build info check only allows that with the allow provider dependency bump label. Could a release manager add it once SMTP 3.2.0 is released? The PR stays in draft until then.

Also, backport-to-v3-4-test was added automatically because an earlier revision touched scripts/. This PR no longer does and is meant for main only, so that label can be removed.

Drafted-by: Claude Code (Opus 5.5); reviewed by @shivaam before posting

shivaam · 2026-10-23T16:19:53Z

@hussein-awala could you help me review this PR?

eladkal · 2026-10-24T17:16:30Z

Later: share the MIME helpers between SmtpHook and the emailer, and move the [smtp] settings into the SMTP provider.

Shouldn't we deprecate the core email settings as part of this PR?

shivaam · 2026-10-24T20:26:41Z

Shouldn't we deprecate the core email settings as part of this PR?

Not yet. The SMTP provider's emailer reads the [smtp] settings and relies on core for their default values. A deprecated setting loses its default, so every email would fail with section/key [smtp/smtp_host] not found in config. I tested this.

Plan:

Next SMTP release: the emailer gets its own defaults.
Then core deprecates [smtp].

The [email] settings stay in core, because the Task SDK uses them for task alerts with any email backend.

Drafted-by: Claude Code (Opus 5.5); reviewed by @shivaam before posting

amoghrajesh

A deprecated public module plus a provider floor going 2.3.1 to 3.2.0 is upgrade-relevant, so significant newsfragment pls.

shivaam · 2026-10-26T13:16:15Z

A deprecated public module plus a provider floor going 2.3.1 to 3.2.0 is upgrade-relevant, so significant newsfragment pls.

Added airflow-core/newsfragments/75414.significant.rst. It covers the deprecation, the new SMTP floor, keeping email_backend as is, and what changes for direct callers.

Drafted-by: Claude Code (Opus 5.5); reviewed by @shivaam before posting

amoghrajesh

Thanks for working on a rather difficult task @shivaam! LGTM

Two providers still import the deprecated module: amazon/aws/hooks/ses.py and sendgrid/utils/emailer.py.

Once this merges, anyone using SesHook gets a import time warning. And all they can do is upgrade to correct airflow for this. Both providers will now need the SMTP provider installed to import at all, and they should declare in dependencies.

The right next step here would be to clean this up

shivaam · 2026-10-27T03:13:29Z

Two providers still import the deprecated module: amazon/aws/hooks/ses.py and sendgrid/utils/emailer.py.

Thanks! The SES and SendGrid PRs are ready: each provider gets its own copy of the helper, so neither imports airflow.utils.email or needs the SMTP provider. I'll open them as soon as I'm under the limit of 5 open PRs. We can discuss more in the PR as I think it might be better to decouple them

Drafted-by: Claude Code (Opus 5.5); reviewed by @shivaam before posting

The email utilities now live in airflow.providers.smtp.utils.emailer. airflow.utils.email keeps its public names as deprecated aliases of the provider functions, so existing imports and email_backend settings keep working with a DeprecatedImportWarning. The default [email] email_backend value is unchanged. The Task SDK treats it as "no custom backend" and sends alerts with SmtpNotifier, as before. A regression test covers email sent from a server process, such as a listener in the API server, with the SMTP login stored only in the metadata database. airflow-core now requires apache-airflow-providers-smtp>=3.2.0, the first release that contains the emailer module.

"from airflow.utils.email import *" imported the email functions before they moved to the SMTP provider. Listing them in __all__ keeps that import working, with the same deprecation warning as named imports. The email configuration guide now says where the functions live, that the default email_backend value is unchanged, and that the moved functions do not see settings changed at runtime with airflow.configuration.conf.set(). The coverage script entry for airflow/utils/email.py is left unchanged, so this change stays within airflow-core.

The SMTP server and connection setup is documented by the SMTP provider, so the core page now only explains email_backend and the [email] options shared by all backends, and links to the provider docs.

Add a significant newsfragment, keep email_backend at its default in the docs note, and name the Connection lookup and configuration source in the deprecation warning.

Outside task and server processes the Connection lookup only searches environment variables and secrets backends, so the email is sent without a login. The newsfragment describes this; the test now covers it.

github-actions · 2026-10-27T04:28:16Z

Backport successfully created: v3-4-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-4-test

…er (#75414) * Deprecate airflow.utils.email in favor of the SMTP provider The email utilities now live in airflow.providers.smtp.utils.emailer. airflow.utils.email keeps its public names as deprecated aliases of the provider functions, so existing imports and email_backend settings keep working with a DeprecatedImportWarning. The default [email] email_backend value is unchanged. The Task SDK treats it as "no custom backend" and sends alerts with SmtpNotifier, as before. A regression test covers email sent from a server process, such as a listener in the API server, with the SMTP login stored only in the metadata database. airflow-core now requires apache-airflow-providers-smtp>=3.2.0, the first release that contains the emailer module. * Keep star imports of airflow.utils.email working and document the move "from airflow.utils.email import *" imported the email functions before they moved to the SMTP provider. Listing them in __all__ keeps that import working, with the same deprecation warning as named imports. The email configuration guide now says where the functions live, that the default email_backend value is unchanged, and that the moved functions do not see settings changed at runtime with airflow.configuration.conf.set(). The coverage script entry for airflow/utils/email.py is left unchanged, so this change stays within airflow-core. * Shorten the core email page and point to the SMTP provider docs The SMTP server and connection setup is documented by the SMTP provider, so the core page now only explains email_backend and the [email] options shared by all backends, and links to the provider docs. * Explain what changes for callers of the deprecated email functions Add a significant newsfragment, keep email_backend at its default in the docs note, and name the Connection lookup and configuration source in the deprecation warning. * Test that a plain script does not see a database-only SMTP Connection Outside task and server processes the Connection lookup only searches environment variables and secrets backends, so the email is sent without a login. The newsfragment describes this; the test now covers it. (cherry picked from commit ad6ec79) Co-authored-by: Shivam Rastogi <6463385+shivaam@users.noreply.github.com>

airflow-core now requires providers-smtp>=3.2.0 (email deprecation, #75414/#75709). The emailer source landed via #75704 but the in-repo provider version was still 3.1.1, so the CI image build failed resolving the floor. Mirrors the smtp slice of the 3.2.0 provider release (#75209): version + provider.yaml + changelog + doc refs.

…er (#75414) (#75709) * Deprecate airflow.utils.email in favor of the SMTP provider The email utilities now live in airflow.providers.smtp.utils.emailer. airflow.utils.email keeps its public names as deprecated aliases of the provider functions, so existing imports and email_backend settings keep working with a DeprecatedImportWarning. The default [email] email_backend value is unchanged. The Task SDK treats it as "no custom backend" and sends alerts with SmtpNotifier, as before. A regression test covers email sent from a server process, such as a listener in the API server, with the SMTP login stored only in the metadata database. airflow-core now requires apache-airflow-providers-smtp>=3.2.0, the first release that contains the emailer module. * Keep star imports of airflow.utils.email working and document the move "from airflow.utils.email import *" imported the email functions before they moved to the SMTP provider. Listing them in __all__ keeps that import working, with the same deprecation warning as named imports. The email configuration guide now says where the functions live, that the default email_backend value is unchanged, and that the moved functions do not see settings changed at runtime with airflow.configuration.conf.set(). The coverage script entry for airflow/utils/email.py is left unchanged, so this change stays within airflow-core. * Shorten the core email page and point to the SMTP provider docs The SMTP server and connection setup is documented by the SMTP provider, so the core page now only explains email_backend and the [email] options shared by all backends, and links to the provider docs. * Explain what changes for callers of the deprecated email functions Add a significant newsfragment, keep email_backend at its default in the docs note, and name the Connection lookup and configuration source in the deprecation warning. * Test that a plain script does not see a database-only SMTP Connection Outside task and server processes the Connection lookup only searches environment variables and secrets backends, so the email is sent without a login. The newsfragment describes this; the test now covers it. (cherry picked from commit ad6ec79) Co-authored-by: Shivam Rastogi <6463385+shivaam@users.noreply.github.com>

The backport of #75414 (#75709) raised the SMTP provider version in its pyproject.toml without refreshing uv.lock. The constraints jobs run `uv sync`, which then tries to rewrite the lock file. The lock is bind-mounted as a single file in the container, so the rename fails with "Device or resource busy" and every "Generate constraints" job on v3-4-test fails.

boring-cyborg Bot added area:dev-tools backport-to-v3-4-test Backport to v3-4-test labels Oct 22, 2026

shivaam marked this pull request as ready for review October 22, 2026 23:52

eladkal added this to the Airflow 3.4.0 milestone Oct 24, 2026

eladkal added the type:misc/internal Changelog: Misc changes that should appear in change log label Oct 24, 2026

eladkal reviewed Oct 24, 2026

View reviewed changes

Comment thread airflow-core/docs/howto/email-config.rst

eladkal approved these changes Oct 26, 2026

View reviewed changes

amoghrajesh reviewed Oct 26, 2026

View reviewed changes

Comment thread airflow-core/docs/howto/email-config.rst

Comment thread airflow-core/src/airflow/utils/email.py

Comment thread airflow-core/src/airflow/utils/email.py Outdated

github-actions Bot added the need workflow approval label Oct 26, 2026

vatsrahul1001 modified the milestones: Airflow 3.4.0, Airflow 3.4.1 Oct 26, 2026

vatsrahul1001 force-pushed the codex/email-core-deprecation branch from fabb999 to ac7c012 Compare October 26, 2026 18:11

vatsrahul1001 requested a review from amoghrajesh October 26, 2026 18:40

amoghrajesh approved these changes Oct 27, 2026

View reviewed changes

Comment thread airflow-core/tests/unit/utils/test_email.py Outdated

Lee-W approved these changes Oct 27, 2026

View reviewed changes

Lee-W added the allow provider dependency bump label Oct 27, 2026

shivaam added 5 commits October 27, 2026 09:10

Shorten the core email page and point to the SMTP provider docs

af796b4

The SMTP server and connection setup is documented by the SMTP provider, so the core page now only explains email_backend and the [email] options shared by all backends, and links to the provider docs.

Explain what changes for callers of the deprecated email functions

c628c17

Add a significant newsfragment, keep email_backend at its default in the docs note, and name the Connection lookup and configuration source in the deprecation warning.

Test that a plain script does not see a database-only SMTP Connection

f327acd

Outside task and server processes the Connection lookup only searches environment variables and secrets backends, so the email is sent without a login. The newsfragment describes this; the test now covers it.

vatsrahul1001 force-pushed the codex/email-core-deprecation branch from af5c809 to f327acd Compare October 27, 2026 03:40

vatsrahul1001 enabled auto-merge (squash) October 27, 2026 03:40

vatsrahul1001 disabled auto-merge October 27, 2026 04:25

vatsrahul1001 merged commit ad6ec79 into apache:main Oct 27, 2026
80 of 89 checks passed

shivaam mentioned this pull request Oct 27, 2026

Stop importing airflow.utils.email in the Amazon SES hook #75711

Merged

1 task done

vatsrahul1001 mentioned this pull request Oct 27, 2026

Add release notes for 3.4.0rc1 #75748

Closed

1 task done

shahar1 mentioned this pull request Oct 27, 2026

[v3-4-test] Update uv.lock for the SMTP provider 3.2.0 version #75757

Merged

1 task

shivaam mentioned this pull request Oct 27, 2026

Stop importing airflow.utils.email in the SendGrid emailer #75771

Merged

1 task done

vatsrahul1001 mentioned this pull request Oct 28, 2026

Status of testing of Apache Airflow 3.4.0rc1 #75865

Open

Navigation Menu

Repository navigation

Deprecate airflow.utils.email in favor of the SMTP provider - #75414

Deprecate airflow.utils.email in favor of the SMTP provider#75414
vatsrahul1001 merged 5 commits into
apache:mainfrom
shivaam:codex/email-core-deprecation

shivaam commented Oct 22, 2026 •

edited

Loading

Uh oh!

shivaam commented Oct 22, 2026 •

edited

Loading

Uh oh!

shivaam commented Oct 23, 2026

Uh oh!

eladkal commented Oct 24, 2026

Uh oh!

Uh oh!

shivaam commented Oct 24, 2026

Uh oh!

amoghrajesh left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

shivaam commented Oct 26, 2026

Uh oh!

amoghrajesh left a comment

Uh oh!

Uh oh!

shivaam commented Oct 27, 2026 •

edited

Loading

Uh oh!

Uh oh!

github-actions Bot commented Oct 27, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

shivaam commented Oct 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Was generative AI tooling used to co-author this PR?

Uh oh!

shivaam commented Oct 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

shivaam commented Oct 23, 2026

Uh oh!

eladkal commented Oct 24, 2026

Uh oh!

Uh oh!

shivaam commented Oct 24, 2026

Uh oh!

amoghrajesh left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

shivaam commented Oct 26, 2026

Uh oh!

amoghrajesh left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

shivaam commented Oct 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Oct 27, 2026

Backport successfully created: v3-4-test

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

shivaam commented Oct 22, 2026 •

edited

Loading

shivaam commented Oct 22, 2026 •

edited

Loading

shivaam commented Oct 27, 2026 •

edited

Loading