Skip to content

Prepare providers release 2026-09-22 - #73506

Merged
shahar1 merged 7 commits into
apache:mainfrom
shahar1:prepare-providers-release-2026-09-22
Sep 24, 2026
Merged

shahar1 merged 7 commits into
apache:mainfrom
shahar1:prepare-providers-release-2026-09-22

Conversation

@shahar1

@shahar1 shahar1 commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Regular provider release wave. 78 providers are released, 2 (microsoft.mssql, apache.kafka) are marked doc-only, and the rest had only build/tooling churn and stay out.

Important

This PR needs the allow provider dependency bump label. Two inter-provider >= floors were resolved from # use next version pins (fab → common-compat>=1.20.0, common.ai → standard>=1.20.0), which trips the check_provider_dependency_bumps selective check.

Major bumps

Provider Version Breaking change
edge3 4.3.2 → 5.0.0 EdgeExecutor now counts queued workloads against [core] parallelism (#72048)
git 0.5.0 → 1.0.0 Token auth moves to a GIT_CONFIG_COUNT credential helper; needs git >= 2.31 (#64105)
microsoft.psrp 3.2.7 → 4.0.0 PsrpOperator no longer defaults task_id to cmdlet (#73036)
openai 1.8.2 → 2.0.0 create_embeddings returns every embedding for list inputs (#71989)
common.ai 0.9.0 → 0.10.0 ErrorClassification reinstated by #73501, but PydanticAI conn types renamed, SQLToolset now rejects allowed_tables=None/[], and execute_complete overrides on LLMOperator/LLMBranchOperator need a decision keyword (#72853, #73381, #73452, #73368)

common.ai stays pre-1.0 deliberately: its breaking changes are real, but the API is still moving, so the 0.x signal is worth more to users than strict semver. It is prepared in a separate commit so it can be dropped without rewriting the release commit.

Notable non-major entries

  • amazon 9.36.0 → 9.37.0. Build deferred AWS hooks from the operator's own settings #72171 was initially classified breaking; the AWS team reviewed it and considers it a bug fix, so amazon is a minor. The .. warning:: about the triggerer no longer inheriting its host's boto3 defaults still ships — the runtime change is real, it just isn't a major. A second .. warning:: covers Fix prefix handling in Amazon transfer operators #73269, merged after the cut: the FTP/SFTP ↔ S3 transfer operators now match *_filenames as a leading prefix, as documented, instead of a substring, so some previously selected files are no longer picked up.
  • duckdb 0.1.0 — initial release. It has never been published, so this is a first release rather than a bump. A .latest-doc-only-change.txt left over from April 2025 was keeping it invisible to the release tooling; removing it brings the provider in. Its DuckDB version-compatibility guidance moved into docs/index.rst above the generated section, so template regeneration stops eating it. Landed as a separate commit. (check-provider-yaml-valid had been failing for duckdb with No module named 'airflow.providers.duckdb' — that was a stale local CI image predating Add DuckDB provider #72845, not a repo defect; it passes after a rebuild.)
  • Template every connection id accepted by provider operators #73286 ("Template every connection id accepted by provider operators") is why the wave is large: it makes every operator's connection id a template field across 61 providers, which is a user-facing feature in each of them.

Validation

  • dev/check_changelog_entries.py --fix — 77 changelogs, 0 unresolved problems.
  • prek run --hook-stage pre-commit — no failures.
  • breeze release-management prepare-provider-documentation --reapply-templates-only is idempotent.
  • breeze release-management update-providers-next-version run; no # use next version pins remain.

Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 5)

Generated-by: Claude Code (Opus 5) following the guidelines

🤖 Generated with Claude Code

@shahar1 shahar1 added the skip common compat check Skips common compat provider modification check label Sep 21, 2026
@shahar1
shahar1 force-pushed the prepare-providers-release-2026-09-22 branch from 01325b9 to ef55e73 Compare September 21, 2026 21:06
@shahar1

shahar1 commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor Author

common.ai - on hold until further notice

Edit: will be released as 0.10.0

@subhramit

subhramit commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

@shahar1 To be included (as per release thread) - #71976 and #72081

Comment thread providers/apache/livy/docs/changelog.rst Outdated

@potiuk potiuk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving — mechanically this is one of the cleanest waves I've checked. Everything
below is context for the release thread rather than a gate on this PR.

What I checked and found clean

  • Version consistency across __init__.py / pyproject.toml / provider.yaml /
    changelog.rst / README.rst / index.rst for all 33 released providers — no drift.
  • # use next version resolution: exactly the two in the description
    (fab → common-compat>=1.20.0, common.ai → standard>=1.19.1), none left over.
  • All five .latest-doc-only-change.txt markers point at commits that really are
    doc-only for that provider's tree (weaviate's slice of #72111 is a code comment).
  • amazon's excluded-changes list is entirely test-only.
  • Semver spot-checks: #72155 as Feature in common.ai and Bug Fix in standard
    is right (the standard slice removes two asserts and fixes a None-responder
    crash). cloudant 4.3.6 for dropping cloudant_fake is fine — internal mypy stubs.
  • All six majors carry a migration note; common.ai covers all three of its
    breaking changes.

Two coordination items for the release thread

Neither is a defect in this diff — both are calls for the release manager.

common.ai. Your own comment on this PR says "common.ai - on hold until further
notice"
, yet the diff bumps it 0.9.0 → 0.10.0 with three breaking changes and resolves
common.ai → standard>=1.19.1. If the hold still stands, that provider needs pulling
before the RC is cut; if it's stale, worth saying so on the thread since the standard
floor resolution rides on it.

#71976 and #72081. subhramit asked for both per the release thread; both are still
open and unmerged on main, so neither is here. #72081 (AthenaSparkOperator) targets
amazon, which is already at 10.0.0 — cheap to fold in while the major is open. #71976
targets influxdb, which isn't in this wave at all.

CI

Special tests / Pendulum2 test: core / Core...Serialization fails on
airflow-core/tests/unit/jobs/test_triggerer_job.py::test_trigger_log[trigger1-1-0]:

E   assert '0 triggers currently running' in "... Triggerer's async thread was
    blocked for 300.00 seconds, exceeding the config..."

This PR touches no airflow-core/ file — loaded-runner flake, not diff-caused. It's
still a failing required check, so that one job needs a rerun before the merge button
lights up.

Smaller observations

Three inline, none blocking — one content-loss item on providers/duckdb/README.rst
(the regeneration is correct, but the hand-written "DuckDB version compatibility"
section needs somewhere to live) and two .. note:: → .. warning:: nits on the
breaking-change blocks this PR authors.


This review was drafted by an AI-assisted tool and
confirmed by an Apache Airflow maintainer. The maintainer
approving this PR has read the findings and signed off. If
something feels off, please reply on the PR and a maintainer
will follow up.

More on how Apache Airflow handles maintainer review:
contributing-docs/05_pull_requests.rst.


Drafted-by: Claude Opus 5; reviewed by @potiuk before posting

Comment thread providers/duckdb/README.rst
Comment thread providers/openai/docs/changelog.rst Outdated
Comment thread providers/microsoft/psrp/docs/changelog.rst Outdated
@shahar1
shahar1 force-pushed the prepare-providers-release-2026-09-22 branch from ef55e73 to bc76c0f Compare September 22, 2026 15:27
Comment thread providers/duckdb/README.rst
Comment thread providers/amazon/docs/changelog.rst Outdated
@shahar1
shahar1 force-pushed the prepare-providers-release-2026-09-22 branch from bc76c0f to 9612792 Compare September 22, 2026 21:36
@shahar1
shahar1 requested a review from subhramit September 23, 2026 06:00
@shahar1
shahar1 force-pushed the prepare-providers-release-2026-09-22 branch from 417ee5c to bb759c1 Compare September 23, 2026 06:06

@subhramit subhramit left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks Shahar!

@shahar1
shahar1 force-pushed the prepare-providers-release-2026-09-22 branch 2 times, most recently from eac95c2 to 5bf9e03 Compare September 23, 2026 07:44

@potiuk potiuk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approving after the rebase. Scope grew from 33 to 78 providers, mostly from #73286. I re-ran the mechanical checks over the new head:

  • Versions are consistent across __init__.py / pyproject.toml / README.rst / index.rst / changelog for every bumped provider.
  • Every provider touched by #73286, #72081, #71976 and #73501 is in the wave, and #73286 is filed as a Feature in all 61 changelogs.
  • Providers left out have only flit-4 build churn since their last release, or are not-ready. The mssql doc-only marker covers a docs-only commit.
  • Excluded-change lists hold no runtime changes (weaviate's #72111 slice is a code comment, as before).
  • The earlier .. warning:: and duckdb docs points are addressed, and the wording pass makes the entries read much better.

One thing to fix before merge (inline): the common.ai → standard>=1.19.1 floor predates the rebase. standard now ships as 1.20.0.

CI: the three red jobs (constraints, sdist, Compat 3.3.2) are all a PyPI 503 on flit's trove-classifier download (list_classifiers). This is not caused by the diff. #73612 on main removes that download, so rebasing avoids a repeat. The Compat 3.3.2 run never reached its tests, so it's worth seeing it green once before merging.


Drafted-by: Claude Code (Opus 5); reviewed by @potiuk before posting

Comment thread providers/common/ai/pyproject.toml Outdated
@shahar1
shahar1 force-pushed the prepare-providers-release-2026-09-22 branch 2 times, most recently from 26af3e9 to f7f4a2e Compare September 24, 2026 06:04
76 providers are released and 1 is marked doc-only.

The four major bumps (edge3, git, microsoft.psrp and openai) each remove or
change a released public API, and every one carries a warning in its version
section explaining the migration. amazon is a minor instead: the AWS team
reviewed apache#72171 and considers the deferred-hook change a bug fix, so the
warning about the triggerer no longer inheriting its host's boto3 defaults
ships under 9.37.0 rather than a major.

The wave is large because apache#73286 made every operator's connection id a
template field across 61 providers, which is a user-facing feature in each
of them.

common.ai is prepared in a follow-up commit rather than this one, so that
the rest of the wave can ship if its release is deferred.
Kept separate from the rest of the wave so this provider can be dropped
without rewriting the release commit.

The bump is 0.10.0 rather than 1.0.0 although the release contains breaking
changes: the provider is still pre-1.0 and its API is expected to keep
moving, so the 0.x signal is worth more to users than strict semver here.
duckdb has never been published: there is no release tag for it, so 0.1.0
is an initial release rather than a bump. A doc-only marker left over from
April 2025 kept the provider invisible to the release tooling; removing it
is what brings duckdb into the wave.

The DuckDB version-compatibility guidance moves into docs/index.rst above
the generated section, where template regeneration cannot eat it again. It
was lost when the hand-written README was regenerated earlier in this wave.
An audit of all 254 entries against each PR's provider-scoped diff found 26
whose text did not describe what the PR did to that provider. Seven asserted
behaviour the slice does not contain, eight carried another distribution's
headline, and eleven were too vague or too internal to act on.

Only the wording changes. Every entry keeps its section and no provider's
version moves: the corrected classifications match the ones already applied,
and each affected provider's bump is anchored by an entry the audit did not
touch.
Add the cncf.kubernetes doc-only entry for apache#73624 and move the
apache.kafka doc-only marker past apache#73588, which only moved message
queue docs out of a class docstring.
@shahar1
shahar1 force-pushed the prepare-providers-release-2026-09-22 branch from f7f4a2e to 051ad93 Compare September 24, 2026 07:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants