Repository navigation
Status of testing Providers that were prepared on October 06, 2026 #74459
Description
Activity
- addedkind:metaHigh-level information important to the communityHigh-level information important to the communitytesting statusStatus of testing releasesStatus of testing releases
on Oct 8, 2026 Verified my changes in
apache-airflow-providers-amazon==9.38.0rc1for #73756 and #73881 using the PyPI wheel, with Airflow 3.3.2 / Python 3.12.14 on Linux ARM64. Updated with live AWS verification inap-northeast-2.- Keep S3 Dag bundle downloads within the configured directory #73756: All 8 checks pass against a real S3 bucket: bundle initialization and repeated refresh with directory prefixes with/without trailing slashes, sibling-prefix objects, and objects whose key equals the configured prefix. Verified downloaded contents, stale-file/directory cleanup, and preservation of excluded remote objects.
- Handle plain Step Functions execution errors #73881: All 9 checks pass using actual Standard Step Functions executions and
DescribeExecutionresponses. Plain, Unicode, and malformed-JSON error strings are returned correctly; JSON-valued errors and successful output retain their types. In the live empty-stringErrorPathcase, AWS omitted theerrorfield and the operator correctly returnedNone.
The 17 live-AWS checks all passed. Running the same checks with provider 9.37.0, using the same bucket and executions, reproduced 7 failures (4 S3, 3 Step Functions); reinstalling 9.38.0rc1 restored all 17 passes. Temporary buckets, state machines, and execution roles were deleted and their absence confirmed.
The earlier 20 local checks using Moto/synthetic HTTP responses also passed (8 failures on 9.37.0), including explicit
error: "", malformed success output, and service-error propagation. Those additional cases remain simulated.pip checkpassed, both merged commits are in the RC tag, and the relevant installed sources match the wheel and tag.Scope: the live checks make real AWS calls from the installed bundle/operator/hook. Operator execution is invoked directly, with UI extra-link persistence mocked; this does not cover scheduler/task-runner/XCom end-to-end behavior, an IAM-policy matrix, or the whole Amazon provider.
Reacted by Jarek Potiuk#73652 tested in
common.ai0.11.0rc1, works as expected.
Thanks for preparing the release, Jarek.Reacted by Jarek Potiuk#73384 tested in common-ai 0.11.0rc1, works as expected too!
Thanks for preparing the release, looking forward to the rapid development of common-ai ahead!
#74017 tested that it's included in common-ai 0.11.0rc1 and works as expected.
Thank you for preparing the release#71676 and #74191 tested on rc1, works.
- Add OpenSandbox backend for sandbox tools #71676 (
common.ai0.11.0rc1): I ran the PyPI wheel on Airflow 3.3.2 / Python 3.12 against a local OpenSandbox server. The PR's system-test Dag and its 64 unit tests pass, and command timeouts and sandbox cleanup behave as expected. One caveat, for a docs follow-up: when the server's egress sidecar runs in its defaultdnsmode, a deny-allSandboxSpec()only blocks DNS, so direct-IP connections still get out. With[egress] mode = "dns+nft"they are blocked too, and everything above still passes. The backend can't tell the two modes apart on policy read-back. - Notify downstream Dags when COPY INTO writes a Unity (Databricks) table #74191 (
databricks7.22.0rc1): on a real SQL warehouse, COPY INTO emitted the canonical Unity table asset event and triggered the downstream Dag. Wrong table, wrong workspace and failed SQL emitted no event.
Drafted-by: Claude Code (Opus 5.5) (no human review before posting)
- Add OpenSandbox backend for sandbox tools #71676 (
#71711 tested in apache-spark 6.3.3rc1, works as expected.
Verified against the RC in a clean venv: all masking forms produce the expected output (
key=value,key value, quoted values spanning tokens, uppercase keys, tab- and multi-space-separated values, and a sensitive flag with no value), and a multiline value no longer swallows the log lines that follow it.The scanner that replaced the regex also closes the complexity gap raised in review on my PR —
"secret" * 10000, which still backtracked quadratically in my version (~2.4s), now completes in 0.16ms. 50k single token: 0.66ms.Thanks for preparing the release.
Drafted-by: Claude Code; reviewed by @divyanshus2404 before posting
#73509 tested with
apache-airflow-providers-amazon==9.38.0rc1(Airflow 3.3.2, common-messaging 2.1.1), works as expected :)Have tested my changes
- Add dedicated InfluxDB 3 sensor #73522 [
influxdb: 2.13.0rc1] - Fix Elasticsearch and OpenSearch response wrapper bugs #73725 [
elasticsearch: 6.9.2rc1] and [opensearch: 1.14.0rc1]
using my agent, looks good:
Thanks!
- Add dedicated InfluxDB 3 sensor #73522 [
Tested both of the changes I am listed for. Both work as expected.
Environment: clean virtualenv on Python 3.11, installed from PyPI,
apache-airflow3.3.2. No dbt Cloud account or GCP project involved, since everything below is observable before the first API call. In each case I also checked the previous stable release to confirm the result depends on the change rather than passing either way.dbt.cloud 4.11.0rc1, Apply hook_params to deferred dbt Cloud job runs (#74199)
Walking the installed wheel, the
DbtCloudRunJobTriggerconstruction inoperators/dbt.pynow passeshook_params; on4.10.0it does not. In both releases the trigger already accepted it, kept it acrossserialize()and built its hook with**self.hook_params, so the operator's call site was the only gap and nothing else had to change.4.10.0 4.11.0rc1 hook_paramspassed at the defer siteabsent present Checked at runtime with
hook_params={"retry_limit": 7, "retry_delay": 3.5}: the operator retains them, its own hook honours them, and a trigger built with them serializes them intact.google 22.7.0rc1, Apply impersonation_chain to deferred BigQuery existence checks (#71648)
Both deferral sites in
sensors/bigquery.pynow passimpersonation_chain, one forBigQueryTableExistenceTriggerand one forBigQueryTablePartitionExistenceTrigger; on22.6.0neither does. As above, the triggers accepted and serialized the argument in both releases, so only the sensors needed the change.22.6.0 22.7.0rc1 BigQueryTableExistenceTriggerabsent present BigQueryTablePartitionExistenceTriggerabsent present Confirmed the trigger accepts
impersonation_chainand serializes it as given.Import smoke check
Importing every module of both providers: all 12 dbt Cloud modules import, and 297 of 333 google modules. The 36 that do not are all missing optional peers in this environment, other providers plus paramiko, kubernetes, apache-beam, cassandra, oracledb and similar, each failing with an explicit message naming what is absent. Nothing failed for a reason internal to either release.
Tested apache.kafka 2.1.0rc1 and it works as expected:
- Add KafkaSharedStreamProducer and KafkaSharedStreamTrigger #68625: I ran two
AssetWatchers withKafkaSharedStreamTriggeron the same topic. The two triggers share one Kafka consumer. Both assets get every message. Offsets are committed per partition only after the asset events are stored. After a restart, committed messages are not delivered again. A connection withoutenable.auto.commit=falseis refused. On Airflow 3.1.8 and 2.11.2, importing the shared-stream triggers raisesAirflowOptionalProviderFeatureException.
I found some issues for following providers:
- google 22.7.0rc1
- Add GKEPodExecOperator for existing Pods #72577:
operators/kubernetes_engine.pynow importsKubernetesPodExecOperatorat module level.KubernetesPodExecOperatoronly exists in cncf-kubernetes 10.22.0 and later (https://airflow.apache.org/docs/apache-airflow-providers-cncf-kubernetes/10.22.0/changelog.html). With an older cncf-kubernetes, importingoperators/kubernetes_engine.pyraisesAirflowOptionalProviderFeatureException. So every GKE operator fails to import, not onlyGKEPodExecOperator. Should we update the GKE module like the EKS module? In Add EKS operator for commands in existing Pods #72542, ifeks.pycannot importKubernetesPodExecOperator, it defines a fallback class with the same name. The__init__of that fallback class raises the same exception that the GKE module raises on import. The other EKS operators still import, and only creatingEksPodExecOperatorfails. I can create a follow-up PR for this.
- Add GKEPodExecOperator for existing Pods #72577:
- dbt.cloud 4.11.0rc1
- Apply hook_params to deferred dbt Cloud job runs #74199: A
DbtCloudRunJobOperatorwithdeferrable=Truethat works on 4.10.0 fails on 4.11.0rc1 whenhook_paramsholdsretry_argswith tenacity objects. Created a follow-up PR for it Fix deferred dbt Cloud tasks failing with tenacity retry_args #74489.
- Apply hook_params to deferred dbt Cloud job runs #74199: A
- Add KafkaSharedStreamProducer and KafkaSharedStreamTrigger #68625: I ran two
Correcting my earlier comment: my verification of #74199 in
dbt.cloud 4.11.0rc1was wrong.I checked that
hook_paramssurvivesserialize(), but I used{"retry_limit": 7, "retry_delay": 3.5}, two JSON serializable scalars.hook_paramsis a free form dict and the one key whose documented values are Python objects isretry_args, so my input could not exercise the case that matters. With a tenacity object inretry_argsthe task fails when it defers, and it fails after the dbt Cloud job has already started, which leaves the run orphaned.Thanks to @FrankYang0529 for catching it and for the fix in #74489.
Treating that as a release blocker for
dbt.cloud 4.11.0: shipping rc1 as is would regress any deferrableDbtCloudRunJobOperatorwhosehook_paramscarriesretry_args, which worked on 4.10.0. The google 22.7.0rc1 item I reported is unaffected and my result there stands.Checked apache-airflow-providers-apache-beam 6.3.1rc1: #72510 is included and the async hook now launches pipelines via create_subprocess_exec without a shell. Works as expected.
I have a kind request for all the contributors to the latest provider distributions release.
Could you please help us to test the RC versions of the providers?
The guidelines on how to test providers can be found in
Verify providers by contributors
Let us know in the comments, whether the issue is addressed.
These are providers that require testing as there were some substantial changes introduced:
Provider airbyte: 6.2.0rc1
Provider akeyless: 0.3.2rc1
Provider amazon: 9.38.0rc1
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Provider anthropic: 1.2.0rc1
AnthropicHook(#73967): @kaxilLinked issues:
Provider apache.beam: 6.3.1rc1
Provider apache.kafka: 2.1.0rc1
Linked issues:
Provider apache.spark: 6.3.3rc1
SparkSubmitHook: Mask_mask_cmdsecrets in linear time (#71711): @divyanshus2404Linked issues:
Provider celery: 3.25.0rc1
Linked issues:
Provider clickhousedb: 1.0.1rc3
Provider cncf.kubernetes: 10.24.0rc1
Linked issues:
Provider common.ai: 0.11.0rc1
max_retriestoAgentSkillsToolset(#74381): @kaxilLinked issues:
include_tracebackto model-backed retry policies (#74308): @kaxilLinked issues:
Linked issues:
capabilitiesto Common AIAgentOperator(#73984): @kaxilLinked issues:
Linked issues:
HookToolsetpin arguments the model must not choose (#73900): @kaxilObjectStorageToolsetfor reading files on object storage (#73899): @kaxilLinked issues:
output_typeafter human review (#73904): @kaxilLinked issues:
common.aitoolsets (#73587): @kaxilModalHookand connection type (#73418): @kaxilLinked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
code_modefromAgentOperatorin favor of theCodeModecapability (#74312): @kaxilexecute_toolthe public methodAirflowToolsetsubclasses implement (#73938): @kaxilLinked issues:
common.sql2.2.0 forcommon.ai's SQL extras (#73867): @kaxilLinked issues:
Linked issues:
common.aitoolset guides (#74379): @kaxilcommon.aisandbox docs with when to use it and where each piece runs (#74297): @kaxilusage_limits=Nonedocstring onAgentOperatorandLLMOperator(#74307): @kaxilLinked issues:
common.aitoolsets (#73902): @kaxilLinked issues:
common.aisidebar (#73837): @kaxilretry_reasonnot just for retries but even when a task fails (#73027): @amoghrajeshProvider common.compat: 1.21.0rc1
Linked issues:
retry_reasonnot just for retries but even when a task fails (#73027): @amoghrajeshProvider common.sql: 2.3.0rc1
Provider databricks: 7.22.0rc1
Linked issues:
Provider dbt.cloud: 4.11.0rc1
Linked issues:
Linked issues:
Provider duckdb: 0.2.1rc1
Provider edge3: 5.0.0rc3
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Provider elasticsearch: 6.9.2rc1
Provider fab: 3.10.1rc1
Linked issues:
Linked issues:
Linked issues:
Provider ftp: 3.16.1rc1
Provider git: 1.0.1rc1
Linked issues:
Linked issues:
Provider google: 22.7.0rc1
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Provider hashicorp: 4.8.3rc1
Provider http: 6.3.0rc1
Linked issues:
Provider ibm.db2: 0.1.0rc1 🎉 New provider
Linked issues:
Linked issues:
Provider ibm.mq: 0.1.0rc1 🎉 New provider
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Provider influxdb: 2.13.0rc1
Linked issues:
Provider keycloak: 0.11.1rc1
Provider microsoft.azure: 15.2.1rc1
Linked issues:
Linked issues:
Linked issues:
Provider modal: 0.1.0rc1 🎉 New provider
ModalHookand connection type (#73418): @kaxilLinked issues:
Provider openai: 2.1.0rc1
Provider openlineage: 2.20.3rc1
Linked issues:
Provider opensearch: 1.14.0rc1
Linked issues:
Linked issues:
Provider sftp: 7.0.0rc1
Linked issues:
Provider slack: 9.11.1rc1
Provider smtp: 3.1.1rc1
Linked issues:
Provider snowflake: 6.19.0rc1
Provider sqlite: 4.3.4rc1
Linked issues:
Provider ssh: 7.0.0rc1
Linked issues:
Provider standard: 1.20.1rc1
Linked issues:
All users involved in the PRs:
@SameerMesiah97 @bugraoz93 @abhinav-phi @atharvaajmera @ColtenOuO @X33834 @abhishekmauryaKsolves @ephraimbuddy @AshleyAHuang @dheerajturaga @haseebmalik18 @vgkowski @namanjain24-sudo @yamakazushi @shivaam @ShubhamKapoor992 @ashb @dabla @kaxil @pankajastro @shahar1 @VladaZakharova @pankajkoti @subhramit @FrankYang0529 @moomindani @Eason09053360 @SEPURI-SAI-KRISHNA @fpiped @Lee-W @sgoel2be24-cyber @r3wretrhy @amoghrajesh @robertpofuk @seanghaeli @MichalJaroslawKrzywanski-TomTom @ucaeon @JelyFishhhhhh @YAshhh29 @ferruzzi @yuseok89 @KarthikMohankumar @Andrushika @zozo123 @bingqin2 @topherinternational @MeghanaGangarapu @drewrukin @SulimanAbdulrazzaq @eladkal @Samin061 @gtxu @Amitkumar293 @kadubhumika @filipeaaoliveira @firasbouzazi @aaron-y-chen @atamagrawal @AlejandroMorgante @jeff3071 @potiuk @radhwene @olegkachur-e @divyanshus2404 @keemgdeok @ccoliu @takayoshi-makabe @o-nikolas @jingi723