Repository navigation
Status of testing Providers that were prepared on February 21, 2025 #46973
Description
Activity
I can see a strange artifact in Pypi docs string for various providers on this release which was not present in previous releases:
Screenshots
I can see a strange artifact in Pypi docs string for various providers on this release which was not present in previous releases:
Let me take a look at that -> likely some template artifact. Probably not a blocker for the release, but would be nice to fix
I can see a strange artifact in Pypi docs string for various providers on this release which was not present in previous releases:
Likely fix here: #46975 - it will not help with the currently released packages (the "final" canidates are already prepared as well - and those are the ones we vote on in SVN - but should likely help in the future.
Reacted by Mikhail Dengin, Amogh Desai and GPKI updated the issue with a small hack that includes all providers
tested #45571 , it works 👍
Reacted by Elad KalifTested the following issues:
The ones with testing impact:
- Fixing log typos in Local & Celery Executors #46866
- AIP-72: Swap KubernetesExecutor to use taskSDK for execution #46860
- Using quotes for file path in find command #46795
- Moving EmptyOperator to standard provider #46231 (imported and ran a dag with emptyoperator from new path)
- Removing feature: send context in venv operators (using
use_airflow_context) #46306 (the feature doesnt work, which is expected)
No testing impact but rather to fix CI
- Adding type ignore to sql hook #46163
- Limit
google-cloud-aiplatformto fix issues in CI #46242 - Fixing return type for azure hook 'run_query' #45470
- Disallow pymongo: 4.11.1 to fix CI tests #46636
- Adding uuid6 as a dependency for openlineage #46653
All of them work as expected.
#44960 works fine 👍
Was about to check the Asana provider change, but I can't see it in the list now.
#46553 example dag works fine. 👍
#45660 works!
Reacted by Elad Kalif#46643 it works fine.
Reacted by Elad Kalif7 remaining items
All following PR are working as expected:
-
Introduce notion of dialects in DbApiHook (Introduce notion of dialects in DbApiHook #41327): @dabla
-
Fixed generated SQL MSSQL dialect when all columns are also primary keys (Fixed generated SQL MSSQL dialect when all columns are also primary keys #46380): @dabla
-
Fix escaping of special characters or reserved words as column names in dialects of common sql provider (Fix escaping of special characters or reserved words as column names in dialects of common sql provider #45640): @dabla
-
Added ADR document describing why the notion of dialects was introduced in the common sql provider (Added ADR document describing why the notion of dialects was introduced in the common sql provider #45456): @dabla
-
Allow check_response to be defined in extra_options of HTTP connection (Allow check_response to be defined in extra_options of HTTP connection #45451): @dabla
-
Added support for certificate authentication with MSGraphAsyncOperator (Added support for certificate authentication with MSGraphAsyncOperator #45935): @dabla
-
fix: KiotaRequestAdapterHook make sure proxy config parameter is parsed correctly, even if it's a string or json (fix:
KiotaRequestAdapterHookmake sure proxy config parameter is parsed correctly, even if it's a string or json #46145): @dabla -
Fixed retry of PowerBIDatasetRefreshOperator when dataset refresh wasn't directly available (Fixed retry of PowerBIDatasetRefreshOperator when dataset refresh wasn't directly available #45513): @Ohashiro
-
Fixes compat issue HTTPX proxy configuration in KiotaRequestAdapterHook and fixed retry in MSGraphSensor (Fixes compat issue HTTPX proxy configuration in KiotaRequestAdapterHook and fixed retry in MSGraphSensor #45746): @dabla
-
Fixed assignment of proxies in KiotaRequestAdapter for HTTPX 0.28.x compliancy (Fixed assignment of proxies in KiotaRequestAdapter for HTTPX 0.28.x compliancy #45741): @dabla
-
Updated instantiation of httpx AsyncClient in KiotaRequestAdapterHook to be httpx 0.28.x compliant (Updated instantiation of httpx AsyncClient in KiotaRequestAdapterHook to be httpx 0.28.x compliant #45464): @dabla
-
Refactor result_processor and event_handler signatures in MSGraphAsyncOperator (Refactor result_processor and event_handler signatures in MSGraphAsyncOperator #46637): @dabla
-
Also allow passing buffer instead of path for retrieve_file and store_file methods in SFTPHook (Also allow passing buffer instead of path for retrieve_file and store_file methods in SFTPHook #44247): @dabla
-
Make sure the SSHClient is also closed when using connection in context manager from SFTPHook (Make sure the SSHClient is also closed when using connection in context manager from SFTPHook #46716): @dabla
This PR is also working but needs an additional patch to make the field_multi_value_leniency parameter working, will open a PR for that:
- Implemented cursor for ElasticsearchSQLHook so it can be used through SQLExecuteQueryOperator (Implemented cursor for ElasticsearchSQLHook so it can be used through SQLExecuteQueryOperator #46439): @dabla (works but need extra patch for field_multi_value_leniency)
-
Render a pod spec using the pod_template_file override, if passed to the executor works well in
apache-airflow-providers-cncf-kubernetes==10.3.0rc1👍@potiuk / @eladkal The Asana Provider is missing from the 1st message, it was in the original message.
Feel free to test and report it regardless @simonprydden . This relase was a bit difficult (and manual) because of a bug found in the releae process, but there is nothing preventing you to test the rc - I think the only "real" change was yours #46553 and you can install the provider from https://pypi.org/project/apache-airflow-providers-asana/2.9.0rc1/
@potiuk / @eladkal The Asana Provider is missing from the 1st message, it was in the original message.
Feel free to test and report it regardless @simonprydden . This relase was a bit difficult (and manual) because of a bug found in the releae process, but there is nothing preventing you to test the rc - I think the only "real" change was yours #46553 and you can install the provider from https://pypi.org/project/apache-airflow-providers-asana/2.9.0rc1/
@potiuk I've tested it and works fine, just wanted to make sure it's not missed, as it's not listed on #46973 (comment) now
@potiuk I've tested it and works fine, just wanted to make sure it's not missed, as it's not listed on #46973 (comment) now
Cool. No. Not forgotten - we just manually generated/updated the issue content and like it's been removed from the description accidentally.
Reacted by Simon PryddenChanges
- Remove deprecated functionality from google provider #46235
- Implement AlloyDB create/update/delete instance operators #45241
- Introduce gcp translation (V3) glossaries providers #45085
- Deprecate gcp AutoML module #44875
for apache-airflow-providers-google 14.0.0rc1 work as expected.
hi!
All my changes work as expected. Thank you!My changes
- Add backward compatibility for old Airflow versions for CloudComposerDAGRunSensor #46011
- Create operators for working with Topics for GCP Apache Kafka #46865
- Create operators for working with Clusters for GCP Apache Kafka #45795
for
apache-airflow-providers-google==14.0.0rc1work as expected.Tested #46579, looks good!
- Fix 3.12 failing providers tests of amazon emr hook #46612 not sure how to test it regarding the package release
- Fix wrong f-string to satisfy ruff static check #46813 changed
- Remove overlooked
pod_generator_deprecated.py#46587 no file in package - Mini-fix: use right operator name in doc string of
@task.kubernetes#46873 can see in package -- it's applied
Body
I have a kind request for all the contributors to the latest provider packages release.
Could you please help us to test the RC versions of the providers?
The guidelines on how to test providers can be found in
Verify providers by contributors
Let us know in the comments, whether the issue is addressed.
These are providers that require testing as there were some substantial changes introduced:
Provider amazon: 9.4.0rc1
Linked issues:
Linked issues:
Linked issues:
EmrContainerHook.create_emr_on_eks_cluster(#46562): @vincbeckLinked issues:
create_emr_on_eks_clustermethod to try when "cluster is not reachable as its connection is currently being updated" (#46497): @vincbeckLinked issues:
Linked issues:
Linked issues:
fetch_access_token_for_clusterin EKS hook (#45469): @vincbeckLinked issues:
example_dms_serverless(#46684): @vincbeckexample_sqsto not uselogical_date(#46696): @vincbeckLinked issues:
usermandatory for all methods in the auth manager interface (#45986): @vincbeckrun_job_kwargsas templated field inGlueJobOperator(#45973): @eladkalOutletEventAccessor(#45762): @kaxiltyping_compatthat can be imported directly (#45589): @kaxilLinked issues:
Linked issues:
Provider apache.beam: 6.0.2rc1
Provider apache.hive: 9.0.2rc1
Provider apache.livy: 4.2.0rc1
endpoint_prefixtoLivyHook(#45811): @gpathak128Provider celery: 3.10.2rc1
Linked issues:
Provider cncf.kubernetes: 10.3.0rc1
pod_template_fileoverride, if passed to the executor (#46374): @brouberolLinked issues:
Linked issues:
Linked issues:
Linked issues:
@task.kubernetes(#46873): @insomnespod_generator_deprecated.py(#46587): @insomnes@task.kubernetespod naming (#46535): @insomnesLinked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Provider cohere: 1.4.2rc1
Linked issues:
Provider common.compat: 1.5.0rc1
Linked issues:
Linked issues:
Provider common.sql: 1.23.0rc1
Linked issues:
Provider databricks: 7.2.0rc1
DatabricksNotebookOperator(#44960): @hardeybiseyLinked issues:
Provider dbt.cloud: 4.2.0rc1
typing_compatthat can be imported directly (#45589): @kaxilProvider docker: 4.2.0rc1
labels(#46643): @phi-fridayProvider elasticsearch: 6.2.0rc1
Linked issues:
Linked issues:
Provider exasol: 4.7.2rc1
Linked issues:
Provider google: 14.0.0rc1
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
google-cloud-aiplatformto fix issues in CI (#46242): @amoghrajeshLinked issues:
Provider http: 5.2.0rc1
Provider microsoft.azure: 12.2.0rc1
KiotaRequestAdapterHookmake sure proxy config parameter is parsed correctly, even if it's a string or json (#46145): @dablaLinked issues:
AzureServiceBusTopicCreateOperatortoAdminClientHook(#45367): @perry2of5Provider microsoft.mssql: 4.2.0rc1
Provider microsoft.winrm: 3.9.0rc1
Provider mongo: 5.0.1rc1
Linked issues:
Provider mysql: 6.1.0rc1
Provider openlineage: 2.1.0rc1
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
typing_compatthat can be imported directly (#45589): @kaxilProvider opensearch: 1.6.1rc1
Provider pagerduty: 4.0.1rc1
create_eventwithsend_eventinPagerdutyNotifier(#45630): @bshea5Linked issues:
Provider papermill: 3.9.1rc1
Linked issues:
Provider postgres: 6.1.0rc1
Provider presto: 5.8.1rc1
Provider redis: 4.0.1rc1
Provider salesforce: 5.10.0rc1
Provider sftp: 5.1.0rc1
Provider smtp: 2.0.0rc1
Provider snowflake: 6.1.0rc1
Linked issues:
Provider sqlite: 4.0.0rc1
Provider ssh: 4.0.0rc1
Provider standard: 0.1.0rc1
BaseEventTriggeras base class for triggers used with event driven scheduling (#46391): @vincbeckLinked issues:
Linked issues:
Linked issues:
use_airflow_context) (#46306): @amoghrajeshLinked issues:
Provider trino: 6.0.1rc1
Provider weaviate: 3.0.1rc1
All users involved in the PRs:
@lendle @darkag @o-nikolas @perry2of5 @amoghrajesh @SuccessMoses @joellabes @MaksYermak @kanagaraj-dhanapal-89 @vatsrahul1001 @plutaniano @kacpermuda @potiuk @chenkovsky @phi-friday @olegkachur-e @kunal
Committer