Repository navigation
Status of testing Providers that were prepared on December 31, 2021 #20615
Description
Body
I have a kind request for all the contributors to the latest provider packages release.
Could you please help us to test the RC versions of the providers?
Let us know in the comment, whether the issue is addressed.
Those are providers that require testing as there were some substantial changes introduced:
Provider amazon: 2.6.0rc1
- Add aws_conn_id to DynamoDBToS3Operator (#20363): @pingzh
- Add two operators in AWS Providers: RedshiftResumeClusterOperator and RedshiftPauseClusterOperator (#19665): @dbarrundiag
- Added function in AWSAthenaHook to get s3 output query results file URI (#20124): @subkanthi
- Add sensor for AWS Batch (#19850) (#19885): @yeshbash
- Add state details to EMR container failure reason (#19579): @Swalloow
- Add support to replace S3 file on MySqlToS3Operator (#20506): @mariotaddeucci
- Fix backwards compatibility issue in AWS provider's _get_credentials (#20463): @potiuk
- Fix deprecation messages after splitting redshift modules (#20366): @dstandish
- ECSOperator: fix KeyError on missing exitCode (#20264): @ssatia
- Bug fix in AWS glue operator when specifying the WorkerType & NumberOfWorkers (#19787): @Ritika-Singhal
- Organize Sagemaker classes in Amazon provider (#20370): @bhavaniravi
- 20139 - move emr_container hook (#20375): @bhavaniravi
- Standardize AWS Athena naming (#20305): @ferruzzi
- Standardize AWS EKS naming (#20354): @ferruzzi
- Standardize AWS Glue naming (#20372): @ferruzzi
- Standardize Amazon SES naming (#20367): @ferruzzi
- Standardize AWS CloudFormation naming (#20357): @ferruzzi
- Standardize AWS Lambda naming (#20365): @ferruzzi
- Standardize AWS Kinesis/Firehose naming (#20362): @ferruzzi
- Standardize Amazon SNS naming (#20368): @ferruzzi
- Split redshift cluster and redshift sql (#20276): @dstandish
- 20139 - organize aws emr operators (#20160): @bhavaniravi
- Standardize AWS DataSync naming (#20328): @ferruzzi
- Deprecate passing execution_date to XCom methods (#19825): @ashb
- #20139 - Move Dms operators and Sensors (#20156): @bhavaniravi
- Organise S3 Classes in Amazon Provider (#20167): @KennyRich
- 20139 - organize aws step_function (#20158): @bhavaniravi
- Organize EC2 classes in Amazon provider (#20157): @eladkal
- Move to watchtower 2.0.1 - Fixes Watchtower crash when logging empty line (#19907): @o-nikolas
- Fix mypy aws example dags (#20497): @subkanthi
- Delete pods by default in KubernetesPodOperator (#20575): @dstandish
Provider apache.druid: 2.2.0rc1
Provider apache.spark: 2.0.3rc1
Provider cncf.kubernetes: 3.0.0rc1
- Simplify
KubernetesPodOperator(#19572): @dstandish - Delete pods by default in KubernetesPodOperator (#20575): @dstandish
- Move pod_mutation_hook call from PodManager to KubernetesPodOperator (#20596): @dstandish
- Rename
PodLaunchertoPodManager(#20576): @dstandish - Add config and context params to KubernetesHook (#19695): @dstandish
- Implement dry_run for KubernetesPodOperator (#20573): @dstandish
- Clarify docstring for
build_pod_request_obj(#20574): @dstandish - Fix Volume/VolumeMount KPO DeprecationWarning (#19726): @dimon222
Provider databricks: 2.2.0rc1
- Add
wait_for_terminationoption for Databricks Operators (#20536): @eskarimov - Update connection object to cached_property in DatabricksHook (#20526): @josh-fell
- Remove
hostas an instance attr inDatabricksHook(#20540): @josh-fell - Databricks: fix verification of Azure Managed Identity (#20550): @alexott
Provider docker: 2.4.0rc1
Provider facebook: 2.2.0rc1
Provider google: 6.3.0rc1
- Add optional location to bigquery data transfer service (#15088) (#20221): @bradleybonitatibus
- Add Google Cloud Tasks how-to documentation (#20145): @kazanzhy
- Added example DAG for MSSQL to Google Cloud Storage (GCS) (#19873): @Bowrna
- Support regional GKE cluster (#18966): @mrkm4ntr
- Delete pods by default in KubernetesPodOperator (#20575): @dstandish
- Fixes for templates for PubSubCreateSubscriptionOperator (#20237): @kolban-google
- Fix missing get_backup method for Dataproc Metastore (#20326): @potiuk
- Fixed the minor typo (#19924): @mai-nakagawa
- Fix passing the gzip compression parameter on sftp_to_gcs (#20553): @qgallet
- switch to follow_redirects on httpx.get call in CloudSQL provider (#20239): @philipherrmann
- avoid deprecation warnings in BigQuery transfer operators (#20502): @eladkal
- Change download_video parameter to resourceName (#20528): @mishc9
- Fix various BigQuery to MSSQL/MySQL transfer issues (#20001): @jon-fearer
- Fix setting of project ID in provide_authorized_gcloud (#20428): @jobegrabber
- Move source_objects datatype check out of GCSToBigQueryOperator.init (#20347): @josh-fell
- Organise S3 Classes in Amazon Provider (#20167): @KennyRich
- Providers facebook hook multiple account (#19377): @bugraoz93
- Remove deprecated method call (blob.download_as_string) (#20091): @kazanzhy
- Remove deprecated template_fields and add update tests (#19977) (#19991): @bradleybonitatibus
Provider http: 2.0.2rc1
Provider imap: 2.1.0rc1
Provider jenkins: 2.0.4rc1
Provider microsoft.azure: 3.5.0rc1
- Azure: New sftp to wasb operator (#18877): @wolvery
- Removes InputRequired validation with azure extra (#20084): @potiuk
- Add operator link to monitor Azure Data Factory pipeline runs (#20207): @josh-fell
Provider mongo: 2.3.0rc1
- Fix: pin pymongo < 4.0.0 (#20511): @ckljohn
- 19223 - add mongo_db param to function doc string (#19280): @bhavaniravi
Provider neo4j: 2.1.0rc1
Provider opsgenie: 3.0.0rc1
- Add
OpsgenieCloseAlertOperator(#20488): @eladkal - Organize Opsgenie provider classes (#20454): @eladkal
- rewrite opsgenie alert hook with official python sdk, related issue #18641 (#20263): @pierrejeambrun
- Rename
OpsgenieAlertOperatortoOpsgenieCreateAlertOperator(#20514): @eladkal
Provider oracle: 2.1.0rc1
- Add method 'callproc' on Oracle hook (#20072): @malthe
- The 'cx_Oracle' driver does support autocommit (#20085): @malthe
Provider papermill: 2.2.0rc1
Provider sftp: 2.4.0rc1
Provider snowflake: 2.4.0rc1
- Support insecure mode in SnowflakeHook (#20106): @mik-laj
- Remove unused code in SnowflakeHook (#20107): @mik-laj
- Improvements for
SnowflakeHook.get_sqlalchemy_engine(#20509): @harishkrao - Exclude snowflake-sqlalchemy v1.2.5 (#20245): @mik-laj
- Limit Snowflake connector to< 2.7.2 (#20395): @potiuk
Provider tableau: 2.1.3rc1
Provider yandex: 2.2.0rc1
- YandexCloud provider: Support new Yandex SDK features: log_group_id, user-agent, maven packages (#20103): @Piatachock
Committer
- I acknowledge that I am a maintainer/committer of the Apache Airflow project.
Activity
PApostol commentedon Jan 2, 2022
#19978 addresses the issue and works as expected.
saveriogzz commentedon Jan 2, 2022
#20164 all good! ✅
mariotaddeucci commentedon Jan 2, 2022
#20506 all good o/
eskarimov commentedon Jan 3, 2022
#20536 tested, works as expected ✅
bhavaniravi commentedon Jan 3, 2022
#19280 is a docstring update, no code changes
alexott commentedon Jan 3, 2022
#20550 is verified, works as intended
pierrejeambrun commentedon Jan 3, 2022
#20263 working 👍
piotrkmita commentedon Jan 3, 2022
#19984 tested. It's working as expected.
harishkrao commentedon Jan 3, 2022
#20509 tested, working as expected.
raphaelauv commentedon Jan 3, 2022
Simplify KubernetesPodOperator (#19572)I tried different scenario it work correctly
jon-fearer commentedon Jan 3, 2022
#20001 Looks good
kazanzhy commentedon Jan 3, 2022
#20145 is devoted to the documentation
#20091 tests also changed as well
Looks good
dstandish commentedon Jan 3, 2022
Thank you @raphaelauv
I tested too... "works on my machine" ;)
mrkm4ntr commentedon Jan 4, 2022
#18966 works as expected.
subkanthi commentedon Jan 4, 2022
checked and tested #20043
dstandish commentedon Jan 4, 2022
i checked off all the PRs related to the KubernetesPodOperator. installed provider and run it through test dags.
potiuk commentedon Jan 4, 2022
Wooho. Getting quite good test coverage :) ! one more day to go !
qgallet commentedon Jan 4, 2022
#20553 works as expected !
#19924 and #20237 are doc fixes.
I believe #20575 can be marked as addressed on amazon and google provider since it's okay on kubernetes ;-)
ericpp commentedon Jan 4, 2022
#20433 is working great for me now!
bugraoz93 commentedon Jan 4, 2022
#19377 works as expected for both cases!
dimon222 commentedon Jan 4, 2022
#19726 briefly tested, seems to work as intended and not raising any problems.
potiuk commentedon Jan 5, 2022
Last chance to make some quick testing :)
josh-fell commentedon Jan 6, 2022
#20207, #20526, #20540, and #20347 look good!
Bowrna commentedon Jan 6, 2022
#19873 verified the example DAG
potiuk commentedon Jan 6, 2022
Thanks everyone for helping to thest that wave. It's been released !
potiuk commentedon Jan 6, 2022
Thanks everyone for helping to thest that wave. It's been released !
ssatia commentedon Jan 14, 2022
#20264 works as expected. We had the unexpected occur again and it was handled correctly this time around 🙌