Repository navigation
Add transfers operator S3 to SQL / SQL to SQL #23666
Description
Description
Should we add S3 to SQL to aws transfers?
Use case/motivation
- After process data from spark/glue(more), we need to publish data to sql
- Synchronize data between 2 sql databases.
Related issues
No response
Are you willing to submit a PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
Activity
eladkal commentedon May 12, 2022
SQL to SQL is GenericTransfer:
airflow/airflow/operators/generic_transfer.py
Line 25 in 7277122
You are more than welcome to work on S3ToSql...
We have dedicated operators like: S3ToMySqlOperator, S3ToSnowflakeOperator
should you be able to create a generic transfer that can be great.
thinhnd2104 commentedon May 12, 2022
In my case genneric transfers is enough because it does not have update or upsert rows.
potiuk commentedon May 17, 2022
Assigned you.
maggesssss commentedon Dec 28, 2022
@thinhnd2104 are you working on this topic?
o-nikolas commentedon Jan 10, 2023
@maggesssss Are you interested in opening a PR for this issue? We can re-assign it to you since we have not heard from @thinhnd2104 in some time.
maggesssss commentedon Jan 12, 2023
@o-nikolas yes, I'd like to pick it up.
o-nikolas commentedon Jan 12, 2023
Perfect, assigned to you :)