Repository navigation
SqlToS3Operator #11128
Description
Recently, all the Sql-related (mysql, postgres...) operators were merged into one. Applying the same, would be great to have a single SqlToS3Operator, so that you don't have to use MySqlToS3Operator, RedshiftToS3Operator, SnowflakeToS3Operator...
Activity
manesioz commentedon Sep 25, 2020
Where is this? When looking for it I still ran into separate MySQL/PostgreSQL/etc db operations/hooks.
akocukcu commentedon Nov 4, 2020
Hi @manesioz ,
I think for the sql to sql transfer there is => GenericTransfer
and for the hook there is => DbApiHook
Btw the name
GenericTransferis a little bit general 😊 MaybeGenericSqlTransferwould be more clear.Anyway, I would like to work on this issue if you think this is still needed @JavierLopezT .
JavierLopezT commentedon Nov 6, 2020
Actually, I was referring to this operator:
https://github.com/apache/airflow/blob/master/airflow/operators/sql.py
And yeah, I think it is totally worth it to have this feature, so it would be great if you start working on it.
PS: sorry @manesioz for not answering you, I didn't realize
akocukcu commentedon Nov 8, 2020
@JavierLopezT Great :) I can also give the option to select an available file format (such as parquet, json, excel ...)
As I see, right now there is only csv option for MySqlToS3Operator.
I would like to work on this. Sorry to disturb you @mik-laj .But could you assign it to me please 🐱
eladkal commentedon Nov 8, 2020
@akocukcu assigned to you
samuelkhtu commentedon Feb 13, 2021
@akocukcu just wondering how's going with the SqlToS3Operator? Let me know if anything I can help with.
akocukcu commentedon Feb 15, 2021
Hi @samuelkhtu . There were some personal obstacles. But I am working on it again.
samuelkhtu commentedon Feb 15, 2021
@akocukcu of course. Let me know if anything I can help with. Thanks.