Apache Airflow version
2.9.1
If "Other Airflow 2 version" selected, which one?
No response
What happened?
in the method _upload_file_temp from DataprocSubmitPySparkJobOperator it calls the function _generate_temp_filename which do this return: f"{time:%Y%m%d%H%M%S}_{str(uuid.uuid4())[:8]}_{ntpath.basename(filename)}" and the first format is bad, it gives the following error:
TypeError: unsupported format string passed to module.__format__
What you think should happen instead?
it should be like this:
date = time.strftime("%Y%m%d%H%M%S") return f"{date}_{str(uuid.uuid4())[:8]}_{ntpath.basename(filename)}"
to get the date string 20240508114811
How to reproduce
you only have to create an instance of DataprocSubmitPySparkJobOperator and call the method _upload_file_temp to reproduce the error
Operating System
Debian GNU/Linux 11 (bullseye)
Versions of Apache Airflow Providers
airflow-providers-google
from 10.7.0 to 10.17.0, in all this versions fail
Deployment
Official Apache Airflow Helm Chart
Deployment details
No response
Anything else?
No response
Are you willing to submit PR?
Code of Conduct
Apache Airflow version
2.9.1
If "Other Airflow 2 version" selected, which one?
No response
What happened?
in the method _upload_file_temp from DataprocSubmitPySparkJobOperator it calls the function _generate_temp_filename which do this return:
f"{time:%Y%m%d%H%M%S}_{str(uuid.uuid4())[:8]}_{ntpath.basename(filename)}"and the first format is bad, it gives the following error:TypeError: unsupported format string passed to module.__format__What you think should happen instead?
it should be like this:
date = time.strftime("%Y%m%d%H%M%S") return f"{date}_{str(uuid.uuid4())[:8]}_{ntpath.basename(filename)}"to get the date string 20240508114811
How to reproduce
you only have to create an instance of DataprocSubmitPySparkJobOperator and call the method _upload_file_temp to reproduce the error
Operating System
Debian GNU/Linux 11 (bullseye)
Versions of Apache Airflow Providers
airflow-providers-google
from 10.7.0 to 10.17.0, in all this versions fail
Deployment
Official Apache Airflow Helm Chart
Deployment details
No response
Anything else?
No response
Are you willing to submit PR?
Code of Conduct