Add new functions to DruidDbApiHook

[yael-is]

Description

The current DruidDbApiHook in Airflow does not support :

• 'get_pandas_df': Executes a SQL and returns a pandas DataFrame. The function will run a query on druid and will return pandas DataFrame containing the query results.

• 'get_conn': Need to get the value of 'ssl_verify_cert' from outside.

Use case/motivation

No response

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[Taragolis]

'get_conn': A function that will establish a connection to the Druid broker with connect to 'Druid' with pydruid.db/connect library. The func retrieves the connection details from Airflow's connection database and uses them to connect to the Druid broker, and will return a connection to the Druid broker.

🤔 🤔 🤔

airflow/airflow/providers/apache/druid/hooks/druid.py

Lines 175 to 188 in 89df63b

	def get_conn(self) -> connect:
	"""Establish a connection to druid broker."""
	conn = self.get_connection(getattr(self, self.conn_name_attr))
	druid_broker_conn = connect(
	host=conn.host,
	port=conn.port,
	path=conn.extra_dejson.get("endpoint", "/druid/v2/sql"),
	scheme=conn.extra_dejson.get("schema", "http"),
	user=conn.login,
	password=conn.password,
	context=self.context,
	)
	self.log.info("Get the connection to druid broker on %s using user %s", conn.host, conn.login)
	return druid_broker_conn

[yael-is]

'get_conn': A function that will establish a connection to the Druid broker with connect to 'Druid' with pydruid.db/connect library. The func retrieves the connection details from Airflow's connection database and uses them to connect to the Druid broker, and will return a connection to the Druid broker.

🤔 🤔 🤔

airflow/airflow/providers/apache/druid/hooks/druid.py

Lines 175 to 188 in 89df63b

	def get_conn(self) -> connect:
	"""Establish a connection to druid broker."""
	conn = self.get_connection(getattr(self, self.conn_name_attr))
	druid_broker_conn = connect(
	host=conn.host,
	port=conn.port,
	path=conn.extra_dejson.get("endpoint", "/druid/v2/sql"),
	scheme=conn.extra_dejson.get("schema", "http"),
	user=conn.login,
	password=conn.password,
	context=self.context,
	)
	self.log.info("Get the connection to druid broker on %s using user %s", conn.host, conn.login)
	return druid_broker_conn

Thank you, but i actually need to send "ssl_verify_cert" to the "connect", and there is no such thing in this current func:

` conn = self.get_connection(getattr(self, self.conn_name_attr))

    druid_broker_conn = connect(
        host=conn.host,  # Druid broker host
        port=conn.port,  # Druid broker port
        path=conn.extra_dejson.get("endpoint", "/druid/v2/sql"),  # Druid endpoint path
        scheme=conn.schema,  # Use HTTPS for secure communication
        user=conn.login,  # Druid broker username
        password=conn.password,  # Druid broker password
        ssl_verify_cert=False,  # Disable SSL certificate verification
    )
    self.log.info(
        "Get the connection to Druid broker on %s using user %s",
        conn.host,
        conn.login,
    )
    return druid_broker_conn`

[Taragolis]

Thank you, but i actually need to send "ssl_verify_cert" to the "connect", and there is no such thing in this current func:

Then better clarify in the description that you want to extend Druid connection and add ability to disable SSL certificate validation.

Or better create separate issue for that, as I could see your request include two different features:

1. Extend Druid connection and make DruidDbApiHook use this option
2. Implements get_pandas_df

[yael-is]

Thank you, but i actually need to send "ssl_verify_cert" to the "connect", and there is no such thing in this current func:

Then better clarify in the description that you want to extend Druid connection and add ability to disable SSL certificate validation.

Or better create separate issue for that, as I could see your request include two different features:

1. Extend Druid connection and make DruidDbApiHook use this option
2. Implements get_pandas_df

Thank you, You're right, I changed the description.

[blcksrx]

I'm gonna work on it. assign it to me please

[potiuk]

feel free