add missing conn_id to string representation of ObjectStoragePath

[Lee-W]

As stated in apache-airflow-providers-common-io - Object Storage XCom Backend, we'll need to set xcom_objectstorage_path to something like s3://conn_id@mybucket/key to use customized xcom backend. As the string representation of ObjectStoragePath does not contain conn_id, this serialization will lost this information which cause this comparison to fail.

---

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

[pankajastro]

lgtm

[TJaniF]

Hello @eladkal 🙂

Very sorry for the late poke about this but is there a chance to include a 1.3.2 release of the common io provider with this bugfix in the current wave of releases (or the next one, since the May 12th one already two votes)?
That would be awesome, as soon as this bugfix is in, I can get a tutorial about using the Object Storage XCom backend in the Astronomer docs merged 🙂.

cc: @pankajkoti

[Lee-W]

Hello @eladkal 🙂

Very sorry for the late poke about this but is there a chance to include a 1.3.2 release of the common io provider with this bugfix in the current wave of releases (or the next one, since the May 12th one already two votes)? That would be awesome, as soon as this bugfix is in, I can get a tutorial about using the Object Storage XCom backend in the Astronomer docs merged 🙂.

cc: @pankajkoti

If I'm not mistaken, we'll need to wait for the next airflow release (not provider release) to include this fix.

[eladkal]

Hello @eladkal 🙂

Very sorry for the late poke about this but is there a chance to include a 1.3.2 release of the common io provider with this bugfix in the current wave of releases (or the next one, since the May 12th one already two votes)? That would be awesome, as soon as this bugfix is in, I can get a tutorial about using the Object Storage XCom backend in the Astronomer docs merged 🙂.

cc: @pankajkoti

This is Airflow core not providers.
It depends when @ephraimbuddy will cut Airflow 2.9.2

[TJaniF]

This is Airflow core not providers. It depends when @ephraimbuddy will cut Airflow 2.9.2

Oh! Apologies 🤦 . Thank you!

[DLT1412]

Just a security concern, is it too risky if we add conn info to xcom value return to UI? IMO xcom values should not have any sensitive value relate to internal process can be misuse/leak when user reach that, and user still need to access xcom value to debug/check etc so we not remove can read on XComs permission out of user role.

[Lee-W]

As long as there's no credential, I think we're good. But it really depends on what you think sensitive values are 🤔 By users, do you mean DAG author or just DAG user? I'm not sure we should grant DAG user XCom permission. For the DAG author, it again depends on the team's policy.

[DLT1412]

@Lee-W with user I mean DAG author. Credential not expose to user in this case but it's a system configuration then I think by default should not expose to anyone except Admin and Op role, with honor with flag webserver.expose_config.
Again, credential not expose but dag author can use this connection just by param pass when init any cloud provider hook/operator(by mistake or intended) and I think we should reduce risk here.

[simi]

Hello @Lee-W, this seems to broke ability to use ObjectStoragePath directly with DuckDB and similar as explained at https://airflow.apache.org/docs/apache-airflow/stable/tutorial/objectstorage.html#analyzing-the-data-with-duckdb since including the conn_id makes invalid bucket name. Is it still needed since there are serialize/deserialize methods implemented for XCom?

[simi]

I'll open PR to fix. I can confirm conn_id is serialized properly outside the URI.

[Lee-W]

yep, it's possible the implementation has changed since 2 to 3. Feel free to send a PR 🙂 Thanks