Skip to content

Return files destination uris in all ToGCS operators #11323

Description

@FHoffmannCode

Description

Currently some storage operators return files destination uris list for example GoogleSheetsToGCSOperator. It would be a good idea to make all of such operators to return this list.

Use case / motivation

  1. Having list of destination uris for files transferred with operator is useful for debugging and troubleshooting
  2. Inconsistent operator behaviour might be confusing for users.

Edit (by @shahar1 on Jan. 31, 2026)
For operators that currently return a single entity, add the flag unwrap_single which controls whether the returned value is either list[obj] or obj (usually obj is str but could also be a dict). Until we apply consistency for all operators (unwrap_single=False), which will be a breaking change - set for now the default to return the single entity (unwrap_single=True).

Tracking (added by @shahar1 on Jan. 24, 2026)

All operators should return list[str] of destination URIs (gs://bucket/object).

To Do (Generated by AI, please double check if current situation is as described)

Activity

  1. boring-cyborg commented on Oct 7, 2020

    @boring-cyborg

    Thanks for opening your first issue here! Be sure to follow the issue template!

  2. eladkal commented on Jan 17, 2022

    @eladkal
    Contributor

    This issue spawn from #10991 (comment)
    It probably can be useful but it's also a breaking change and so far I didn't see users complaining about it.
    Could you please clarify the TODO part on this issue?
    What operators need adjustments? Is it localized to GCP or also other providers?

  3. uranusjr commented on Jan 27, 2022

    @uranusjr
    Member

    Yeah I don’t see how this could break user code in practice. Should be doable.

  4. potiuk commented on Feb 6, 2022

    @potiuk
    Member

    Yeah. Nice feature to add. Would you like to implement it @FHoffmannCode ? Should I assign it to you ?

  5. eladkal commented on Feb 6, 2022

    @eladkal
    Contributor

    Yeah I don’t see how this could break user code in practice.

    The suggestion was to change the returned value to make operators consistent. If changed then the value pushed to xcom is not as users are expecting.

  6. uranusjr commented on Feb 6, 2022

    @uranusjr
    Member

    It depends on what are correctly being returned; I’m assuming most (if not all) are currently returning None, which won’t have backward compatibility issues (except if someone is depending on an XCom is not pushed, but why would anyone do that). Are there instances of an operator currently pushing something else?

  7. potiuk commented on Feb 6, 2022

    @potiuk
    Member

    Yeah. Same here. I think it's no harm to return "more".

  8. github-actions commented on Mar 9, 2022

    @github-actions
    Contributor

    This issue has been automatically marked as stale because it has been open for 30 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.

  9. added
    staleStale PRs per the .github/workflows/stale.yml policy file
    on Mar 9, 2022
  10. 48 remaining items

  11. potiuk commented on Mar 2, 2026

    @potiuk
    Member

    @yuseok89 We are unassigning you from this issue as part of our updated assignment policy.

    This is not meant to discourage your contribution — quite the opposite! You are still very welcome to work on this issue and submit a PR for it. Simply comment that you are working on it and open a PR when ready.

    We found that formal assignments were not working well, as they often prevented others from contributing when the assignee was not actively working on the issue.

  12. potiuk commented on Mar 2, 2026

    @potiuk
    Member

    @nailo2c We are unassigning you from this issue as part of our updated assignment policy.

    This is not meant to discourage your contribution — quite the opposite! You are still very welcome to work on this issue and submit a PR for it. Simply comment that you are working on it and open a PR when ready.

    We found that formal assignments were not working well, as they often prevented others from contributing when the assignee was not actively working on the issue.

  13. potiuk commented on Mar 2, 2026

    @potiuk
    Member

    @Abhishekmishra2808 We are unassigning you from this issue as part of our updated assignment policy.

    This is not meant to discourage your contribution — quite the opposite! You are still very welcome to work on this issue and submit a PR for it. Simply comment that you are working on it and open a PR when ready.

    We found that formal assignments were not working well, as they often prevented others from contributing when the assignee was not actively working on the issue.

  14. potiuk commented on Mar 2, 2026

    @potiuk
    Member

    @Prab-27 We are unassigning you from this issue as part of our updated assignment policy.

    This is not meant to discourage your contribution — quite the opposite! You are still very welcome to work on this issue and submit a PR for it. Simply comment that you are working on it and open a PR when ready.

    We found that formal assignments were not working well, as they often prevented others from contributing when the assignee was not actively working on the issue.

  15. potiuk commented on Mar 2, 2026

    @potiuk
    Member

    @Srabasti We are unassigning you from this issue as part of our updated assignment policy.

    This is not meant to discourage your contribution — quite the opposite! You are still very welcome to work on this issue and submit a PR for it. Simply comment that you are working on it and open a PR when ready.

    We found that formal assignments were not working well, as they often prevented others from contributing when the assignee was not actively working on the issue.

  16. aaron-y-chen commented on Mar 9, 2026

    @aaron-y-chen
    Contributor

    Hi @shahar1, if there are any PRs that need system tests, please let me know, I think I could help.

  17. shahar1 commented on Apr 19, 2026

    @shahar1
    Contributor

    Closing this issue, as there's only a single operator left which will be treated separately.
    Thanks again everyone!

  18. yuseok89 commented on Apr 19, 2026

    @yuseok89
    Contributor

    This issue was open for a long time, and it's great to see it finally closed. Thanks to everyone who contributed, and special thanks to @shahar1 for driving this to completion.

  19. aaron-y-chen commented on Apr 19, 2026

    @aaron-y-chen
    Contributor

    Thanks for organizing this issue, great job @shahar1!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions