Skip to content

Allow user-specified object attributes to be used in check_fn for S3KeySensor - #39950

Merged
vincbeck merged 4 commits into
apache:mainfrom
ellisms:main
May 31, 2024
Merged

vincbeck merged 4 commits into
apache:mainfrom
ellisms:main

Conversation

@ellisms

@ellisms ellisms commented May 30, 2024

Copy link
Copy Markdown
Contributor

Closes #39898
Allows user to specify list of attributes from head_object that will be included in args for check_fn. Defaults to 'Size' to maintain backwards compatibility for current implementations.

@ellisms
ellisms requested review from eladkal and o-nikolas as code owners May 30, 2024 18:22
@boring-cyborg boring-cyborg Bot added area:providers provider:amazon AWS/Amazon - related issues labels May 30, 2024
Comment thread airflow/providers/amazon/aws/sensors/s3.py Outdated
Comment thread airflow/providers/amazon/aws/sensors/s3.py Outdated
ellisms and others added 2 commits May 31, 2024 06:37
Co-authored-by: Vincent <97131062+vincbeck@users.noreply.github.com>

@eladkal eladkal left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@vincbeck
vincbeck merged commit fcd1a26 into apache:main May 31, 2024
fdemiane pushed a commit to fdemiane/airflow that referenced this pull request Jun 6, 2024
…eySensor (apache#39950)

* Ability to specify s3 object attributes for check_fn

* removed unncessary size check

* Update airflow/providers/amazon/aws/sensors/s3.py

Co-authored-by: Vincent <97131062+vincbeck@users.noreply.github.com>

---------

Co-authored-by: Vincent <97131062+vincbeck@users.noreply.github.com>
romsharon98 pushed a commit to romsharon98/airflow that referenced this pull request Jul 26, 2024
…eySensor (apache#39950)

* Ability to specify s3 object attributes for check_fn

* removed unncessary size check

* Update airflow/providers/amazon/aws/sensors/s3.py

Co-authored-by: Vincent <97131062+vincbeck@users.noreply.github.com>

---------

Co-authored-by: Vincent <97131062+vincbeck@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:amazon AWS/Amazon - related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow check_fn of S3KeySensor to be used in more use cases

3 participants