Skip to content

Deferrable HttpSensor does not move to Triggerer when using response_check - #47719

Closed
dabla wants to merge 49 commits into
apache:mainfrom
dabla:feature/deferrable-httpsensor-response-check
Closed

dabla wants to merge 49 commits into
apache:mainfrom
dabla:feature/deferrable-httpsensor-response-check

Conversation

@dabla

@dabla dabla commented Mar 13, 2025 •

Copy link
Copy Markdown
Contributor

Closes: #40209

DAG Code:
image

Task going to Deferrable mode:
image

Output of the task:
image


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

Comment thread airflow/exceptions.py Outdated
davidblain-infrabel and others added 3 commits March 13, 2025 11:59
# Conflicts:
#	providers/http/src/airflow/providers/http/sensors/http.py
#	providers/http/src/airflow/providers/http/triggers/http.py
Comment thread providers/http/tests/unit/http/sensors/test_http.py Outdated
Comment thread providers/http/tests/unit/http/sensors/test_http.py Outdated
@dabla

dabla commented Mar 13, 2025

Copy link
Copy Markdown
Contributor Author

@eladkal Still need to remove the pickling, probably the check should move so pickling won't be necessary anymore

@dabla
dabla marked this pull request as draft March 13, 2025 14:23
@dabla
dabla marked this pull request as ready for review April 15, 2025 11:40
@dabla
dabla requested a review from eladkal April 16, 2025 09:32
@dabla

dabla commented Apr 23, 2025

Copy link
Copy Markdown
Contributor Author

I think this one is ready

Comment thread providers/http/src/airflow/providers/http/triggers/http.py
Comment thread providers/http/src/airflow/providers/http/triggers/http.py
@dabla

dabla commented Feb 11, 2026

Copy link
Copy Markdown
Contributor Author

Will close this PR as we cannot pass callable's or lambda's to a Trigger as those are not serializable, which is a requirement for Triggers. Applying custom serialization or using third party libraries to attempt it isn't a good idea, Triggerers should be static and thus not contain dynamic code as Triggers must be executed from their serialized form, they don't have access to the Dag file contrary to workers/executors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Deferrable HttpSensor does not move to Triggerer when using response_check

6 participants