Repository navigation
Consider swapping httpx usage for httpx2 #70522
Description
Activity
I guess until starlette migrates, it makes little sense for us to keep both dependencies - so I would rather wait for them - maybe someone can open a PR doing the migration to test it and see if there are any compatibility issues - if you would like to do it - feel free.
I've migrated Starlette.
I was curious to see if / how FastAPI was migrating to httpx2. Currently, it looks like there might be some work towards a gradual migration.
Given the split components of Airflow (core, sdk, providers), I think it might make sense to address each individually. My hunch is that this would be a good way to split out the migration:
airflow-core: Temporarily support bothhttpxandhttpx2, with some tentative future release targetting the removal ofhttpx.task-sdk: It looks like this may be able to fully cutover in a single release.airflow-providers: Looking into this one currently, hoping to put up a PR for it if there aren't any unexpected issues.
I've also followed up on the FastAPI discussion thread about the httpx → httpx2 migration here. I believe that discussion thread is the main place to follow any progress on the migration in FastAPI.
Under which category would you file this issue?
Airflow Core
Apache Airflow version
3.x
What happened and how to reproduce it?
At the time of writing, it appears that the
httpxlibrary might not be actively maintained. There are a number of bug-fixes forhttpxwhich have not landed, but which are present inhttpx2(forked fromhttpx==0.28.1).Additionally, the maintainer of
starlette(one of the key sub-dependencies ofFastAPI) has confirmed that starlette will migrate tohttpx2. So it seems the migration fromhttpxinFastAPIis an eventuality.What you think should happen instead?
Any direct usage of
httpxwithin airflow-core / the task-sdk / airflow's providers should probably migrate to httpx2.The scope of this change is obviously large; but the migration of direct code reliance on
httpxis relatively straightforward.I'd imagine it would be good to add individual, well-defined / scoped sub-issues for this.
Work Tracking
The following subtasks need to be completed in order for this issue to be marked as closed:
httpx2#72111ignoreentries for providers which havehttpxas sub-dependency (requires potential modification of dependencies of those providers, less relevant).httpx2#73116Are you willing to submit PR?
Code of Conduct