Repository navigation
AIP-84 Migrate public endpoints from api_connexion to the new Rest API #42370
Description
Body
#42366 is taking care of migrating private endpoints.
This issue focuses on migrating all the public ones. Ideally they shouldn't change too much and be feature rich compare to airflow 2.x.
An exemple PR can be found here #42196
Tasks (total ~ 81 endpoints)
Backfill (6 endpoints) @dstandish
list_backfillsAIP-84 Migrate backfill API to fast api #43496 @dstandishget_backfillAIP-84 Migrate backfill API to fast api #43496 @dstandishpause_backfillAIP-84 Migrate backfill API to fast api #43496 @dstandishunpause_backfillAIP-84 Migrate backfill API to fast api #43496 @dstandishcancel_backfillAIP-84 Migrate backfill API to fast api #43496 @dstandishcreate_backfillAIP-84 Migrate backfill API to fast api #43496 @dstandish
Config (2 endpoints) @rawwar
Connection @bugraoz93 (6 endpoints)
api_connexion/endpoints/connection_endpoint.pydelete_connectionAIP-84 Migrate the public endpoint Delete Connection to FastAPI #42559 @bugraoz93api_connexion/endpoints/connection_endpoint.pyget_connectionAIP-84 Migrate the public endpoint Get Connection to FastAPI #42590 @rawwarapi_connexion/endpoints/connection_endpoint.pyget_connectionsAIP-84 Migrate the public endpoint Get Connections to FastAPI #42591 @bugraoz93api_connexion/endpoints/connection_endpoint.pypatch_connectionAIP-84 Migrate the public endpoint Patch Connection to FastAPI #42592 @bugraoz93api_connexion/endpoints/connection_endpoint.pypost_connectionAIP-84 Migrate the public endpoint Post Connection to FastAPI #42593 @bugraoz93api_connexion/endpoints/connection_endpoint.pytest_connectionAIP-84 Migrate the public endpoint Test Connection to FastAPI #42594 @bugraoz93
DAG (5 endpoints)
api_connexion/endpoints/dag_endpoint.pyget_dag_detailsAIP-84 Migrate the public endpoint DAG Details to FastAPI #42453 @omkar-fossapi_connexion/endpoints/dag_endpoint.pypatch_dagAIP-84 Migrate the public endpoint Update DAG to FastAPI #42468 @pierrejeambrunapi_connexion/endpoints/dag_endpoint.pypatch_dagsAIP-84 Migrate the public endpoint Batch Update DAG to FastAPI #42544 @pierrejeambrunapi_connexion/endpoints/dag_endpoint.pyget_dagAIP-84 Migrate the public endpoint Get DAG to FastAPI #42652 @omkar-fossapi_connexion/endpoints/dag_endpoint.pydelete_dagAIP-84 Migrate the public endpoint Delete DAG to FastAPI #42650 @omkar-foss
Dag Parsing (1 endpoint)
Dag Runs @rawwar (9 endpoints)
- All dag run endpoints
api_connexion/endpoints/dag_run_endpoint.pyAIP-84 Migrate Dag Run related public endpoint to FastAPI #42701 @rawwar
Dag Source (1 endpoint)
Dag Stats (1 endpoint)
Dag Warnings (1 endpoint) @rawwar
Dataset / Assets (10 endpoints) @amoghrajesh
get_assetsAIP-84 Migrating GET ASSETS Legacy API to fastAPI #43783 @amoghrajeshget_assetAIP 84: Migrate GET one ASSET legacy API to fast API #43825 @amoghrajeshget_upstream_asset_eventsAIP-84: Migrating GET Dataset events for DAG runs api to fastAPI #43874 @amoghrajeshget_asset_eventsAIP 84: Migrate GET ASSET EVENTS legacy API to fast API #43881 @vatsrahul1001create_asset_eventAIP 84: Migrate POST ASSET EVENT legacy API to fast API. #43984 @vatsrahul1001get_asset_queued_eventsAIP-84: Migrating GET queued asset events for assets to FASTAPI #44139 @vatsrahul1001delete_asset_queued_eventsAIP-84: Migrating DELETE queued asset events for assets to FASTAPI #44138 @vatsrahul1001get_dag_asset_queued_eventsAIP-84: Migrating GET queued asset events for DAG to fastAPI #44124 @amoghrajeshdelete_dag_asset_queued_eventsAIP-84: Migrating DELETE queued asset events for DAG to fastAPI #44129 @amoghrajeshdelete_dag_asset_queued_eventAIP-84: Migrating DELETE a queued asset events for DAG to fastAPI #44130 @amoghrajeshget_dag_asset_queued_eventAIP-84: Migrating GET one queued asset events for DAG to fastAPI #44128 @amoghrajesh
Event Logs (2 endpoints)
- AIP-84 Migrate the public endpoint Get Event Logs to FastAPI #43326 @jason810496
- AIP-84 Migrate the public endpoint Get Event Log to FastAPI #43327 @jason810496
Extra Link (1 endpoint)
Monitor (2 endpoints)
- AIP-84 Migrate public get Health info to FastAPI #42937 @bbovenzi
- AIP-84 Migrate the public endpoint Get Airflow Version Info to FastAPI #42879 @omkar-foss
Import Error (2 endpoints) @jason810496
- AIP-84 Migrate public endpoint Get Import Error to FastAPI #43594 @jason810496
AIP-84 Migrate public endpoint to list import errors to FastAPI #43595 @jason810496
Log (1 endpoint)
Plugin (1 endpoint)
Pool (5 endpoints) @pierrejeambrun
- AIP-84 Delete Pool #43165 @pierrejeambrun
- AIP-84 Get Pool #43221 @pierrejeambrun
- AIP-84 Get Pools #43223 @pierrejeambrun
- AIP-84 Patch Pool #43266 @pierrejeambrun
- AIP-84 Post Pool #43317 @pierrejeambrun
Provider (1 endpoint)
Task (2 endpoints) @omkar-foss
- AIP-84 Migrate the public endpoint Get Task to FastAPI #42874 @omkar-foss
- AIP-84 Migrate the public endpoint Get Tasks to FastAPI #42875 @omkar-foss
Task Instance (15 endpoints) @pierrejeambrun
- AIP-84 Get Task Instance #43485 @pierrejeambrun
- AIP-84 Get Mapped Task Instance #43548 @pierrejeambrun
- AIP-84 List Mapped Task Instances #43642 @pierrejeambrun
- AIP-84 Get Task Instances #43748 @pierrejeambrun
- AIP-84 Get Task Instance Dependencies #43749 @pierrejeambrun
- AIP-84 Get Mapped Task Instance Dependencies #43750 @pierrejeambrun
- AIP-84 Get Task Instances Batch #43756 @pierrejeambrun
- AIP-84 Post Clear Task Instances #43751 @omkar-foss
- AIP-84 Patch Task Instance #43753 @omkar-foss
- AIP-84 Patch Mapped Task Instance #43754 @omkar-foss
- AIP-84 Set Task Instances State #43752 @omkar-foss
- AIP-84 Get Task Instance Try Details #43675 @kandharvishnu
- AIP-84 Get Mapped Task Instance Try Details #44206 @kandharvishnu
- AIP-84 Get Task Instance Tries #44301 @kandharvishnu
- AIP-84 Get Mapped Task Instance Tries #44303 @kandharvishnu
Variables (5 endpoints)
- AIP-84 Delete Variable #42798 @pierrejeambrun
- AIP-84 Get Variable #42834 @pierrejeambrun
- AIP-84 Patch Variable #42929 @pierrejeambrun
- AIP-84 Post variable #42948 @pierrejeambrun
- AIP-84 Get Variables #43083 @pierrejeambrun
XCom (2 endpoints)
- AIP-84 Migrate XCom get entry public endpoint to FastAPI #42978 @michaeljs-c
- AIP-84 Migrate XCom get entries public endpoint to FastAPI #42980 @michaeljs-c
Committer
- I acknowledge that I am a maintainer/committer of the Apache Airflow project.
Activity
omkar-foss commentedon Sep 23, 2024
Trimming down
views.pyis very tempting 😄May I pick this up? If yes, please assign this issue to me. Thanks!
pierrejeambrun commentedon Sep 23, 2024
Hello @omkar-foss,
views.pyare mostly 'rendered' front end that will get replaced by the new UI react, inviews.pyyou also findprivate/UIREST route, most of them are with the/object/prefix, such asdef grid_data. If you want to work on those UI dedicated endpoints you can that would be part of #42366.This issue focuses on the public endpoints, those are located under the
api_connexionfolder.In anycase, just mention the endpoint you are working on, so people do not do the same ones, and I will update the description to track that :). I think we should do one at the time.
omkar-foss commentedon Sep 23, 2024
Hi @pierrejeambrun, thank you for explaining it to me, much appreciate it.
I'll start migrating the public endpoints, beginning with migration of the DAG Details API (
get_dag_details) inapi_connexion/endpoints/dag_endpoint.py.Since this is a meta issue (missed that previously!), I'll create the issue/PR corresponding to the endpoint(s) that I'll be working on and link to this issue (like usual).
P.S:
views.pywill have to wait :)bugraoz93 commentedon Sep 28, 2024
Thanks for creating the ticket @pierrejeambrun! Let me take connection endpoints
pierrejeambrun commentedon Sep 30, 2024
Great, thanks @bugraoz93
bugraoz93 commentedon Sep 30, 2024
Thanks for assigning! Let me save you from regular updates :) I created issues for all the endpoints in connections.
pierrejeambrun commentedon Oct 1, 2024
Great, thanks @bugraoz93. I just updated the meta issue with those new additions. Let me know if you plan to work on all of them so I can assign you, or just some of them so I can label the others with
good first issueand let other contributors grab them :)bugraoz93 commentedon Oct 1, 2024
I am planning to work on all of them @pierrejeambrun. if you can assign them to me, that would be great! Thanks :)
omkar-foss commentedon Oct 2, 2024
Hey folks! After #42631 (open for review), I can pick up these two next in order:
get_dag- AIP-84 Migrate the public endpoint Get DAG to FastAPI #42652delete_dag- AIP-84 Migrate the public endpoint Delete DAG to FastAPI #42650With these 2 done, I suppose we should have the
dags.pymigration complete. When you find some time, please update this meta and assign these to me, thank you! 😃13 remaining items
jason810496 commentedon Nov 1, 2024
Hi @pierrejeambrun, I can handle the migrations for Import Error
get_import_errorget_import_errorsI’m still encountering the issue where I can’t create the correct issue type.
Could you please create it and assign it to me? Thanks!
rawwar commentedon Nov 2, 2024
I've unassigned myself for the config endpoint issue. If no one takes it, I'll pick it up once I finish DagRun and DagWarning endpoints.
Issue: #42745 <- If anyone wants to take this, please let me know and I can assign it to you.
omkar-foss commentedon Nov 6, 2024
Hi @pierrejeambrun, could you please assign these TI-related endpoints to me in above issue description:
GET:
POST:
PUT:
pierrejeambrun commentedon Nov 6, 2024
Hey omkar, as you can see in I'm already assigned to the
TaskInstancesendpoint.Thanks for creating the issue, we can split the work on that matter, i'll assign some of them to you maybe all the
PUT/POSTas I am already working on GET.pierrejeambrun commentedon Nov 6, 2024
You're assigned, issue updated, let me know if that shounds good for you :).
omkar-foss commentedon Nov 8, 2024
Yes sure, that works. Thank you :)
bbovenzi commentedon Nov 20, 2024
@prabhusneha I see you're assigned to Logs. What is the status of getting the logs for a task instance? That would be a useful one to get in the UI soon!
utkarsharma2 commentedon Nov 21, 2024
Hey, @bbovenzi I'm taking care of
task instance logsendpoint, I'm mostly done with it, but there are a couple of test cases that require setting users in fastapi which requires a little more work. I'll keep you posted.phanikumv commentedon Nov 21, 2024
Thanks @utkarsharma2 (updated the description accordingly).
pierrejeambrun commentedon Nov 28, 2024
Thanks a lot to everyone that contributed, closing a big one 🎉
[-]AIP-84 Migrate public endpoints from api_connexion to the new rest api[/-][+]AIP-84 Migrate public endpoints from api_connexion to the new Rest API[/+]