Skip to content

TS SDK: answer TaskHandlerParseRequest from bundle.serve - #73976

Draft
jason810496 wants to merge 3 commits into
jason/core-taskhandler-refactor/08-java-sdk-task-handler-parsefrom
jason/core-taskhandler-refactor/07-ts-sdk-task-handler-parse
Draft

jason810496 wants to merge 3 commits into
jason/core-taskhandler-refactor/08-java-sdk-task-handler-parsefrom
jason/core-taskhandler-refactor/07-ts-sdk-task-handler-parse

Conversation

@jason810496

@jason810496 jason810496 commented Sep 30, 2026 •

Copy link
Copy Markdown
Member

Stack (bottom to top), on native #74037 (stack #74170): #73973, #73974, #73975, #74317, #73976, #74067, #74135, #74140, #74141, #73971, #74030, #74031, #74032, #74136, #74137, #74138, #74139

bundle.serve() now answers #73974's probe with every TaskHandler the bundle registers, and NodeCoordinator can find and start the bundle a stub task would run. Until now a TypeScript bundle failed on a TaskHandlerParseRequest as an unsupported first frame. Nothing calls the probe yet: #74135 does.

bundle.register(
  new TaskHandler("etl", "summarize", summarize),
  new TaskHandler("etl", "report", withArgNames({ label: "run_label" }, report)),
);
await bundle.serve();

answers:

TaskHandlerParsingResult(
    fileloc="/bundles/ts/etl.min.mjs",
    task_handlers={"etl": [
        TaskHandlerDeclaration(task_id="summarize", binding="named", params=None),
        TaskHandlerDeclaration(task_id="report", binding="named", params=None),
    ]},
)
  • Only TaskHandlers are declared, keyed by Dag id in registration order. Tasks of a Dag declared in TypeScript are left out.
  • TypeScript types are erased, so the SDK cannot list a handler's params, even with withArgNames. Each handler is named with params=None, and the Dag processor checks only that it exists.
  • bundle.serve() waits up to 30 s for the parent to acknowledge the answer. An error, a closed socket or a timeout fails the process.
  • The probe checks the bundle a worker would run: _find_task_handler_artifact is the worker's own scan, and _build_parse_task_handler_command is native Parse native TypeScript Dags with the Node coordinator #74037's _build_bundle_command, which verifies the bundle and runs it with node as a task does.

Compatibility: a bundle packed from main before this change reports 2026-10-30 but cannot answer, so its probe fails and #74135 logs a warning. Packing it again fixes it. A bundle packed with the published 0.1.0-beta1 is a bundle.mjs, which the scan does not find, so it is not probed (a worker cannot run it either).


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: Claude Code (Opus 5.5) following the guidelines

@boring-cyborg boring-cyborg Bot added area:coordinator Coordinator: The interface to spawn Lang-SDK subprocesses area:DAG-processing area:task-sdk area:ts-sdk labels Sep 30, 2026
@jason810496
jason810496 added this pull request to stack #73978 September 30, 2026 17:59
@jason810496 jason810496 changed the title jason/core taskhandler refactor/07 ts sdk task handler parse TS SDK: answer TaskHandlerParseRequest from bundle.serve Sep 30, 2026
@jason810496
jason810496 force-pushed the jason/core-taskhandler-refactor/07-ts-sdk-task-handler-parse branch from 892f414 to e48a8b6 Compare October 1, 2026 06:26
@jason810496
jason810496 force-pushed the jason/core-taskhandler-refactor/07-ts-sdk-task-handler-parse branch from e48a8b6 to 5b770b6 Compare October 1, 2026 06:30
@jason810496
jason810496 force-pushed the jason/core-taskhandler-refactor/07-ts-sdk-task-handler-parse branch from 5b770b6 to 3d1332e Compare October 1, 2026 10:55
@jason810496
jason810496 force-pushed the jason/core-taskhandler-refactor/07-ts-sdk-task-handler-parse branch from 3d1332e to dc5a082 Compare October 1, 2026 14:44
@jason810496
jason810496 force-pushed the jason/core-taskhandler-refactor/07-ts-sdk-task-handler-parse branch 2 times, most recently from 23843ba to 0f344dc Compare October 2, 2026 05:48
The Dag processor asks the runtime a stub task would run which task handlers it registers, and a TypeScript bundle fails on that first frame as an unsupported message. bundle.serve() now replies with one TaskHandlerParsingResult listing every TaskHandler, keyed by Dag id with each Dag's handlers in registration order, so the answer depends only on the bundle. Types are erased, so each handler declares named with null params, and the Dag processor checks only that it exists. The tasks of a Dag declared in TypeScript are never task handlers and are left out. The result goes out as a request, and the runtime waits for the parent's acknowledgement before it exits, as the Go and Java runtimes do.
The probe from #73974 needs each coordinator to find the artifact a stub task runs and to start its runtime. NodeCoordinator finds the bundle with the scan that execute_task uses and returns its path with symlinks resolved, as the Go and Java coordinators do, since the scan returns the path it walked, which can pass through symlinks. The probe verifies the bundle again and runs it with node, as a task does.
The coordinator and SDK tests each fake one side, so nothing showed that a bundle packed with airflow-ts-pack answers the way the Dag processor parses it. The test packs ts-sdk/example from a copy of the SDK sources, finds it as a stub task of typescript_example would, and probes it. It needs Node.js 22 and pnpm, so it runs only with AIRFLOW_LANG_SDK_REAL_PROBE_TESTS=1, and it skips with the reason when node is missing, too old, or cannot report its version.
@jason810496
jason810496 force-pushed the jason/core-taskhandler-refactor/07-ts-sdk-task-handler-parse branch from 6c3e3e3 to b52f31f Compare October 5, 2026 18:12
@jason810496
jason810496 removed this pull request from stack #73978 October 6, 2026 02:27
@jason810496
jason810496 changed the base branch from jason/core-taskhandler-refactor/06-go-sdk-task-handler-parse to jason/core-taskhandler-refactor/08-java-sdk-task-handler-parse October 6, 2026 02:34
@jason810496
jason810496 added this pull request to stack #74318 October 6, 2026 02:35
@jason810496
jason810496 force-pushed the jason/core-taskhandler-refactor/07-ts-sdk-task-handler-parse branch from b52f31f to 9c284fc Compare October 6, 2026 02:44

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:coordinator Coordinator: The interface to spawn Lang-SDK subprocesses area:DAG-processing area:task-sdk area:ts-sdk

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant