Skip to content

Java SDK: Resolve a task's arguments from the Dag's own wiring - #73596

Merged
jason810496 merged 3 commits into
apache:mainfrom
jason810496:feature/java-sdk-wired-inputs
Oct 2, 2026
Merged

Java SDK: Resolve a task's arguments from the Dag's own wiring#73596
jason810496 merged 3 commits into
apache:mainfrom
jason810496:feature/java-sdk-wired-inputs

Conversation

@jason810496

@jason810496 jason810496 commented Sep 23, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Merge order:

  1. Java SDK: Resolve a task's arguments from the Dag's own wiring #73596 — Resolve a task's arguments from the Dag's own wiring (current one)
  2. Java SDK: Declare a Dag's task graph with @Builder.Deps #71189 — Declare a Dag's task graph with @Builder.Deps
  3. Java SDK: Serialize native Dags to DagSerialization v3 #71190 — Serialize native Dags to DagSerialization v3

Why

This PR add the argument binding for native Dag TaskFlow.

How

  • Binding semantic:
    • Positional-based should raise error for both getting "more" and "less" arguments on the other side
    • Struct-based should be warning instead
  • The XCom(s) are resolved concurrently
  • A value that resolves to nothing still fails the same way: null for a boxed or reference parameter, MissingXComException naming the task for a primitive.

This is the runtime half on its own. Nothing declares wired inputs until the next PR adds @Builder.Deps.

Was generative AI tooling used to co-author this PR?

This was referenced Sep 23, 2026
Merged
Merged
Merged
Merged
@jason810496
jason810496 force-pushed the feature/java-sdk-wired-inputs branch 6 times, most recently from 77d08ec to 7a07fa3 Compare September 27, 2026 14:48
@jason810496 jason810496 mentioned this pull request Sep 27, 2026
Closed
1 task done
@jason810496
jason810496 force-pushed the feature/java-sdk-wired-inputs branch 2 times, most recently from 5d53589 to da51a93 Compare September 27, 2026 15:39
@jason810496 jason810496 mentioned this pull request Sep 28, 2026
Closed
1 task done
@jason810496
jason810496 force-pushed the feature/java-sdk-wired-inputs branch 3 times, most recently from 0621b3d to 9a6166e Compare September 30, 2026 08:40
@jason810496 jason810496 self-assigned this Sep 30, 2026
@jason810496
jason810496 marked this pull request as ready for review September 30, 2026 09:01
guan404ming
guan404ming reviewed Sep 30, 2026
View reviewed changes

@guan404ming guan404ming left a comment •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not really confident with my knowledge about Java. I reviewed this with Claude Code's assistance and checked by myself after that. Please feel free to correct me if I have any misunderstanding. Thanks!

Comment thread java-sdk/sdk/src/main/kotlin/org/apache/airflow/sdk/Context.kt Outdated
Comment thread java-sdk/sdk/src/main/kotlin/org/apache/airflow/sdk/internal/ArgValues.kt Outdated
@jason810496
jason810496 marked this pull request as draft October 1, 2026 06:55
@jason810496
jason810496 marked this pull request as ready for review October 1, 2026 08:31
jason810496
jason810496 commented Oct 1, 2026
View reviewed changes

@jason810496 jason810496 left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review

Comment thread java-sdk/sdk/src/main/kotlin/org/apache/airflow/sdk/Context.kt Outdated
Comment thread java-sdk/sdk/src/main/kotlin/org/apache/airflow/sdk/internal/ArgValues.kt Outdated
guan404ming
guan404ming approved these changes Oct 2, 2026
View reviewed changes

@guan404ming guan404ming left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@jason810496 @guan404ming
Java SDK: Resolve a task's arguments from the Dag's own wiring
06a45fa 
A Dag authored in Java has no Python call site, so the supervisor sends no
argument bindings for it and a task's data parameters had nothing to
resolve against. Where the Dag wired inputs for a task, those stand in;
a stub-backed task keeps reading its bindings, including when the call
site bound none.

The authoring surface that records those inputs lands in the next commit,
so nothing declares them yet.
@jason810496 @guan404ming
Keep taskDef out of Context's constructor and reuse resolveWiredAll
04855b4
@guan404ming
guan404ming force-pushed the feature/java-sdk-wired-inputs branch from 24bffde to 04855b4 Compare October 2, 2026 07:50
henry3260
henry3260 approved these changes Oct 2, 2026
View reviewed changes
Comment thread java-sdk/sdk/src/main/kotlin/org/apache/airflow/sdk/internal/ArgValues.kt Outdated
uranusjr
uranusjr approved these changes Oct 2, 2026
View reviewed changes

@uranusjr uranusjr left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The one open nit should be addressed, otherwise lgtm.

@jason810496
jason810496 merged commit 402d788 into apache:main Oct 2, 2026
91 checks passed
@jason810496 jason810496 mentioned this pull request Oct 2, 2026
Merged
1 task done
@FrankYang0529 FrankYang0529 mentioned this pull request Oct 5, 2026
Merged
1 task done
@jason810496 jason810496 mentioned this pull request Oct 5, 2026
Merged
1 task done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@uranusjr uranusjr uranusjr approved these changes
@pierrejeambrun pierrejeambrun pierrejeambrun approved these changes
@guan404ming guan404ming guan404ming approved these changes
@henry3260 henry3260 henry3260 approved these changes
+1 more reviewer
@FrankYang0529 FrankYang0529 FrankYang0529 approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@jason810496 jason810496

Labels

area:dev-tools area:e2e-tests area:java-sdk kind:documentation

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@jason810496 @uranusjr @FrankYang0529 @pierrejeambrun @guan404ming @henry3260