Skip to content

[Feature] hbg: Port Graph Execution to a5 #1715

Description

@ChaoZheng109

Summary

Port Graph Execution to src/a5/runtime/host_build_graph/. It landed for a2a3 in #1444 and
was never brought over, and it is now the only capability a5 HBG lacks relative to a2a3.

A file-by-file comparison of the two trees shows five a2a3-only files, all of them Graph
Execution:

runtime/graph_cache.h
runtime/graph_execution.h
runtime/graph_host_state.h
runtime/scheduler/graph_execution.cpp
docs/GRAPH_EXECUTION.md

Its plumbing also accounts for most of the content divergence in the files both trees share,
so the port is wider than those five files:

Shared file What a5 is missing
orchestration/pto_orchestration_api.h rt_submit_graph and the GRAPH_KEY identity overload
runtime/orchestrator_core/pto_orchestrator.cpp recording (GraphRecording, GraphRecordedNode, tensor-source classification), Definition compaction, boundary-contract validation, cache-hit submission
runtime/pto_runtime2_types.h the TaskKind enum (KERNEL / DUMMY / GRAPH / GRAPH_NODE) and the slot's task_kind field — a5 carries uint8_t _async_pad in that byte
runtime/scheduler/pto_scheduler.h, scheduler_dispatch.cpp TaskCompletionOutcome / stream_tasks_completed, so a graph-internal node contributes 0 to the host-visible completion count instead of 1; graph_prepare_queue / graph_ready_queue and their bounded per-loop expansion
runtime/pto_runtime2.h, host/runtime_maker.cpp active_callable_hash (the Definition cache key) and the exact-size POD submission-image upload path
runtime/scheduler/scheduler_cold_path.cpp graph slot reporting in the cold-path dump

Scene coverage comes with it: tests/st/a2a3/host_build_graph/graph_execution/ has four
scenes (AIV fanin/fanout DAG, Qwen-style decoder-layer DAG, three-slot multi-block MIX/SPMD,
manual Qwen3-14B three-layer decode) and a5 has none.

Motivation / Use Case

Graph Execution is the mechanism that makes host orchestration affordable for repetitive
graphs: the first invocation records the DAG, later invocations submit one GRAPH task whose
internal nodes the device Scheduler expands from a saved Definition, consuming no ring slots
and costing no re-submission. Without it, every repeated subgraph on a5 — an LLM decoder
layer submitted per layer and per token being the motivating shape — is rebuilt on the host
in full, every run.

That makes this a gap in the runtime's purpose, not just in its feature list. The
host_build_graph vision in #1706 rests on amortizing the host build cost, so an a5 HBG
without Graph Execution cannot reach the same destination as a2a3.

It is also the standing rule in #1706 that HBG is one runtime with two arch trees, and that a
deliberate divergence needs a stated hardware reason. There is no hardware reason here — the
absence is lag from #1444 not being mirrored.

Proposed API / Behavior

No new API. The a5 orchestration surface, wire format, and device-side behavior should match
a2a3's exactly, so docs/GRAPH_EXECUTION.md describes both after the port (either copied to
the a5 tree or made arch-neutral and shared).

Arch-specific points to verify rather than copy blindly:

  • Core geometry. a5 is RUNTIME_MAX_WORKER 108 (36 AIC + 72 AIV) against a2a3's 72
    (24 AIC + 48 AIV). Check the MIX placement bitset, whose flattened offset is cluster * 3,
    and any node/tensor ceiling derived from core counts.
  • Completion mailbox ABI. DeferredCompletionEntry is 32 bytes on a5 against 24 on a2a3.
    Confirm graph-internal node completion routes through the same endpoint on a5.
  • AICPU thread topology. a5 resolves a different aicpu_thread_num; the 3S+1P split and
    the graph prepare/ready queues are per-thread structures.
  • DFX. The Graph Execution and graph_prepare swimlane lanes need the a5 collectors,
    and a5's PMU path is pmu_aicpu_complete_record where a2a3 uses pmu_aicpu_record_task.

Bring the four graph_execution scenes over as part of the same change, so the port is
verified rather than merely compiled.

Alternatives Considered

Leave Graph Execution a2a3-only. Rejected: it would be a permanent capability split with
no hardware justification, which is exactly what #1706 asks contributors not to create
silently. If a5 is ever to be excluded on purpose, that needs its own decision and a stated
reason.

Wait for Graph Execution step 2 to settle first, then port once. Reasonable in principle,
but step 2 is open-ended (#1712, #1713 and the rest of the "Current unsupported cases" list),
so this would defer a5 indefinitely. Porting step 1 now and carrying step-2 work in both trees
per the usual same-commit rule keeps the trees converged.

Additional Context

Parent tracking issue: #1706 (host_build_graph — intent, vision & change tracking).

Related: #1444 (the a2a3 implementation), #1712 and #1713 (Graph Execution step-2 work on
a2a3), #1582 (the analogous a2a3-vs-a5 divergence reconciliation for
tensormap_and_ringbuffer).

Whatever a2a3 Graph Execution looks like when this is picked up is what should be ported —
this issue is not pinned to #1444's exact state.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions