Add: predicated dispatch for host_build_graph (a2a3) - #1434
Merged
ChaoWao merged 1 commit intoJul 22, 2026
Merged
Conversation
Port the per-task dispatch predicate from tensormap_and_ringbuffer to
host_build_graph so the two a2a3 runtimes stay consistent. The predicate
is resolved NOT in orchestration but by the scheduler at the dispatch
point, so a statically-materialized task can be retired without dispatch
based on a value only a prior task produces — without the orchestrator
stalling to read it (MoE "skip empty experts" is the motivating case).
- pto_submit_types.h: PredicateOp {NONE,EQ,NE,GT,LT,GE,LE}, a
DispatchPredicate {addr,target,elem_size,op} with pass(), and the
PTO2_SUBTASK_FLAG_HAS_PREDICATE active_mask bit + accessors.
- pto_types.h: Arg carries an L0TaskPredicate {operand tensor+indices,
op, target}, set via args.set_predicate(pred).
- pto_runtime2_types.h / pto2_dispatch_payload.h: DispatchPredicate lives
on PTO2TaskPayload at a fixed cache line 9 (byte 576, before tensors so
it never moves as MAX_TENSOR_ARGS / MAX_SCALAR_ARGS change); tensors
shift to 640, scalars to 4736. AICore never reads it.
- pto_orchestrator.cpp: submit resolves the (tensor, indices) to an
absolute GM address and sets the flag when op != NONE.
- scheduler/pto_scheduler.h: every ready-routing point evaluates the
predicate — PASS dispatches normally, FAIL routes to dummy_ready_queue
and retires inline via the dep-only path. Predicated tasks are excluded
from early-dispatch staging.
Contract: the predicate tensor's producer MUST be a dependency of the
predicated task, so the value is current once the task becomes ready.
Test: tests/st/a2a3/host_build_graph/predicated_dispatch mirrors the tmr
scene test (gate_producer -> x_producer -> predicated clobber -> consumer).
gate=0 => predicate false => clobber not dispatched => sentinel survives
and the consumer still unlocks; gate=1 => dispatched => poison propagates.
Passes on a2a3sim.
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
|
Warning Review limit reached
Next review available in: 56 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (12)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
ChaoWao
approved these changes
Jul 22, 2026
ChaoWao
added a commit
to huawei-csl/simpler
that referenced
this pull request
Jul 22, 2026
… predicated dispatch Base advanced 7 commits; hw-native-sys#1434 (predicated dispatch for host_build_graph) touched the same slot-state / payload structs and scheduler ready-routing the polling port rewrote. Conflicts in two files, resolved as: - pto_runtime2_types.h: keep the polling payload layout (fanin_local_ids, spill/inline fanin removed) AND hw-native-sys#1434's alignas(64) DispatchPredicate at the fixed byte 576 (tensors -> 640). Static-asserts collapsed to the merged layout (fanin_local_ids@12, predicate@576, tensors@640). - scheduler/pto_scheduler.h: take the polling side. hw-native-sys#1434's predicate guards on the early-dispatch / release path (try_early_dispatch_release, propagate_dispatch_fanin, route_ready_once, release_fanin_and_check_ready) are moot -- polling deletes that whole path. Predicate ENFORCEMENT survives via push_ready_routed, which auto-merged cleanly: a task whose predicate fails routes to dummy_ready_queue and retires inline, exactly like a DUMMY task. The predicate composes with polling: it is orthogonal to the fanin/completion mechanism (when a task becomes ready) and gates only whether a ready task dispatches. tests/st/a2a3/host_build_graph/predicated_dispatch passes on a2a3sim (both PredicateFalseSkips and PredicateTrueDispatches); full hbg sim suite 11 passed; a2a3sim dfx-off build clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2 tasks
ChaoWao
added a commit
to huawei-csl/simpler
that referenced
this pull request
Jul 22, 2026
Replace host_build_graph's wiring dependency subsystem with the polling completion scheduler (introduced for tensormap_and_ringbuffer in hw-native-sys#1137). Removed: dep_pool / PTO2FaninPool / PTO2DepListPool, fanin_refcount / fanout_refcount / fanout_head / fanout_lock, the two-phase deferred producer-release, and most of relocate_host_orch_image. Added: per-slot completion_flags byte + monotonic completed_watermark + intrusive wake_list; position-independent inline integer fanin (fanin_local_ids) with last_consumer_local_id; a device boot initial-classify pass. Fits hbg's single-ring, whole-graph-resident, host-orchestrated model: integer fanin collapses the host->device pointer relocation to task/payload only. Only the dependency subsystem changes — dispatch, core management, sync_start, and the async/deferred-completion mailbox are preserved (deferred tasks reach the same on_mixed_task_complete endpoint, so they publish completion_flags too). Early-dispatch is stubbed for this milestone. Host-orch specifics: task_state is kept as the host-visible completion mirror (host polls it in wait_for_tensor_ready, the allocator deadlock detector, and the cold-path dump); completed_watermark is load-bearing — wait_for_consumers gates on watermark >= producer.last_consumer_local_id; hidden-alloc producers completed on the host pre-set their completion_flags in the H2D image. completed_watermark advances over the full contiguous completed prefix (bounded by current_task_index, not the completing task's id): a my_id cap would make the final value completion-order-dependent and hang wait_for_consumers when a low-id task completes last. Reconciles with predicated dispatch (hw-native-sys#1434): a predicate-failing task routes to dummy_ready_queue and retires inline, evaluated at push_ready_routed. docs: replace the migration plan with an updated RUNTIME_LOGIC.md describing the polling dependency/completion model. Validated: onboard + sim builds (host/aicpu/aicore); full hbg st suite passes on a2a3sim (11) and onboard a2a3 (11 pass / 1 skip), including predicated_dispatch; a2a3sim dfx-off build clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-Authored-By: Sergio Martin <sergio.miguel.martin@huawei.com>
ChaoWao
added a commit
to huawei-csl/simpler
that referenced
this pull request
Jul 22, 2026
Replace host_build_graph's wiring dependency subsystem with the polling completion scheduler (introduced for tensormap_and_ringbuffer in hw-native-sys#1137). Removed: dep_pool / PTO2FaninPool / PTO2DepListPool, fanin_refcount / fanout_refcount / fanout_head / fanout_lock, the two-phase deferred producer-release, and most of relocate_host_orch_image. Added: per-slot completion_flags byte + monotonic completed_watermark + intrusive wake_list; position-independent inline integer fanin (fanin_local_ids) with last_consumer_local_id; a device boot initial-classify pass. Fits hbg's single-ring, whole-graph-resident, host-orchestrated model: integer fanin collapses the host->device pointer relocation to task/payload only. Only the dependency subsystem changes — dispatch, core management, sync_start, and the async/deferred-completion mailbox are preserved (deferred tasks reach the same on_mixed_task_complete endpoint, so they publish completion_flags too). Early-dispatch is stubbed for this milestone. Host-orch specifics: task_state is kept as the host-visible completion mirror (host polls it in wait_for_tensor_ready, the allocator deadlock detector, and the cold-path dump); completed_watermark is load-bearing — wait_for_consumers gates on watermark >= producer.last_consumer_local_id; hidden-alloc producers completed on the host pre-set their completion_flags in the H2D image. completed_watermark advances over the full contiguous completed prefix (bounded by current_task_index, not the completing task's id): a my_id cap would make the final value completion-order-dependent and hang wait_for_consumers when a low-id task completes last. Reconciles with predicated dispatch (hw-native-sys#1434): a predicate-failing task routes to dummy_ready_queue and retires inline, evaluated at push_ready_routed. docs: replace the migration plan with an updated RUNTIME_LOGIC.md describing the polling dependency/completion model. Validated: onboard + sim builds (host/aicpu/aicore); full hbg st suite passes on a2a3sim (11) and onboard a2a3 (11 pass / 1 skip), including predicated_dispatch; a2a3sim dfx-off build clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-Authored-By: Sergio Martin <sergio.miguel.martin@huawei.com>
ChaoWao
added a commit
to huawei-csl/simpler
that referenced
this pull request
Jul 22, 2026
Replace host_build_graph's wiring dependency subsystem with the polling completion scheduler (introduced for tensormap_and_ringbuffer in hw-native-sys#1137). Removed: dep_pool / PTO2FaninPool / PTO2DepListPool, fanin_refcount / fanout_refcount / fanout_head / fanout_lock, the two-phase deferred producer-release, and most of relocate_host_orch_image. Added: per-slot completion_flags byte + monotonic completed_watermark + intrusive wake_list; position-independent inline integer fanin (fanin_local_ids) with last_consumer_local_id; a device boot initial-classify pass. Fits hbg's single-ring, whole-graph-resident, host-orchestrated model: integer fanin collapses the host->device pointer relocation to task/payload only. Only the dependency subsystem changes — dispatch, core management, sync_start, and the async/deferred-completion mailbox are preserved (deferred tasks reach the same on_mixed_task_complete endpoint, so they publish completion_flags too). Early-dispatch is stubbed for this milestone. Host-orch specifics: task_state is kept as the host-visible completion mirror (host polls it in wait_for_tensor_ready, the allocator deadlock detector, and the cold-path dump); completed_watermark is load-bearing — wait_for_consumers gates on watermark >= producer.last_consumer_local_id; hidden-alloc producers completed on the host pre-set their completion_flags in the H2D image. completed_watermark advances over the full contiguous completed prefix (bounded by current_task_index, not the completing task's id): a my_id cap would make the final value completion-order-dependent and hang wait_for_consumers when a low-id task completes last. Reconciles with predicated dispatch (hw-native-sys#1434): a predicate-failing task routes to dummy_ready_queue and retires inline, evaluated at push_ready_routed. docs: replace the migration plan with an updated RUNTIME_LOGIC.md describing the polling dependency/completion model. Validated: onboard + sim builds (host/aicpu/aicore); full hbg st suite passes on a2a3sim (11) and onboard a2a3 (11 pass / 1 skip), including predicated_dispatch; a2a3sim dfx-off build clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-Authored-By: Sergio Martin <sergio.miguel.martin@huawei.com>
ChaoWao
added a commit
that referenced
this pull request
Jul 22, 2026
Replace host_build_graph's wiring dependency subsystem with the polling completion scheduler (introduced for tensormap_and_ringbuffer in #1137). Removed: dep_pool / PTO2FaninPool / PTO2DepListPool, fanin_refcount / fanout_refcount / fanout_head / fanout_lock, the two-phase deferred producer-release, and most of relocate_host_orch_image. Added: per-slot completion_flags byte + monotonic completed_watermark + intrusive wake_list; position-independent inline integer fanin (fanin_local_ids) with last_consumer_local_id; a device boot initial-classify pass. Fits hbg's single-ring, whole-graph-resident, host-orchestrated model: integer fanin collapses the host->device pointer relocation to task/payload only. Only the dependency subsystem changes — dispatch, core management, sync_start, and the async/deferred-completion mailbox are preserved (deferred tasks reach the same on_mixed_task_complete endpoint, so they publish completion_flags too). Early-dispatch is stubbed for this milestone. Host-orch specifics: task_state is kept as the host-visible completion mirror (host polls it in wait_for_tensor_ready, the allocator deadlock detector, and the cold-path dump); completed_watermark is load-bearing — wait_for_consumers gates on watermark >= producer.last_consumer_local_id; hidden-alloc producers completed on the host pre-set their completion_flags in the H2D image. completed_watermark advances over the full contiguous completed prefix (bounded by current_task_index, not the completing task's id): a my_id cap would make the final value completion-order-dependent and hang wait_for_consumers when a low-id task completes last. Reconciles with predicated dispatch (#1434): a predicate-failing task routes to dummy_ready_queue and retires inline, evaluated at push_ready_routed. docs: replace the migration plan with an updated RUNTIME_LOGIC.md describing the polling dependency/completion model. Validated: onboard + sim builds (host/aicpu/aicore); full hbg st suite passes on a2a3sim (11) and onboard a2a3 (11 pass / 1 skip), including predicated_dispatch; a2a3sim dfx-off build clean. Co-authored-by: Sergio Martin <sergio.miguel.martin@huawei.com>
ChaoZheng109
added a commit
to ChaoZheng109/simpler
that referenced
this pull request
Jul 23, 2026
…hbg) Mirror of the tensormap_and_ringbuffer consolidation (hw-native-sys#1402) to the a2a3 host_build_graph runtime: fold the four scattered per-task markers into a single TaskAttrs byte on PTO2TaskSlotState, and revert active_mask to a pure subtask-slot mask. hbg gained predicated dispatch in hw-native-sys#1434, so it now carries the same four markers as tmr (allow_early_resolve, sync_start, has_predicate, timing tag) and this is a 1:1 port of the settled tmr design. - Add TaskAttrs (pto_submit_types.h): bit0 allow_early_resolve, bit1 sync_start, bit2 has_predicate, bit3 is_timed, bits4-7 timing_tag. - Drop SYNC_START/HAS_PREDICATE flag bits and accessors from ActiveMask; it now carries only core_mask (bits 0-2). - Remove task_timing_slot from PTO2TaskDescriptor; the tag now rides the scheduler's hot slot_state line. Descriptor size/ABI (40B, packed_buffer_base offset) unchanged; PTO2TaskSlotState stays 64B (task_attrs reuses the byte allow_early_resolve occupied, alongside the atomic lifecycle_flags). - prepare_task writes task_attrs per submit alongside active_mask; attributes are assembled at each submit entry point (submit_task, submit_dummy_task, alloc_tensors). - Update all scheduler read sites (dispatch/completion/scheduler.h) to read the attributes off slot_state. - Static-assert NUM_TASK_TIMING_SLOTS <= 16 and TASK_TIMING_SLOT_NONE == -1 so the 4-bit tag field and the untagged sentinel stay coupled. Accessor naming matches the settled tmr baseline: predicate getters (allow_early_resolve / requires_sync_start / has_predicate / is_timed), set_<concept> setters (set_early_resolve / set_sync_start / set_predicate / set_timing_slot). Behavior-neutral refactor. Verified: a2a3sim st (predicated_dispatch, matmul, bgemm, paged_attention, vector_example). The a2a3 cpput timing/wiring tests build against tmr headers (unchanged here). Part of hw-native-sys#1402.
ChaoZheng109
added a commit
that referenced
this pull request
Jul 23, 2026
…hbg) (#1441) Mirror of the tensormap_and_ringbuffer consolidation (#1402) to the a2a3 host_build_graph runtime: fold the four scattered per-task markers into a single TaskAttrs byte on PTO2TaskSlotState, and revert active_mask to a pure subtask-slot mask. hbg gained predicated dispatch in #1434, so it now carries the same four markers as tmr (allow_early_resolve, sync_start, has_predicate, timing tag) and this is a 1:1 port of the settled tmr design. - Add TaskAttrs (pto_submit_types.h): bit0 allow_early_resolve, bit1 sync_start, bit2 has_predicate, bit3 is_timed, bits4-7 timing_tag. - Drop SYNC_START/HAS_PREDICATE flag bits and accessors from ActiveMask; it now carries only core_mask (bits 0-2). - Remove task_timing_slot from PTO2TaskDescriptor; the tag now rides the scheduler's hot slot_state line. Descriptor size/ABI (40B, packed_buffer_base offset) unchanged; PTO2TaskSlotState stays 64B (task_attrs reuses the byte allow_early_resolve occupied, alongside the atomic lifecycle_flags). - prepare_task writes task_attrs per submit alongside active_mask; attributes are assembled at each submit entry point (submit_task, submit_dummy_task, alloc_tensors). - Update all scheduler read sites (dispatch/completion/scheduler.h) to read the attributes off slot_state. - Static-assert NUM_TASK_TIMING_SLOTS <= 16 and TASK_TIMING_SLOT_NONE == -1 so the 4-bit tag field and the untagged sentinel stay coupled. Accessor naming matches the settled tmr baseline: predicate getters (allow_early_resolve / requires_sync_start / has_predicate / is_timed), set_<concept> setters (set_early_resolve / set_sync_start / set_predicate / set_timing_slot). Behavior-neutral refactor. Verified: a2a3sim st (predicated_dispatch, matmul, bgemm, paged_attention, vector_example). The a2a3 cpput timing/wiring tests build against tmr headers (unchanged here). Part of #1402.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Port the per-task dispatch predicate from
tensormap_and_ringbuffertohost_build_graphso the two a2a3 runtimes stay consistent (host_build_graphwas the only a2a3 runtime still missing it). Mirrors the tmr feature
(originally #1314) file-for-file.
The predicate is resolved not in orchestration but by the scheduler at the
dispatch point, so a statically-materialized task can be retired without
dispatch based on a value only a prior task produces — without the orchestrator
stalling to read it (MoE "skip empty experts" is the motivating case).
PredicateOp {NONE,EQ,NE,GT,LT,GE,LE}, aDispatchPredicate {addr,target,elem_size,op}withpass(), and thePTO2_SUBTASK_FLAG_HAS_PREDICATEactive_mask bit + accessors.Argcarries anL0TaskPredicate {operand tensor+indices, op, target}, set viaargs.set_predicate(pred).DispatchPredicateliveson
PTO2TaskPayloadat a fixed cache line 9 (byte 576, beforetensorsso itnever moves as
MAX_TENSOR_ARGS/MAX_SCALAR_ARGSchange);tensorsshift to640,
scalarsto 4736. AICore never reads it.indices) to an absolute GM address and sets the flag when
op != NONE.predicate — PASS dispatches normally, FAIL routes to
dummy_ready_queueandretires inline via the dep-only path. Predicated tasks are excluded from
early-dispatch staging.
Contract: the predicate tensor's producer MUST be a dependency of the
predicated task, so the value is current once the task becomes ready.
Testing
tests/st/a2a3/host_build_graph/predicated_dispatchmirrors the tmr scene test:gate_producer -> x_producer -> predicated clobber -> consumer.case=1(gate=0): predicate FALSE → clobber not dispatched → sentinelsurvives, consumer still unlocks (Y = 42.0).
case=2(gate=1): predicate TRUE → clobber dispatched → poison propagates(Y = 999.0).
Simulation tests pass (a2a3sim, both cases)
Hardware tests pass — not run locally: NPU driver unavailable on this box
(
dcmi module initialize failed); relying on CI for onboard a2a3.