Refactor: unify TaskArgs on strided Tensor, drop ContinuousTensor - #1093
Merged
Merged
Conversation
TaskArgs / wire / bindings previously carried the 40 B ContinuousTensor (contiguous-only). They now carry the unified 128 B strided Tensor, so a single tensor type spans host construction, the wire, and the runtime. Behavior is unchanged: this construction path stays contiguous (row-major strides, start_offset == 0), enforced at the entry points. - Sink Tensor / PTOBufferHandle / make_tensor_external / TensorArgType into src/common/task_interface/tensor.h; move pto_task_id.h to common. - Keep runtime-only pieces in runtime: Segment / OverlapStatus -> pto_tensormap.h; TensorCreateInfo + materialization (now free functions) -> tensor_create_info.h. - Factor always_assert / AssertionError into shared assert_compat.h; add a host-side assert_compat.cpp linked into the binding and the cpp UTs. - Add child_memory (byte 43, cache line 1) + is_child_memory()/nbytes()/ data_as<T>() to Tensor; make the default ctor public for POD/array storage. - TaskArgs / ChipStorageTaskArgs / TaskArgsView use Tensor; bump MAILBOX_SIZE 16384 -> 32768 (128 B x 128 slots exceeds the old mailbox) and update the capacity static_assert. - Migrate consumers off .data -> buffer.addr (orchestrator, runtime_maker, aicpu_executor, remote_endpoint, remote_wire encode/decode, tensor_info). - make_tensor_arg returns Tensor and rejects non-contiguous torch tensors; from_tensor_arg becomes a const Tensor& passthrough. - Delete tensor_arg.h; rename RUNTIME_MAX_TENSOR_DIMS -> MAX_TENSOR_DIMS. - Update cpp/py unit tests, examples, and docs (wire layout 40 B -> 128 B).
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughReplaces the compact ChangesContinuousTensor → Tensor Migration
Sequence Diagram(s)sequenceDiagram
participant User as Python User
participant Orchestrator
participant TaskArgs as TaskArgs(Tensor)
participant MailboxWorker as LocalMailboxEndpoint
participant RemoteEndpoint as RemoteL3Endpoint
participant RemoteWire
User->>Orchestrator: alloc(shape, dtype) → Tensor (buffer.addr set)
User->>TaskArgs: add_tensor(Tensor, TensorArgType)
User->>Orchestrator: submit(task, TaskArgs)
Orchestrator->>Orchestrator: infer_deps via tensor.buffer.addr
Orchestrator->>Orchestrator: reserve_outputs_and_slot, write heap addr → tensor.buffer.addr
alt Local submission
Orchestrator->>MailboxWorker: write_blob (tensors as sizeof(Tensor) each)
MailboxWorker->>MailboxWorker: read_blob → TaskArgsView (const Tensor*)
else Remote submission
Orchestrator->>RemoteEndpoint: build_task_payload
RemoteEndpoint->>RemoteWire: encode_tensor(Tensor) → wire bytes
RemoteWire->>RemoteEndpoint: decode_tensor → Tensor via make_tensor_external
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request unifies the tensor representation across the codebase by replacing the compact 40-byte ContinuousTensor with a unified, strided 128-byte Tensor descriptor. This change spans the C++ runtime, Python bindings, documentation, examples, and tests, and requires increasing the shared memory mailbox size to 32 KB. The review feedback highlights several critical memory safety issues where fields or padding bytes are left uninitialized in the default constructors and initialization helpers of Tensor and TensorCreateInfo across both the a2a3 and a5 platforms. Since these structures are copied via memcpy or transmitted over shared memory, explicitly zero-initializing these fields as suggested is highly recommended to prevent potential data leakage, non-deterministic behavior, or MemorySanitizer (MSan) errors.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 13
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
src/common/hierarchical/remote_wire.cpp (2)
339-375:⚠️ Potential issue | 🟠 Major | ⚡ Quick winValidate contiguity before dropping stride metadata.
Line 343 serializes only contiguous-defining fields, while Lines 371-375 rebuild row-major strides and
start_offset == 0. A stridedTensorthat reaches this boundary would be decoded as a different layout. Add an encode-side invariant check for contiguous, zero-offset tensors, or serialize the strided fields too.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/common/hierarchical/remote_wire.cpp` around lines 339 - 375, The encode_tensor function only serializes contiguous-defining fields and discards stride and start_offset information, while decode_tensor assumes row-major strides and zero start_offset when reconstructing the tensor. This means non-contiguous tensors or tensors with non-zero start_offset would be silently decoded incorrectly. Add an ensure check in the encode_tensor function to validate that the input tensor is contiguous and has a zero start_offset before encoding, preventing corruption of strided or offset tensors that should not be serialized in this format.
423-435:⚠️ Potential issue | 🟠 Major | ⚡ Quick winMirror the no-sidecar tensor invariant in the wire codec.
RemoteL3Endpoint::build_task_payloadrejectschild_memoryor non-zerotensor.nbytes()whentensor_sidecar.presentis false, butencode_remote_task_args/decode_remote_task_argscan still accept the same invalid zero-address metadata when called directly or fed by a peer. Validate absent sidecars here too, before malformed payloads can reach execution.Suggested validation shape
for (size_t i = 0; i < args.tensor_metadata.size(); ++i) { + const Tensor &tensor = args.tensor_metadata[i]; RemoteTensorSidecar sidecar{}; if (!args.remote_desc.empty()) sidecar = args.remote_desc[i]; + if (!sidecar.present) { + ensure(!tensor.is_child_memory(), "remote_wire: child-memory tensor submitted without remote sidecar"); + ensure(tensor.nbytes() == 0, "remote_wire: tensor payload submitted without remote sidecar"); + } put_u8(out, sidecar.present ? 1 : 0); if (sidecar.present) { validate_desc_against_inline_payload(sidecar.desc, args.inline_payload.size()); auto encoded = encode_remote_tensor_desc(sidecar.desc); put_bytes(out, encoded.data(), encoded.size()); @@ - for (const auto &sidecar : args.remote_desc) { - if (sidecar.present) validate_desc_against_inline_payload(sidecar.desc, args.inline_payload.size()); + for (size_t i = 0; i < args.remote_desc.size(); ++i) { + const auto &tensor = args.tensor_metadata[i]; + const auto &sidecar = args.remote_desc[i]; + if (!sidecar.present) { + ensure(!tensor.is_child_memory(), "remote_wire: child-memory tensor submitted without remote sidecar"); + ensure(tensor.nbytes() == 0, "remote_wire: tensor payload submitted without remote sidecar"); + continue; + } + validate_desc_against_inline_payload(sidecar.desc, args.inline_payload.size()); }Also applies to: 455-479
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/common/hierarchical/remote_wire.cpp` around lines 423 - 435, In the tensor sidecar encoding loop within the function handling remote task arguments, when `sidecar.present` is false, add validation to ensure that the corresponding tensor metadata does not have `child_memory` or non-zero `tensor.nbytes()`, mirroring the invariant checks enforced in `RemoteL3Endpoint::build_task_payload`. Add this validation in the else branch (when the sidecar is not present) before processing continues, and apply the same validation logic to the corresponding decode functions as well to prevent malformed payloads from being accepted when called directly or by peer endpoints.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/task-flow.md`:
- Line 593: Update the broken documentation link on line 593 in
docs/task-flow.md that currently references the deleted tensor_arg.h file.
Replace the reference to point to the correct file path
../src/common/task_interface/tensor.h where the Tensor POD and TensorArgType
enum are now located. Keep the description of what these symbols are (Tensor POD
and TensorArgType enum) intact while only updating the file path in the markdown
link.
In `@python/bindings/task_interface.cpp`:
- Around line 151-159: The ndims property setter allows invalid values (0 or
above MAX_TENSOR_DIMS) without validation or stride/size rebuilding, which can
cause buffer overflows when accessing the fixed shapes/strides arrays. Remove
the setter lambda from the def_prop_rw call for the ndims property so it becomes
read-only, keeping only the getter lambda. Users should change tensor dimensions
through the existing shapes setter instead of directly modifying ndims.
In `@python/simpler/remote_l3_protocol.py`:
- Around line 440-442: The validation in the ndims check after reader.u32() is
incomplete and does not match the C++ decoder behavior. Currently it only
validates the upper bound with ndims > MAX_TENSOR_DIMS, but it also needs to
reject the lower bound. Add a validation check to ensure ndims is greater than 0
(rejecting zero-dimensional or invalid tensor dimensions) to match the C++
protocol requirement of ndims > 0 && ndims <= MAX_TENSOR_DIMS. Either add a
separate condition before the existing MAX_TENSOR_DIMS check or combine both
bounds into a single conditional statement.
In `@python/simpler/task_interface.py`:
- Around line 34-44: The removal of ContinuousTensor and
CONTINUOUS_TENSOR_MAX_DIMS from the public re-export surface breaks backward
compatibility for existing code importing these names. Add compatibility aliases
after the import statements in task_interface.py by creating assignments like
ContinuousTensor = Tensor and CONTINUOUS_TENSOR_MAX_DIMS = MAX_TENSOR_DIMS,
which map the old public names to their new equivalents (Tensor and
MAX_TENSOR_DIMS). This maintains backward compatibility while allowing the
internal refactoring to use the newer names.
In `@simpler_setup/torch_interop.py`:
- Around line 83-91: The `make_tensor_arg` function in torch_interop.py
currently only checks tensor contiguity but does not verify the tensor is on the
CPU device. A CUDA tensor can be contiguous yet pass an incorrect device pointer
to `Tensor.make()` with the wrong memory semantics, causing silent runtime
corruption. Add an explicit device check in `make_tensor_arg` after
`_ensure_torch_map()` to ensure the input tensor is on CPU, raising a ValueError
with a descriptive message if the tensor is on any other device (such as CUDA).
This check should occur alongside or before the existing contiguity validation
to catch the issue early.
In `@src/a2a3/runtime/tensormap_and_ringbuffer/runtime/tensor_create_info.h`:
- Around line 33-49: The TensorCreateInfo constructor does not validate that
ndims_in is within valid bounds before using it in the loop that writes to the
shapes array. Add a bounds check before the for loop to ensure ndims_in does not
exceed MAX_TENSOR_DIMS, and use an assertion or error handling to prevent
out-of-bounds memory writes to the shapes array. This validation should occur
after the member variable assignments and before the loop that copies shape
values from shapes_in into shapes.
In `@src/a5/runtime/tensormap_and_ringbuffer/runtime/tensor_create_info.h`:
- Around line 33-49: The TensorCreateInfo constructor has an unchecked loop that
iterates up to ndims_in when populating the shapes array, which can cause
out-of-bounds writes if ndims_in exceeds MAX_TENSOR_DIMS. Add a guard to cap
ndims_in to MAX_TENSOR_DIMS before using it in the loop where shapes are
assigned (line 46), ensuring that the loop bound cannot exceed the bounds of the
shapes array. This should apply the same defensive check that exists elsewhere
in the codebase for similar ndims_in usage.
In `@src/common/hierarchical/orchestrator.cpp`:
- Around line 70-72: The Orchestrator::alloc method does not validate whether
the shape vector is empty before attempting allocation, which can result in a
Tensor with an invalid buffer address. Add a check at the beginning of the
Orchestrator::alloc method to reject empty shapes by throwing an
std::invalid_argument exception with a descriptive message, similar to the
existing MAX_TENSOR_DIMS validation. Apply the same check to any other
allocation methods in the same range (lines 127-136) that may have similar
issues.
- Around line 501-502: The issue is that when output_alloc_bytes returns 0 for a
zero-byte output, the code still performs pointer arithmetic and assigns a
non-null address to t.buffer.addr via the reinterpret_cast. This can cause
null-pointer arithmetic and incorrectly assign non-null keys to zero-byte
outputs. Add a conditional check before the buffer address assignment to skip it
when slab equals 0, thereby preserving the null-tensor sentinel value for
zero-byte outputs.
In `@src/common/hierarchical/remote_wire.h`:
- Around line 167-168: The encode_tensor function silently drops strides and
start_offset information when encoding a Tensor, causing strided tensors to
become row-major contiguous on the remote side without warning. Add a validation
guard in encode_tensor that checks if the tensor has non-standard strides or
non-zero start_offset and either rejects the encoding with an error message or
serializes the full layout metadata. This prevents silent data layout corruption
during remote transmission by ensuring decode_tensor can properly reconstruct
the original tensor layout.
- Line 100: The wire format for remote task tensor metadata has changed from
40-byte to 128-byte structures, but the PROTOCOL_VERSION constant has not been
incremented in either the C++ or Python implementations, which will cause
mismatches between old and new endpoints. Locate the PROTOCOL_VERSION constant
in remote_wire.h and increment its value from 1 to 2, then locate the
PROTOCOL_VERSION constant in python/simpler/remote_l3_protocol.py and increment
it from 1 to 2 as well. This ensures both implementations signal the wire format
change and properly negotiate compatibility during the HELLO handshake.
In `@src/common/task_interface/task_args.h`:
- Around line 279-280: The read_blob function on lines 279-280 creates undefined
behavior by reinterpreting raw bytes from blob storage as const Tensor* pointers
without guaranteeing alignment, while Tensor is declared with alignas(64).
Instead of casting the raw buffer pointer to Tensor*, maintain the blob region
as byte-addressed and copy individual Tensor objects from the buffer into
properly aligned Tensor instances before accessing their fields. This ensures
that all field accesses to view.tensors[i] happen on correctly aligned objects.
In `@src/common/task_interface/tensor.h`:
- Around line 115-120: The comments in the Tensor class contain conflicting
information about default construction. The comments at lines 88-90 state that
users cannot default-construct Tensor, while the comments at lines 115-120 (near
the Tensor() = default declaration) explain that default construction is public
and required for various use cases. Update the earlier comment block (around
lines 88-90) to align with the actual API contract documented at the Tensor() =
default declaration, ensuring the comments accurately reflect that default
construction is publicly available and explain the rationale consistently in one
clear location.
---
Outside diff comments:
In `@src/common/hierarchical/remote_wire.cpp`:
- Around line 339-375: The encode_tensor function only serializes
contiguous-defining fields and discards stride and start_offset information,
while decode_tensor assumes row-major strides and zero start_offset when
reconstructing the tensor. This means non-contiguous tensors or tensors with
non-zero start_offset would be silently decoded incorrectly. Add an ensure check
in the encode_tensor function to validate that the input tensor is contiguous
and has a zero start_offset before encoding, preventing corruption of strided or
offset tensors that should not be serialized in this format.
- Around line 423-435: In the tensor sidecar encoding loop within the function
handling remote task arguments, when `sidecar.present` is false, add validation
to ensure that the corresponding tensor metadata does not have `child_memory` or
non-zero `tensor.nbytes()`, mirroring the invariant checks enforced in
`RemoteL3Endpoint::build_task_payload`. Add this validation in the else branch
(when the sidecar is not present) before processing continues, and apply the
same validation logic to the corresponding decode functions as well to prevent
malformed payloads from being accepted when called directly or by peer
endpoints.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: a3e6f484-4c1b-41cd-a55d-8823a393ba03
📒 Files selected for processing (97)
docs/orchestrator.mddocs/remote-l3-worker-design.mddocs/remote-l3-worker-design/buffers-and-transports.mddocs/remote-l3-worker-design/implementation-plan.mddocs/remote-l3-worker-design/protocol.mddocs/task-flow.mddocs/testing.mddocs/worker-manager.mdexamples/a2a3/tensormap_and_ringbuffer/async_notify_demo/test_async_notify_demo.pyexamples/a2a3/tensormap_and_ringbuffer/deferred_notify_demo/test_deferred_notify_demo.pyexamples/a2a3/tensormap_and_ringbuffer/sdma_async_completion_demo/test_sdma_async_completion_demo.pyexamples/a5/tensormap_and_ringbuffer/async_notify_demo/test_async_notify_demo.pyexamples/a5/tensormap_and_ringbuffer/bgemm/kernels/orchestration/bgemm_orch.cppexamples/a5/tensormap_and_ringbuffer/deferred_notify_demo/test_deferred_notify_demo.pyexamples/workers/l2/per_task_runtime_env/main.pyexamples/workers/l2/vector_add/README.mdexamples/workers/l2/vector_add/main.pyexamples/workers/l2/vector_add/test_run_timing.pyexamples/workers/l3/README.mdexamples/workers/l3/all_to_all_distributed/main.pyexamples/workers/l3/allgather_distributed/main.pyexamples/workers/l3/allreduce_distributed/kernels/aiv/allreduce_kernel.cppexamples/workers/l3/allreduce_distributed/main.pyexamples/workers/l3/allreduce_ring_distributed/kernels/aiv/allreduce_ring_kernel.cppexamples/workers/l3/allreduce_ring_distributed/main.pyexamples/workers/l3/broadcast_distributed/main.pyexamples/workers/l3/child_memory/README.mdexamples/workers/l3/child_memory/main.pyexamples/workers/l3/domain_rank_map/main.pyexamples/workers/l3/dual_domain_overlap/kernels/aiv/domain_allreduce_sum.cppexamples/workers/l3/dual_domain_overlap/main.pyexamples/workers/l3/ep_dispatch_combine/main.pyexamples/workers/l3/ffn_tp_parallel/main.pyexamples/workers/l3/reduce_scatter_distributed/main.pypython/bindings/CMakeLists.txtpython/bindings/task_interface.cpppython/bindings/worker_bind.hpython/simpler/orchestrator.pypython/simpler/remote_l3_protocol.pypython/simpler/task_interface.pypython/simpler/worker.pysimpler_setup/goldens/paged_attention.pysimpler_setup/torch_interop.pysrc/a2a3/docs/runtimes.mdsrc/a2a3/platform/include/common/platform_config.hsrc/a2a3/runtime/host_build_graph/runtime/tensor_info.hsrc/a2a3/runtime/tensormap_and_ringbuffer/aicpu/aicpu_executor.cppsrc/a2a3/runtime/tensormap_and_ringbuffer/host/dep_gen_replay.cppsrc/a2a3/runtime/tensormap_and_ringbuffer/host/runtime_maker.cppsrc/a2a3/runtime/tensormap_and_ringbuffer/orchestration/pto_orchestration_api.hsrc/a2a3/runtime/tensormap_and_ringbuffer/runtime/common.hsrc/a2a3/runtime/tensormap_and_ringbuffer/runtime/pto_dep_compute.hsrc/a2a3/runtime/tensormap_and_ringbuffer/runtime/pto_runtime2_types.hsrc/a2a3/runtime/tensormap_and_ringbuffer/runtime/pto_tensormap.hsrc/a2a3/runtime/tensormap_and_ringbuffer/runtime/pto_types.hsrc/a2a3/runtime/tensormap_and_ringbuffer/runtime/tensor_create_info.hsrc/a5/platform/include/common/platform_config.hsrc/a5/runtime/host_build_graph/runtime/tensor_info.hsrc/a5/runtime/tensormap_and_ringbuffer/aicpu/aicpu_executor.cppsrc/a5/runtime/tensormap_and_ringbuffer/host/dep_gen_replay.cppsrc/a5/runtime/tensormap_and_ringbuffer/host/runtime_maker.cppsrc/a5/runtime/tensormap_and_ringbuffer/orchestration/pto_orchestration_api.hsrc/a5/runtime/tensormap_and_ringbuffer/runtime/common.hsrc/a5/runtime/tensormap_and_ringbuffer/runtime/pto_dep_compute.hsrc/a5/runtime/tensormap_and_ringbuffer/runtime/pto_runtime2_types.hsrc/a5/runtime/tensormap_and_ringbuffer/runtime/pto_task_id.hsrc/a5/runtime/tensormap_and_ringbuffer/runtime/pto_tensormap.hsrc/a5/runtime/tensormap_and_ringbuffer/runtime/pto_types.hsrc/a5/runtime/tensormap_and_ringbuffer/runtime/tensor.hsrc/a5/runtime/tensormap_and_ringbuffer/runtime/tensor_create_info.hsrc/common/hierarchical/orchestrator.cppsrc/common/hierarchical/orchestrator.hsrc/common/hierarchical/remote_endpoint.cppsrc/common/hierarchical/remote_wire.cppsrc/common/hierarchical/remote_wire.hsrc/common/hierarchical/types.hsrc/common/hierarchical/worker_manager.cppsrc/common/hierarchical/worker_manager.hsrc/common/task_interface/arg_direction.hsrc/common/task_interface/assert_compat.cppsrc/common/task_interface/assert_compat.hsrc/common/task_interface/pto_task_id.hsrc/common/task_interface/task_args.hsrc/common/task_interface/tensor.hsrc/common/task_interface/tensor_arg.htests/ut/cpp/CMakeLists.txttests/ut/cpp/a2a3/test_tensormap.cpptests/ut/cpp/a5/test_tensormap.cpptests/ut/cpp/hierarchical/test_orchestrator.cpptests/ut/cpp/hierarchical/test_remote_endpoint.cpptests/ut/cpp/hierarchical/test_remote_wire.cpptests/ut/cpp/hierarchical/test_scheduler.cpptests/ut/cpp/types/test_child_memory.cpptests/ut/cpp/types/test_chip_max_tensor_args.cpptests/ut/py/test_task_interface.pytests/ut/py/test_worker/test_group_task.pytests/ut/py/test_worker/test_host_worker.py
💤 Files with no reviewable changes (5)
- src/a5/runtime/tensormap_and_ringbuffer/runtime/pto_dep_compute.h
- src/a5/runtime/tensormap_and_ringbuffer/runtime/pto_task_id.h
- src/a5/runtime/tensormap_and_ringbuffer/runtime/tensor.h
- src/a2a3/runtime/tensormap_and_ringbuffer/runtime/pto_dep_compute.h
- src/common/task_interface/tensor_arg.h
…B guards, doc fixes Review fixes (bot + manual) on top of the TaskArgs/Tensor unification: - remote_wire encode_tensor: assert is_contiguous && start_offset == 0 — the wire is contiguous-only (strides rebuilt row-major on decode); document in protocol.md. test_remote_wire builds metadata via make_tensor_external. - TensorCreateInfo ctor: guard ndims_in in [1, MAX_TENSOR_DIMS] before writing shapes[] (a2a3 + a5). - binding Tensor.ndims: read-only (rank changes go through the shapes setter). - make_tensor_arg: reject non-CPU tensors (device pointer needs child_memory). - remote_l3_protocol decode_tensor: reject ndims == 0 (match C++ decoder). - orchestrator.alloc: reject empty (rank-0) shape instead of returning an addr==0 orphan; drop the now-dead empty-shape guard. - docs: fix stale tensor_arg.h references (task-flow.md x2, developer-guide.md). - tensor.h: correct the construction-contract comment (default ctor public but uninitialized). tests/ut/cpp CMake: note the assert_compat ODR invariant. from_tensor_arg: document the external/contiguous precondition.
jvjhfhg
approved these changes
Jun 22, 2026
jvjhfhg
pushed a commit
that referenced
this pull request
Jun 22, 2026
) Fixes #1098 — a standards-level alignment UB introduced by #1093. `read_blob` returned a `TaskArgsView` whose `tensors` was a `const Tensor*` formed by `reinterpret_cast` over the **8-byte-aligned** mailbox blob, while `Tensor` is `alignas(64)`. Forming a pointer/reference to an under-aligned over-aligned object is UB even when no access faults. Before #1093 the blob element was the 40 B / align-8 `ContinuousTensor`, so `src + 8` was naturally aligned; the over-aligned `Tensor` made it UB. Safe in practice on aarch64 (every `Tensor` field is ≤8-byte aligned, the blob is 8-aligned, and consumers copy via trivially-copyable copy / `memcpy`), but this removes the UB at the root.
lwDavid
added a commit
to lwDavid/simpler
that referenced
this pull request
Jun 24, 2026
…TestCase Self-contained SceneTestCase port of pypto-lib decode_fwd_layers (N=2): a fused 2-layer Qwen3-14B decode chunk (hidden -> hidden, no LM head), harvested verbatim from pypto codegen (36 sources: orchestration + 35 incores, 8 AIC + 27 AIV) with a 2-layer torch golden. Parameter regime matches stress_profile.py: BATCH=16, MAX_SEQ=5500, decode seq_len=3500. Replaces the earlier single-layer example. The orchestration is migrated to main's TaskArgs ABI (hw-native-sys#1093/hw-native-sys#1104): entry takes const L2TaskArgs&, externals via orch_args.tensor(N).ref(), per-task args use L0TaskArgs. The incores are used unmodified. Loader fix (elf_parser) so the harvested kernels run unmodified. The fused-attention kernels read their AIV sub-block id via a [[block_local]] static (pypto_runtime_subblock_id) populated from the dispatch context, which emits a benign data relocation against .text. extract_text_section rejected ANY .rela.text entry, forcing a hw-native-sys#900 hand-edit (native get_subblockid()) that changed the sub-block id semantics and produced output NaN (simpler#1102). elf_parser now rejects only genuine branch relocations (R_AARCH64_CALL26/JUMP26 -- the BL/B-to-garbage case hw-native-sys#830/hw-native-sys#831) and lets benign non-branch relocations and unreferenced .text._Z* COMDAT duplicates through, matching pypto's loader. test_elf_parser.py pins the new contract. Verified deterministic PASS on a2a3 (out + both KV pools match the torch reference), matching pypto execute_compiled on the identical artifacts.
lwDavid
added a commit
to lwDavid/simpler
that referenced
this pull request
Jun 25, 2026
Self-contained SceneTestCase port of pypto-lib decode_fwd_layers (N=2): a fused 2-layer Qwen3-14B decode chunk (hidden -> hidden, no LM head), harvested verbatim from pypto codegen (36 sources: orchestration + 35 incores, 8 AIC + 27 AIV) with a 2-layer torch golden. Parameter regime matches stress_profile.py: BATCH=16, MAX_SEQ=5500, decode seq_len=3500. Replaces the earlier single-layer example. The orchestration is migrated to main's TaskArgs ABI (hw-native-sys#1093/hw-native-sys#1104): entry takes const L2TaskArgs&, externals via orch_args.tensor(N).ref(), per-task args use L0TaskArgs. The incores are used unmodified. The test is SKIPPED on device. fa_fused_aiv emits a [[block_local]] static (pypto_runtime_subblock_id) for the AIV sub-block id, whose benign non-branch .text relocation is rejected by simpler's strict .text-only loader. The loader is intentionally left unchanged -- the hw-native-sys#900/hw-native-sys#830/hw-native-sys#831 guard against unapplied branch relocations is NOT relaxed. Un-skipping requires removing the [[block_local]] dependency: pto-isa accepting an explicit sub_block_id (so the kernel passes get_sub_block_id(args)), or the runtime programming the FFTS sub-block register so native get_subblockid() returns 0/1 (today 0 for both AIVs, confirmed by tensor dump -- both lanes collide -> attention NaN). The math is correct on a loader that accepts the benign relocation. See README.md.
lwDavid
added a commit
to lwDavid/simpler
that referenced
this pull request
Jun 25, 2026
Self-contained SceneTestCase port of pypto-lib decode_fwd_layers (N=2): a fused 2-layer Qwen3-14B decode chunk (hidden -> hidden, no LM head), harvested verbatim from pypto codegen (36 sources: orchestration + 35 incores, 8 AIC + 27 AIV) with a 2-layer torch golden. Parameter regime matches stress_profile.py: BATCH=16, MAX_SEQ=5500, decode seq_len=3500. Replaces the earlier single-layer example. The orchestration is migrated to main's TaskArgs ABI (hw-native-sys#1093/hw-native-sys#1104): entry takes const L2TaskArgs&, externals via orch_args.tensor(N).ref(), per-task args use L0TaskArgs. The incores are used unmodified. The test is SKIPPED on device. fa_fused_aiv emits a [[block_local]] static (pypto_runtime_subblock_id) for the AIV sub-block id, whose benign non-branch .text relocation is rejected by simpler's strict .text-only loader. The loader is intentionally left unchanged -- the hw-native-sys#900/hw-native-sys#830/hw-native-sys#831 guard against unapplied branch relocations is NOT relaxed. Un-skipping requires removing the [[block_local]] dependency: pto-isa accepting an explicit sub_block_id (so the kernel passes get_sub_block_id(args)), or the runtime programming the FFTS sub-block register so native get_subblockid() returns 0/1 (today 0 for both AIVs, confirmed by tensor dump -- both lanes collide -> attention NaN). The math is correct on a loader that accepts the benign relocation. See README.md.
ChaoZheng109
pushed a commit
to lwDavid/simpler
that referenced
this pull request
Jun 25, 2026
Self-contained SceneTestCase port of pypto-lib decode_fwd_layers (N=2): a fused 2-layer Qwen3-14B decode chunk (hidden -> hidden, no LM head), harvested verbatim from pypto codegen (36 sources: orchestration + 35 incores, 8 AIC + 27 AIV) with a 2-layer torch golden. Parameter regime matches stress_profile.py: BATCH=16, MAX_SEQ=5500, decode seq_len=3500. Replaces the earlier single-layer example. The orchestration is migrated to main's TaskArgs ABI (hw-native-sys#1093/hw-native-sys#1104): entry takes const L2TaskArgs&, externals via orch_args.tensor(N).ref(), per-task args use L0TaskArgs. The incores are used unmodified. The test is SKIPPED on device. fa_fused_aiv emits a [[block_local]] static (pypto_runtime_subblock_id) for the AIV sub-block id, whose benign non-branch .text relocation is rejected by simpler's strict .text-only loader. The loader is intentionally left unchanged -- the hw-native-sys#900/hw-native-sys#830/hw-native-sys#831 guard against unapplied branch relocations is NOT relaxed. Un-skipping requires removing the [[block_local]] dependency: pto-isa accepting an explicit sub_block_id (so the kernel passes get_sub_block_id(args)), or the runtime programming the FFTS sub-block register so native get_subblockid() returns 0/1 (today 0 for both AIVs, confirmed by tensor dump -- both lanes collide -> attention NaN). The math is correct on a loader that accepts the benign relocation. See README.md.
ChaoZheng109
pushed a commit
that referenced
this pull request
Jun 25, 2026
…1088) Self-contained SceneTestCase port of pypto-lib decode_fwd_layers (N=2): a fused 2-layer Qwen3-14B decode chunk (hidden -> hidden, no LM head), harvested verbatim from pypto codegen (36 sources: orchestration + 35 incores, 8 AIC + 27 AIV) with a 2-layer torch golden. Parameter regime matches stress_profile.py: BATCH=16, MAX_SEQ=5500, decode seq_len=3500. Replaces the earlier single-layer example. The orchestration is migrated to main's TaskArgs ABI (#1093/#1104): entry takes const L2TaskArgs&, externals via orch_args.tensor(N).ref(), per-task args use L0TaskArgs. The incores are used unmodified. The test is SKIPPED on device. fa_fused_aiv emits a [[block_local]] static (pypto_runtime_subblock_id) for the AIV sub-block id, whose benign non-branch .text relocation is rejected by simpler's strict .text-only loader. The loader is intentionally left unchanged -- the #900/#830/#831 guard against unapplied branch relocations is NOT relaxed. Un-skipping requires removing the [[block_local]] dependency: pto-isa accepting an explicit sub_block_id (so the kernel passes get_sub_block_id(args)), or the runtime programming the FFTS sub-block register so native get_subblockid() returns 0/1 (today 0 for both AIVs, confirmed by tensor dump -- both lanes collide -> attention NaN). The math is correct on a loader that accepts the benign relocation. See README.md.
doraemonmj
pushed a commit
to doraemonmj/simpler_wc
that referenced
this pull request
Jul 1, 2026
…w-native-sys#1088) Self-contained SceneTestCase port of pypto-lib decode_fwd_layers (N=2): a fused 2-layer Qwen3-14B decode chunk (hidden -> hidden, no LM head), harvested verbatim from pypto codegen (36 sources: orchestration + 35 incores, 8 AIC + 27 AIV) with a 2-layer torch golden. Parameter regime matches stress_profile.py: BATCH=16, MAX_SEQ=5500, decode seq_len=3500. Replaces the earlier single-layer example. The orchestration is migrated to main's TaskArgs ABI (hw-native-sys#1093/hw-native-sys#1104): entry takes const L2TaskArgs&, externals via orch_args.tensor(N).ref(), per-task args use L0TaskArgs. The incores are used unmodified. The test is SKIPPED on device. fa_fused_aiv emits a [[block_local]] static (pypto_runtime_subblock_id) for the AIV sub-block id, whose benign non-branch .text relocation is rejected by simpler's strict .text-only loader. The loader is intentionally left unchanged -- the hw-native-sys#900/hw-native-sys#830/hw-native-sys#831 guard against unapplied branch relocations is NOT relaxed. Un-skipping requires removing the [[block_local]] dependency: pto-isa accepting an explicit sub_block_id (so the kernel passes get_sub_block_id(args)), or the runtime programming the FFTS sub-block register so native get_subblockid() returns 0/1 (today 0 for both AIVs, confirmed by tensor dump -- both lanes collide -> attention NaN). The math is correct on a loader that accepts the benign relocation. See README.md.
nalinaly
pushed a commit
to nalinaly/simpler
that referenced
this pull request
Jul 31, 2026
…-native-sys#1093) TaskArgs / wire / bindings previously carried the 40 B contiguous-only `ContinuousTensor`. They now carry the runtime's unified 128 B strided `Tensor`, so a single tensor type spans host construction, the wire, and the runtime. **Behavior is unchanged**: this construction path stays contiguous (row-major strides, `start_offset == 0`), enforced at the entry points. Strided views are now expressible end-to-end but `make_tensor_arg` does not yet produce them.
nalinaly
pushed a commit
to nalinaly/simpler
that referenced
this pull request
Jul 31, 2026
…sys#1098) (hw-native-sys#1101) Fixes hw-native-sys#1098 — a standards-level alignment UB introduced by hw-native-sys#1093. `read_blob` returned a `TaskArgsView` whose `tensors` was a `const Tensor*` formed by `reinterpret_cast` over the **8-byte-aligned** mailbox blob, while `Tensor` is `alignas(64)`. Forming a pointer/reference to an under-aligned over-aligned object is UB even when no access faults. Before hw-native-sys#1093 the blob element was the 40 B / align-8 `ContinuousTensor`, so `src + 8` was naturally aligned; the over-aligned `Tensor` made it UB. Safe in practice on aarch64 (every `Tensor` field is ≤8-byte aligned, the blob is 8-aligned, and consumers copy via trivially-copyable copy / `memcpy`), but this removes the UB at the root.
ChaoWao
added a commit
to ChaoWao/simpler-fork
that referenced
this pull request
Aug 24, 2026
…nsor An argument arrives at L2 as a `ChipTensor`, and the runtime then decides things about it: which task produced it, the version its OverlapMap keys on, whether dependency tracking is creator-only. Those decisions lived on `ChipTensor` itself, in `src/common/task_interface/`, so one type served the boundary and both runtimes' working state. `create_from_chip_args` recorded the mismatch as an assertion — `debug_assert(!t.manual_dep && t.version == 0)`, two fields that are meaningless on an argument. `docs/buffer-abi.md` recorded it too, calling `ChipTensor` "L2 leaf, internal" and saying "You never build a `ChipTensor`" while `ChipWorker.run` took a container of them and `nb::class_<ChipTensor>` exposed a `make()` factory. The fusion dates to hw-native-sys#1093, which needed the strided view hw-native-sys#808 had just given the runtime's tensor to reach the argument boundary, and got there by promoting the runtime-private header into `task_interface/` rather than adding strides to the 40 B `ContinuousTensor` it replaced. It cost 40 B → 128 B per wire tensor and a mailbox doubling for a capability that PR noted was not yet used: two months on, `make_tensor_arg` and `make_chip_tensor_arg` still reject non-contiguous tensors. hw-native-sys#1729 undid the fusion on the L3 wire and re-split `Tensor` from `ChipTensor`, with a cost argument that applies equally here; nobody undid it at L2. ## The two types `ChipTensor` (72 B) is a task argument: a resolved buffer and a strided view. struct ChipTensor { PTOBufferHandle buffer; uint64_t start_offset; uint32_t shapes[MAX_TENSOR_DIMS]; uint32_t strides[MAX_TENSOR_DIMS]; uint32_t ndims; DataType dtype; AddressSpace address_space; }; `simpler::hbg::Tensor` and `simpler::tmr::Tensor` (128 B) are the same geometry plus `owner_task_id`, `version`, `manual_dep` and the two caches derived from the geometry. One copy each, shared across architectures, placed the way `src/common/host_build_graph/graph_cache.h` already established. `simpler::hbg` was an existing namespace. `Runtime::set_orch_args` is the single place one becomes the other. Both runtimes call it only from `host/runtime_maker.cpp`, so adoption happens on the host before any orchestration runs — including for `tensormap_and_ringbuffer`, whose AICPU executor now reads an already-adopted `simpler::tmr::EntryArgsStorage` instead of converting per run. From there inward nothing holds the argument form: orchestration, the payload, the TensorMap and the kernels all name their runtime's type. A `tensor.h` in each runtime's `runtime/` directory sits first on that runtime's include path, so existing `#include "tensor.h"` lines resolve to both types without an include sweep. ## What the boundary type dropped `is_contiguous` and `extent_elem` become methods that compute from the geometry; a runtime that reads them per task caches them on its own `Tensor`. The view ops and the copy helpers that maintained those caches move with them — an argument is not a thing you take views of, and nothing outside a runtime called them. `init_external` and both factories lose their `manual_dep` / `version` parameters, which every caller passed as `false` / `0`. `PTO2TaskPayload` is no longer a friend, and the boundary header no longer includes `task_id.h`: it names no TaskId at all. `sizeof(ChipStorageTaskArgs)` falls from ~33.8 KB to 19464 B at the 256-tensor cap. `MAILBOX_SIZE` is unaffected — since hw-native-sys#1729 the frame is sized by the 144 B wire `Tensor`. ## Prerequisite: three structs, one byte layout `ChipTensor`, `PTO2TensorMapEntry` and `TensorCreateInfo` shared a byte-level layout so three copy paths could each be a single 64-byte `memcpy`, held by 21 hand-written `offsetof` assertions across four trees. Two of the three were shaped backwards to satisfy it: `TensorCreateInfo` carried `__pad0__` / `__pad2__` / `__pad_flags__` purely to occupy `ChipTensor::buffer` / `::owner_task_id` / `::address_space`, and the entry's `memcpy` wrote `ChipTensor::buffer.size` into a `PTO2TensorMapEntry *` field, its comment noting this was "harmless because `link_entry()` overwrites `next_in_bucket` immediately after". Each struct now assigns the fields it wants, by name. The optimization the `memcpy` existed for survives as intent: a canonically contiguous source has the derived pair recomputed from `shapes` rather than read across. `PTO2TensorMapEntry::copy_tensor_create_info` is deleted — it had no callers in any tree, and was the only reason `TensorCreateInfo` had to be punnable onto an entry. The entry keeps its own two-cache-line split and the assertions describing it; what goes is the claim that its bytes agree with a different struct's. ## Consequence: each case owns the sources it compiles A source that names one runtime's `Tensor` cannot be compiled under the other, and 57 scene-test sources were: a `host_build_graph` case naming the `tensormap_and_ringbuffer` case's kernels, and two `tensormap_and_ringbuffer` classes taking an hbg class's `CALLABLE` verbatim. That worked only because `ChipTensor` happened to be identical in both trees — the coupling this change removes. Each borrowing case now carries its own sources, 67 files, and names local paths. The two class-reuse cases rebase the inherited `CALLABLE` onto their own `kernels/` copy through a `_rebase_callable` helper, so the Python that drives them stays in one file while the C++ each runtime compiles is its own. The cost is real: `spmd_multiblock_mix`, `alternating_matmul_add`, `batch_paged_attention`, `paged_attention_unroll` and `benchmark_bgemm` now exist once per runtime and the copies can drift. A case that asserts something about `host_build_graph` should not depend on a file the other runtime owns. `examples/` orchestration and kernel sources lose their `Generated by PyPTO IR Compiler` marker: they are this repository's sources, and this change edits them. `tensor_create_info.h`'s doc paragraph said `host_build_graph` has no initial-value fill "unlike the tensormap_and_ringbuffer copy of this header", which hw-native-sys#1981 made false by removing that fill; both copies now describe what is there. Verification: full rebuild of both runtimes on both architectures, cpput 117/117, pyut 1908 passed / 14 skipped, a2a3sim 21 cases and a5sim 17 cases green. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
6 tasks
ChaoWao
added a commit
to ChaoWao/simpler-fork
that referenced
this pull request
Aug 24, 2026
…nsor An argument arrives at L2 as a `ChipTensor`, and the runtime then decides things about it: which task produced it, the version its OverlapMap keys on, whether dependency tracking is creator-only. Those decisions lived on `ChipTensor` itself, in `src/common/task_interface/`, so one type served the boundary and both runtimes' working state. `create_from_chip_args` recorded the mismatch as an assertion — `debug_assert(!t.manual_dep && t.version == 0)`, two fields that are meaningless on an argument. `docs/buffer-abi.md` recorded it too, calling `ChipTensor` "L2 leaf, internal" and saying "You never build a `ChipTensor`" while `ChipWorker.run` took a container of them and `nb::class_<ChipTensor>` exposed a `make()` factory. The fusion dates to hw-native-sys#1093, which needed the strided view hw-native-sys#808 had just given the runtime's tensor to reach the argument boundary, and got there by promoting the runtime-private header into `task_interface/` rather than adding strides to the 40 B `ContinuousTensor` it replaced. It cost 40 B → 128 B per wire tensor and a mailbox doubling for a capability that PR noted was not yet used: two months on, `make_tensor_arg` and `make_chip_tensor_arg` still reject non-contiguous tensors. hw-native-sys#1729 undid the fusion on the L3 wire and re-split `Tensor` from `ChipTensor`, with a cost argument that applies equally here; nobody undid it at L2. ## The two types `ChipTensor` (72 B) is a task argument: a resolved buffer and a strided view. struct ChipTensor { PTOBufferHandle buffer; uint64_t start_offset; uint32_t shapes[MAX_TENSOR_DIMS]; uint32_t strides[MAX_TENSOR_DIMS]; uint32_t ndims; DataType dtype; AddressSpace address_space; }; `simpler::hbg::Tensor` and `simpler::tmr::Tensor` (128 B) are the same geometry plus `owner_task_id`, `version`, `manual_dep` and the two caches derived from the geometry. One copy each, shared across architectures, placed the way `src/common/host_build_graph/graph_cache.h` already established. `simpler::hbg` was an existing namespace. `Runtime::set_orch_args` is the single place one becomes the other. Both runtimes call it only from `host/runtime_maker.cpp`, so adoption happens on the host before any orchestration runs — including for `tensormap_and_ringbuffer`, whose AICPU executor now reads an already-adopted `simpler::tmr::EntryArgsStorage` instead of converting per run. From there inward nothing holds the argument form: orchestration, the payload, the TensorMap and the kernels all name their runtime's type. A `tensor.h` in each runtime's `runtime/` directory sits first on that runtime's include path, so existing `#include "tensor.h"` lines resolve to both types without an include sweep. ## What the boundary type dropped `is_contiguous` and `extent_elem` become methods that compute from the geometry; a runtime that reads them per task caches them on its own `Tensor`. The view ops and the copy helpers that maintained those caches move with them — an argument is not a thing you take views of, and nothing outside a runtime called them. `init_external` and both factories lose their `manual_dep` / `version` parameters, which every caller passed as `false` / `0`. `PTO2TaskPayload` is no longer a friend, and the boundary header no longer includes `task_id.h`: it names no TaskId at all. `sizeof(ChipStorageTaskArgs)` falls from ~33.8 KB to 19464 B at the 256-tensor cap. `MAILBOX_SIZE` is unaffected — since hw-native-sys#1729 the frame is sized by the 144 B wire `Tensor`. ## Prerequisite: three structs, one byte layout `ChipTensor`, `PTO2TensorMapEntry` and `TensorCreateInfo` shared a byte-level layout so three copy paths could each be a single 64-byte `memcpy`, held by 21 hand-written `offsetof` assertions across four trees. Two of the three were shaped backwards to satisfy it: `TensorCreateInfo` carried `__pad0__` / `__pad2__` / `__pad_flags__` purely to occupy `ChipTensor::buffer` / `::owner_task_id` / `::address_space`, and the entry's `memcpy` wrote `ChipTensor::buffer.size` into a `PTO2TensorMapEntry *` field, its comment noting this was "harmless because `link_entry()` overwrites `next_in_bucket` immediately after". Each struct now assigns the fields it wants, by name. The optimization the `memcpy` existed for survives as intent: a canonically contiguous source has the derived pair recomputed from `shapes` rather than read across. `PTO2TensorMapEntry::copy_tensor_create_info` is deleted — it had no callers in any tree, and was the only reason `TensorCreateInfo` had to be punnable onto an entry. The entry keeps its own two-cache-line split and the assertions describing it; what goes is the claim that its bytes agree with a different struct's. ## Consequence: each case owns the sources it compiles A source that names one runtime's `Tensor` cannot be compiled under the other, and 57 scene-test sources were: a `host_build_graph` case naming the `tensormap_and_ringbuffer` case's kernels, and two `tensormap_and_ringbuffer` classes taking an hbg class's `CALLABLE` verbatim. That worked only because `ChipTensor` happened to be identical in both trees — the coupling this change removes. Each borrowing case now carries its own sources, 67 files, and names local paths. The two class-reuse cases rebase the inherited `CALLABLE` onto their own `kernels/` copy through a `_rebase_callable` helper, so the Python that drives them stays in one file while the C++ each runtime compiles is its own. The cost is real: `spmd_multiblock_mix`, `alternating_matmul_add`, `batch_paged_attention`, `paged_attention_unroll` and `benchmark_bgemm` now exist once per runtime and the copies can drift. A case that asserts something about `host_build_graph` should not depend on a file the other runtime owns. `examples/` orchestration and kernel sources lose their `Generated by PyPTO IR Compiler` marker: they are this repository's sources, and this change edits them. `tensor_create_info.h`'s doc paragraph said `host_build_graph` has no initial-value fill "unlike the tensormap_and_ringbuffer copy of this header", which hw-native-sys#1981 made false by removing that fill; both copies now describe what is there. Verification: full rebuild of both runtimes on both architectures, cpput 117/117, pyut 1908 passed / 14 skipped, a2a3sim 21 cases and a5sim 17 cases green. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ChaoWao
added a commit
to ChaoWao/simpler-fork
that referenced
this pull request
Aug 24, 2026
…nsor An argument arrives at L2 as a `ChipTensor`, and the runtime then decides things about it: which task produced it, the version its OverlapMap keys on, whether dependency tracking is creator-only. Those decisions lived on `ChipTensor` itself, in `src/common/task_interface/`, so one type served the boundary and both runtimes' working state. `create_from_chip_args` recorded the mismatch as an assertion — `debug_assert(!t.manual_dep && t.version == 0)`, two fields that are meaningless on an argument. `docs/buffer-abi.md` recorded it too, calling `ChipTensor` "L2 leaf, internal" and saying "You never build a `ChipTensor`" while `ChipWorker.run` took a container of them and `nb::class_<ChipTensor>` exposed a `make()` factory. The fusion dates to hw-native-sys#1093, which needed the strided view hw-native-sys#808 had just given the runtime's tensor to reach the argument boundary, and got there by promoting the runtime-private header into `task_interface/` rather than adding strides to the 40 B `ContinuousTensor` it replaced. It cost 40 B → 128 B per wire tensor and a mailbox doubling for a capability that PR noted was not yet used: two months on, `make_tensor_arg` and `make_chip_tensor_arg` still reject non-contiguous tensors. hw-native-sys#1729 undid the fusion on the L3 wire and re-split `Tensor` from `ChipTensor`, with a cost argument that applies equally here; nobody undid it at L2. ## The two types `ChipTensor` (72 B) is a task argument: a resolved buffer and a strided view. struct ChipTensor { PTOBufferHandle buffer; uint64_t start_offset; uint32_t shapes[MAX_TENSOR_DIMS]; uint32_t strides[MAX_TENSOR_DIMS]; uint32_t ndims; DataType dtype; AddressSpace address_space; }; `simpler::hbg::Tensor` and `simpler::tmr::Tensor` (128 B) are the same geometry plus `owner_task_id`, `version`, `manual_dep` and the two caches derived from the geometry. One copy each, shared across architectures, placed the way `src/common/host_build_graph/graph_cache.h` already established. `simpler::hbg` was an existing namespace. `Runtime::set_orch_args` is the single place one becomes the other. Both runtimes call it only from `host/runtime_maker.cpp`, so adoption happens on the host before any orchestration runs — including for `tensormap_and_ringbuffer`, whose AICPU executor now reads an already-adopted `simpler::tmr::EntryArgsStorage` instead of converting per run. From there inward nothing holds the argument form: orchestration, the payload, the TensorMap and the kernels all name their runtime's type. A `tensor.h` in each runtime's `runtime/` directory sits first on that runtime's include path, so existing `#include "tensor.h"` lines resolve to both types without an include sweep. ## What the boundary type dropped `is_contiguous` and `extent_elem` become methods that compute from the geometry; a runtime that reads them per task caches them on its own `Tensor`. The view ops and the copy helpers that maintained those caches move with them — an argument is not a thing you take views of, and nothing outside a runtime called them. `init_external` and both factories lose their `manual_dep` / `version` parameters, which every caller passed as `false` / `0`. `PTO2TaskPayload` is no longer a friend, and the boundary header no longer includes `task_id.h`: it names no TaskId at all. `sizeof(ChipStorageTaskArgs)` falls from ~33.8 KB to 19464 B at the 256-tensor cap. `MAILBOX_SIZE` is unaffected — since hw-native-sys#1729 the frame is sized by the 144 B wire `Tensor`. ## Prerequisite: three structs, one byte layout `ChipTensor`, `PTO2TensorMapEntry` and `TensorCreateInfo` shared a byte-level layout so three copy paths could each be a single 64-byte `memcpy`, held by 21 hand-written `offsetof` assertions across four trees. Two of the three were shaped backwards to satisfy it: `TensorCreateInfo` carried `__pad0__` / `__pad2__` / `__pad_flags__` purely to occupy `ChipTensor::buffer` / `::owner_task_id` / `::address_space`, and the entry's `memcpy` wrote `ChipTensor::buffer.size` into a `PTO2TensorMapEntry *` field, its comment noting this was "harmless because `link_entry()` overwrites `next_in_bucket` immediately after". Each struct now assigns the fields it wants, by name. The optimization the `memcpy` existed for survives as intent: a canonically contiguous source has the derived pair recomputed from `shapes` rather than read across. `PTO2TensorMapEntry::copy_tensor_create_info` is deleted — it had no callers in any tree, and was the only reason `TensorCreateInfo` had to be punnable onto an entry. The entry keeps its own two-cache-line split and the assertions describing it; what goes is the claim that its bytes agree with a different struct's. ## Consequence: each case owns the sources it compiles A source that names one runtime's `Tensor` cannot be compiled under the other, and 57 scene-test sources were: a `host_build_graph` case naming the `tensormap_and_ringbuffer` case's kernels, and two `tensormap_and_ringbuffer` classes taking an hbg class's `CALLABLE` verbatim. That worked only because `ChipTensor` happened to be identical in both trees — the coupling this change removes. Each borrowing case now carries its own sources, 67 files, and names local paths. The two class-reuse cases rebase the inherited `CALLABLE` onto their own `kernels/` copy through a `_rebase_callable` helper, so the Python that drives them stays in one file while the C++ each runtime compiles is its own. The cost is real: `spmd_multiblock_mix`, `alternating_matmul_add`, `batch_paged_attention`, `paged_attention_unroll` and `benchmark_bgemm` now exist once per runtime and the copies can drift. A case that asserts something about `host_build_graph` should not depend on a file the other runtime owns. `examples/` orchestration and kernel sources lose their `Generated by PyPTO IR Compiler` marker: they are this repository's sources, and this change edits them. `tensor_create_info.h`'s doc paragraph said `host_build_graph` has no initial-value fill "unlike the tensormap_and_ringbuffer copy of this header", which hw-native-sys#1981 made false by removing that fill; both copies now describe what is there. The 67 copies were taken from sources whose spellings `main` has since retired, so they carry the current ones: `PTO2OrchestrationConfig` / `PTO2Runtime` / `PTO2RuntimeArenaLayout` / `PTO2RuntimeMode` / `PTO2RuntimeOps` / `PTO2ScopeGuard` / `PTO2TaskSlotState` lose the prefix, and `PTO2_SCOPE` / `PTO2_SCOPE_GUARD` become `SIMPLER_SCOPE` / `SIMPLER_SCOPE_GUARD`. Verification: full rebuild of both runtimes on both architectures, cpput 117/117, pyut 1908 passed / 14 skipped, a2a3sim 21 cases and a5sim 17 cases green. The sim sweeps build only the cases they run, so every orchestration source was also syntax-checked against its runtime's include path — 129 (source, runtime, arch) compiles, which is what caught a copy under host_build_graph/ still calling simpler::tmr::make_tensor_external. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ChaoWao
added a commit
to ChaoWao/simpler-fork
that referenced
this pull request
Aug 24, 2026
…nsor An argument arrives at L2 as a `ChipTensor`, and the runtime then decides things about it: which task produced it, the version its OverlapMap keys on, whether dependency tracking is creator-only. Those decisions lived on `ChipTensor` itself, in `src/common/task_interface/`, so one type served the boundary and both runtimes' working state. `create_from_chip_args` recorded the mismatch as an assertion — `debug_assert(!t.manual_dep && t.version == 0)`, two fields that are meaningless on an argument. `docs/buffer-abi.md` recorded it too, calling `ChipTensor` "L2 leaf, internal" and saying "You never build a `ChipTensor`" while `ChipWorker.run` took a container of them and `nb::class_<ChipTensor>` exposed a `make()` factory. The fusion dates to hw-native-sys#1093, which needed the strided view hw-native-sys#808 had just given the runtime's tensor to reach the argument boundary, and got there by promoting the runtime-private header into `task_interface/` rather than adding strides to the 40 B `ContinuousTensor` it replaced. It cost 40 B → 128 B per wire tensor and a mailbox doubling for a capability that PR noted was not yet used: two months on, `make_tensor_arg` and `make_chip_tensor_arg` still reject non-contiguous tensors. hw-native-sys#1729 undid the fusion on the L3 wire and re-split `Tensor` from `ChipTensor`, with a cost argument that applies equally here; nobody undid it at L2. ## The two types `ChipTensor` (72 B) is a task argument: a resolved buffer and a strided view. struct ChipTensor { PTOBufferHandle buffer; uint64_t start_offset; uint32_t shapes[MAX_TENSOR_DIMS]; uint32_t strides[MAX_TENSOR_DIMS]; uint32_t ndims; DataType dtype; AddressSpace address_space; }; `simpler::hbg::Tensor` and `simpler::tmr::Tensor` (128 B) are the same geometry plus `owner_task_id`, `version`, `manual_dep` and the two caches derived from the geometry. One copy each, shared across architectures, placed the way `src/common/host_build_graph/graph_cache.h` already established. `simpler::hbg` was an existing namespace. `Runtime::set_orch_args` is the single place one becomes the other. Both runtimes call it only from `host/runtime_maker.cpp`, so adoption happens on the host before any orchestration runs — including for `tensormap_and_ringbuffer`, whose AICPU executor now reads an already-adopted `simpler::tmr::EntryArgsStorage` instead of converting per run. From there inward nothing holds the argument form: orchestration, the payload, the TensorMap and the kernels all name their runtime's type. A `tensor.h` in each runtime's `runtime/` directory sits first on that runtime's include path, so existing `#include "tensor.h"` lines resolve to both types without an include sweep. That shim names the two headers by their path under `src/common`, which every platform target but `aicore` already had on its include path; the four `aicore` CMakeLists gain the line their `aicpu` and `host` siblings carry. ## What the boundary type dropped `is_contiguous` and `extent_elem` become methods that compute from the geometry; a runtime that reads them per task caches them on its own `Tensor`. The view ops and the copy helpers that maintained those caches move with them — an argument is not a thing you take views of, and nothing outside a runtime called them. `init_external` and both factories lose their `manual_dep` / `version` parameters, which every caller passed as `false` / `0`. `PTO2TaskPayload` is no longer a friend, and the boundary header no longer includes `task_id.h`: it names no TaskId at all. `sizeof(ChipStorageTaskArgs)` falls from ~33.8 KB to 19464 B at the 256-tensor cap. `MAILBOX_SIZE` is unaffected — since hw-native-sys#1729 the frame is sized by the 144 B wire `Tensor`. ## Prerequisite: three structs, one byte layout `ChipTensor`, `PTO2TensorMapEntry` and `TensorCreateInfo` shared a byte-level layout so three copy paths could each be a single 64-byte `memcpy`, held by 21 hand-written `offsetof` assertions across four trees. Two of the three were shaped backwards to satisfy it: `TensorCreateInfo` carried `__pad0__` / `__pad2__` / `__pad_flags__` purely to occupy `ChipTensor::buffer` / `::owner_task_id` / `::address_space`, and the entry's `memcpy` wrote `ChipTensor::buffer.size` into a `PTO2TensorMapEntry *` field, its comment noting this was "harmless because `link_entry()` overwrites `next_in_bucket` immediately after". Each struct now assigns the fields it wants, by name. The optimization the `memcpy` existed for survives as intent: a canonically contiguous source has the derived pair recomputed from `shapes` rather than read across. `PTO2TensorMapEntry::copy_tensor_create_info` is deleted — it had no callers in any tree, and was the only reason `TensorCreateInfo` had to be punnable onto an entry. The entry keeps its own two-cache-line split and the assertions describing it; what goes is the claim that its bytes agree with a different struct's. ## Consequence: each case owns the sources it compiles A source that names one runtime's `Tensor` cannot be compiled under the other, and 57 scene-test sources were: a `host_build_graph` case naming the `tensormap_and_ringbuffer` case's kernels, and two `tensormap_and_ringbuffer` classes taking an hbg class's `CALLABLE` verbatim. That worked only because `ChipTensor` happened to be identical in both trees — the coupling this change removes. Each borrowing case now carries its own sources, 67 files, and names local paths. The two class-reuse cases rebase the inherited `CALLABLE` onto their own `kernels/` copy through a `_rebase_callable` helper, so the Python that drives them stays in one file while the C++ each runtime compiles is its own. The cost is real: `spmd_multiblock_mix`, `alternating_matmul_add`, `batch_paged_attention`, `paged_attention_unroll` and `benchmark_bgemm` now exist once per runtime and the copies can drift. A case that asserts something about `host_build_graph` should not depend on a file the other runtime owns. `examples/` orchestration and kernel sources lose their `Generated by PyPTO IR Compiler` marker: they are this repository's sources, and this change edits them. `tensor_create_info.h`'s doc paragraph said `host_build_graph` has no initial-value fill "unlike the tensormap_and_ringbuffer copy of this header", which hw-native-sys#1981 made false by removing that fill; both copies now describe what is there. The 67 copies were taken from sources whose spellings `main` has since retired, so they carry the current ones: `PTO2OrchestrationConfig` / `PTO2Runtime` / `PTO2RuntimeArenaLayout` / `PTO2RuntimeMode` / `PTO2RuntimeOps` / `PTO2ScopeGuard` / `PTO2TaskSlotState` lose the prefix, and `PTO2_SCOPE` / `PTO2_SCOPE_GUARD` become `SIMPLER_SCOPE` / `SIMPLER_SCOPE_GUARD`. Verification: full rebuild of both runtimes on both architectures, cpput 117/117, pyut 1908 passed / 14 skipped, a2a3sim 21 cases and a5sim 17 cases green. The sim sweeps build only the cases they run, so every orchestration source was also syntax-checked against its runtime's include path — 129 (source, runtime, arch) compiles, which is what caught a copy under host_build_graph/ still calling simpler::tmr::make_tensor_external. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ChaoWao
added a commit
to ChaoWao/simpler-fork
that referenced
this pull request
Aug 25, 2026
…nsor An argument arrives at L2 as a `ChipTensor`, and the runtime then decides things about it: which task produced it, the version its OverlapMap keys on, whether dependency tracking is creator-only. Those decisions lived on `ChipTensor` itself, in `src/common/task_interface/`, so one type served the boundary and both runtimes' working state. `create_from_chip_args` recorded the mismatch as an assertion — `debug_assert(!t.manual_dep && t.version == 0)`, two fields that are meaningless on an argument. `docs/buffer-abi.md` recorded it too, calling `ChipTensor` "L2 leaf, internal" and saying "You never build a `ChipTensor`" while `ChipWorker.run` took a container of them and `nb::class_<ChipTensor>` exposed a `make()` factory. The fusion dates to hw-native-sys#1093, which needed the strided view hw-native-sys#808 had just given the runtime's tensor to reach the argument boundary, and got there by promoting the runtime-private header into `task_interface/` rather than adding strides to the 40 B `ContinuousTensor` it replaced. It cost 40 B → 128 B per wire tensor and a mailbox doubling for a capability that PR noted was not yet used: two months on, `make_tensor_arg` and `make_chip_tensor_arg` still reject non-contiguous tensors. with a cost argument that applies equally here; nobody undid it at L2. `ChipTensor` (72 B) is a task argument: a resolved buffer and a strided view. struct ChipTensor { PTOBufferHandle buffer; uint64_t start_offset; uint32_t shapes[MAX_TENSOR_DIMS]; uint32_t strides[MAX_TENSOR_DIMS]; uint32_t ndims; DataType dtype; AddressSpace address_space; }; `simpler::hbg::Tensor` and `simpler::tmr::Tensor` (128 B) are the same geometry plus `owner_task_id`, `version`, `manual_dep` and the two caches derived from the geometry. One copy each, shared across architectures, placed the way `src/common/host_build_graph/graph_cache.h` already established. `simpler::hbg` was an existing namespace. `Runtime::set_orch_args` is the single place one becomes the other. Both runtimes call it only from `host/runtime_maker.cpp`, so adoption happens on the host before any orchestration runs — including for `tensormap_and_ringbuffer`, whose AICPU executor now reads an already-adopted `simpler::tmr::EntryArgsStorage` instead of converting per run. From there inward nothing holds the argument form: orchestration, the payload, the TensorMap and the kernels all name their runtime's type. A `tensor.h` in each runtime's `runtime/` directory sits first on that runtime's include path, so existing `#include "tensor.h"` lines resolve to both types without an include sweep. That shim names the two headers by their path under `src/common`, which every platform target but `aicore` already had on its include path; the four `aicore` CMakeLists gain the line their `aicpu` and `host` siblings carry. `is_contiguous` and `extent_elem` become methods that compute from the geometry; a runtime that reads them per task caches them on its own `Tensor`. The view ops and the copy helpers that maintained those caches move with them — an argument is not a thing you take views of, and nothing outside a runtime called them. `init_external` and both factories lose their `manual_dep` / `version` parameters, which every caller passed as `false` / `0`. `PTO2TaskPayload` is no longer a friend, and the boundary header no longer includes `task_id.h`: it names no TaskId at all. `sizeof(ChipStorageTaskArgs)` falls from ~33.8 KB to 19464 B at the 256-tensor cap. `MAILBOX_SIZE` is unaffected — since hw-native-sys#1729 the frame is sized by the 144 B wire `Tensor`. `ChipTensor`, `PTO2TensorMapEntry` and `TensorCreateInfo` shared a byte-level layout so three copy paths could each be a single 64-byte `memcpy`, held by 21 hand-written `offsetof` assertions across four trees. Two of the three were shaped backwards to satisfy it: `TensorCreateInfo` carried `__pad0__` / `__pad2__` / `__pad_flags__` purely to occupy `ChipTensor::buffer` / `::owner_task_id` / `::address_space`, and the entry's `memcpy` wrote `ChipTensor::buffer.size` into a `PTO2TensorMapEntry *` field, its comment noting this was "harmless because `link_entry()` overwrites `next_in_bucket` immediately after". Each struct now assigns the fields it wants, by name. The optimization the `memcpy` existed for survives as intent: a canonically contiguous source has the derived pair recomputed from `shapes` rather than read across. `PTO2TensorMapEntry::copy_tensor_create_info` is deleted — it had no callers in any tree, and was the only reason `TensorCreateInfo` had to be punnable onto an entry. The entry keeps its own two-cache-line split and the assertions describing it; what goes is the claim that its bytes agree with a different struct's. A source that names one runtime's `Tensor` cannot be compiled under the other, and 57 scene-test sources were: a `host_build_graph` case naming the `tensormap_and_ringbuffer` case's kernels, and two `tensormap_and_ringbuffer` classes taking an hbg class's `CALLABLE` verbatim. That worked only because `ChipTensor` happened to be identical in both trees — the coupling this change removes. Each borrowing case now carries its own sources, 73 files, and names local paths. The two class-reuse cases rebase the inherited `CALLABLE` onto their own `kernels/` copy through a `_rebase_callable` helper, so the Python that drives them stays in one file while the C++ each runtime compiles is its own. `task_timing_slots` carries the runtime in a function default rather than a `@scene_test` decorator, so its sources move under `kernels/<runtime>/` and the helper builds the path from the runtime it is driving. The cost is real: `spmd_multiblock_mix`, `alternating_matmul_add`, `batch_paged_attention`, `paged_attention_unroll`, `benchmark_bgemm` and the task-timing kernels now exist once per runtime and the copies can drift. A case that asserts something about `host_build_graph` should not depend on a file the other runtime owns. `examples/` orchestration and kernel sources lose their `Generated by PyPTO IR Compiler` marker: they are this repository's sources, and this change edits them. `tensor_create_info.h`'s doc paragraph said `host_build_graph` has no initial-value fill "unlike the tensormap_and_ringbuffer copy of this header", which hw-native-sys#1981 made false by removing that fill; both copies now describe what is there. The 67 copies were taken from sources whose spellings `main` has since retired, so they carry the current ones: `PTO2OrchestrationConfig` / `PTO2Runtime` / `PTO2RuntimeArenaLayout` / `PTO2RuntimeMode` / `PTO2RuntimeOps` / `PTO2ScopeGuard` / `PTO2TaskSlotState` lose the prefix, and `PTO2_SCOPE` / `PTO2_SCOPE_GUARD` become `SIMPLER_SCOPE` / `SIMPLER_SCOPE_GUARD`. `test_hbg_sm_compaction` compared `GraphTensor` against `ChipTensor` to justify packing Graph boundaries into the payload's tensor slots, and read that pool as `ChipTensor`. The pool is the payload's, so both now name the runtime's `Tensor` — the assertion was true only while one type served both roles. Verification: full rebuild of both runtimes on both architectures, cpput 119/119, pyut 1908 passed / 14 skipped, a2a3sim 21 cases and a5sim 17 cases green. The sim sweeps build only the cases they run, so every orchestration source was also syntax-checked against its runtime's include path — 129 (source, runtime, arch) compiles, which is what caught a copy under host_build_graph/ still calling simpler::tmr::make_tensor_external. A second check reads which runtime *strings* a Python driver mentions rather than its @scene_test decorator; that is what caught task_timing_slots. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ChaoWao
added a commit
to ChaoWao/simpler-fork
that referenced
this pull request
Aug 25, 2026
…nsor An argument arrives at L2 as a `ChipTensor`, and the runtime then decides things about it: which task produced it, the version its OverlapMap keys on, whether dependency tracking is creator-only. Those decisions lived on `ChipTensor` itself, in `src/common/task_interface/`, so one type served the boundary and both runtimes' working state. `create_from_chip_args` recorded the mismatch as an assertion — `debug_assert(!t.manual_dep && t.version == 0)`, two fields that are meaningless on an argument. `docs/buffer-abi.md` recorded it too, calling `ChipTensor` "L2 leaf, internal" and saying "You never build a `ChipTensor`" while `ChipWorker.run` took a container of them and `nb::class_<ChipTensor>` exposed a `make()` factory. The fusion dates to hw-native-sys#1093, which needed the strided view hw-native-sys#808 had just given the runtime's tensor to reach the argument boundary, and got there by promoting the runtime-private header into `task_interface/` rather than adding strides to the 40 B `ContinuousTensor` it replaced. It cost 40 B → 128 B per wire tensor and a mailbox doubling for a capability that PR noted was not yet used: two months on, `make_tensor_arg` and `make_chip_tensor_arg` still reject non-contiguous tensors. with a cost argument that applies equally here; nobody undid it at L2. `ChipTensor` (72 B) is a task argument: a resolved buffer and a strided view. struct ChipTensor { PTOBufferHandle buffer; uint64_t start_offset; uint32_t shapes[MAX_TENSOR_DIMS]; uint32_t strides[MAX_TENSOR_DIMS]; uint32_t ndims; DataType dtype; AddressSpace address_space; }; `simpler::hbg::Tensor` and `simpler::tmr::Tensor` (128 B) are the same geometry plus `owner_task_id`, `version`, `manual_dep` and the two caches derived from the geometry. One copy each, shared across architectures, placed the way `src/common/host_build_graph/graph_cache.h` already established. `simpler::hbg` was an existing namespace. `Runtime::set_orch_args` is the single place one becomes the other. Both runtimes call it only from `host/runtime_maker.cpp`, so adoption happens on the host before any orchestration runs — including for `tensormap_and_ringbuffer`, whose AICPU executor now reads an already-adopted `simpler::tmr::EntryArgsStorage` instead of converting per run. From there inward nothing holds the argument form: orchestration, the payload, the TensorMap and the kernels all name their runtime's type. A `tensor.h` in each runtime's `runtime/` directory sits first on that runtime's include path, so existing `#include "tensor.h"` lines resolve to both types without an include sweep. That shim names the two headers by their path under `src/common`, which every platform target but `aicore` already had on its include path; the four `aicore` CMakeLists gain the line their `aicpu` and `host` siblings carry. `is_contiguous` and `extent_elem` become methods that compute from the geometry; a runtime that reads them per task caches them on its own `Tensor`. The view ops and the copy helpers that maintained those caches move with them — an argument is not a thing you take views of, and nothing outside a runtime called them. `init_external` and both factories lose their `manual_dep` / `version` parameters, which every caller passed as `false` / `0`. `PTO2TaskPayload` is no longer a friend, and the boundary header no longer includes `task_id.h`: it names no TaskId at all. `sizeof(ChipStorageTaskArgs)` falls from ~33.8 KB to 19464 B at the 256-tensor cap. `MAILBOX_SIZE` is unaffected — since hw-native-sys#1729 the frame is sized by the 144 B wire `Tensor`. `ChipTensor`, `PTO2TensorMapEntry` and `TensorCreateInfo` shared a byte-level layout so three copy paths could each be a single 64-byte `memcpy`, held by 21 hand-written `offsetof` assertions across four trees. Two of the three were shaped backwards to satisfy it: `TensorCreateInfo` carried `__pad0__` / `__pad2__` / `__pad_flags__` purely to occupy `ChipTensor::buffer` / `::owner_task_id` / `::address_space`, and the entry's `memcpy` wrote `ChipTensor::buffer.size` into a `PTO2TensorMapEntry *` field, its comment noting this was "harmless because `link_entry()` overwrites `next_in_bucket` immediately after". Each struct now assigns the fields it wants, by name. The optimization the `memcpy` existed for survives as intent: a canonically contiguous source has the derived pair recomputed from `shapes` rather than read across. `PTO2TensorMapEntry::copy_tensor_create_info` is deleted — it had no callers in any tree, and was the only reason `TensorCreateInfo` had to be punnable onto an entry. The entry keeps its own two-cache-line split and the assertions describing it; what goes is the claim that its bytes agree with a different struct's. A source that names one runtime's `Tensor` cannot be compiled under the other, and 57 scene-test sources were: a `host_build_graph` case naming the `tensormap_and_ringbuffer` case's kernels, and two `tensormap_and_ringbuffer` classes taking an hbg class's `CALLABLE` verbatim. That worked only because `ChipTensor` happened to be identical in both trees — the coupling this change removes. Each borrowing case now carries its own sources, 73 files, and names local paths. The two class-reuse cases rebase the inherited `CALLABLE` onto their own `kernels/` copy through a `_rebase_callable` helper, so the Python that drives them stays in one file while the C++ each runtime compiles is its own. `task_timing_slots` carries the runtime in a function default rather than a `@scene_test` decorator, so its sources move under `kernels/<runtime>/` and the helper builds the path from the runtime it is driving. The cost is real: `spmd_multiblock_mix`, `alternating_matmul_add`, `batch_paged_attention`, `paged_attention_unroll`, `benchmark_bgemm` and the task-timing kernels now exist once per runtime and the copies can drift. A case that asserts something about `host_build_graph` should not depend on a file the other runtime owns. `examples/` orchestration and kernel sources lose their `Generated by PyPTO IR Compiler` marker: they are this repository's sources, and this change edits them. `tensor_create_info.h`'s doc paragraph said `host_build_graph` has no initial-value fill "unlike the tensormap_and_ringbuffer copy of this header", which hw-native-sys#1981 made false by removing that fill; both copies now describe what is there. The copies were taken from sources whose spellings `main` has since retired — hw-native-sys#1980 finished that retirement repo-wide — so they carry the current ones. `test_hbg_sm_compaction` compared `GraphTensor` against `ChipTensor` to justify packing Graph boundaries into the payload's tensor slots, and read that pool as `ChipTensor`. The pool is the payload's, so both now name the runtime's `Tensor` — the assertion was true only while one type served both roles. Verification: full rebuild of both runtimes on both architectures, cpput 119/119, pyut 1908 passed / 14 skipped, a2a3sim 21 cases and a5sim 17 cases green. The sim sweeps build only the cases they run, so every orchestration source was also syntax-checked against its runtime's include path — 129 (source, runtime, arch) compiles, which is what caught a copy under host_build_graph/ still calling simpler::tmr::make_tensor_external. A second check reads which runtime *strings* a Python driver mentions rather than its @scene_test decorator; that is what caught task_timing_slots. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ChaoWao
added a commit
to ChaoWao/simpler-fork
that referenced
this pull request
Aug 25, 2026
…nsor An argument arrives at L2 as a `ChipTensor`, and the runtime then decides things about it: which task produced it, the version its OverlapMap keys on, whether dependency tracking is creator-only. Those decisions lived on `ChipTensor` itself, in `src/common/task_interface/`, so one type served the boundary and both runtimes' working state. `create_from_chip_args` recorded the mismatch as an assertion — `debug_assert(!t.manual_dep && t.version == 0)`, two fields that are meaningless on an argument. `docs/buffer-abi.md` recorded it too, calling `ChipTensor` "L2 leaf, internal" and saying "You never build a `ChipTensor`" while `ChipWorker.run` took a container of them and `nb::class_<ChipTensor>` exposed a `make()` factory. The fusion dates to hw-native-sys#1093, which needed the strided view hw-native-sys#808 had just given the runtime's tensor to reach the argument boundary, and got there by promoting the runtime-private header into `task_interface/` rather than adding strides to the 40 B `ContinuousTensor` it replaced. It cost 40 B → 128 B per wire tensor and a mailbox doubling for a capability that PR noted was not yet used: two months on, `make_tensor_arg` and `make_chip_tensor_arg` still reject non-contiguous tensors. with a cost argument that applies equally here; nobody undid it at L2. `ChipTensor` (72 B) is a task argument: a resolved buffer and a strided view. struct ChipTensor { PTOBufferHandle buffer; uint64_t start_offset; uint32_t shapes[MAX_TENSOR_DIMS]; uint32_t strides[MAX_TENSOR_DIMS]; uint32_t ndims; DataType dtype; AddressSpace address_space; }; `simpler::hbg::Tensor` and `simpler::tmr::Tensor` (128 B) are the same geometry plus `owner_task_id`, `version`, `manual_dep` and the two caches derived from the geometry. One copy each, shared across architectures, placed the way `src/common/host_build_graph/graph_cache.h` already established. `simpler::hbg` was an existing namespace. `Runtime::set_orch_args` is the single place one becomes the other. Both runtimes call it only from `host/runtime_maker.cpp`, so adoption happens on the host before any orchestration runs — including for `tensormap_and_ringbuffer`, whose AICPU executor now reads an already-adopted `simpler::tmr::EntryArgsStorage` instead of converting per run. From there inward nothing holds the argument form: orchestration, the payload, the TensorMap and the kernels all name their runtime's type. A `tensor.h` in each runtime's `runtime/` directory sits first on that runtime's include path, so existing `#include "tensor.h"` lines resolve to both types without an include sweep. That shim names the two headers by their path under `src/common`, which every platform target but `aicore` already had on its include path; the four `aicore` CMakeLists gain the line their `aicpu` and `host` siblings carry. `is_contiguous` and `extent_elem` become methods that compute from the geometry; a runtime that reads them per task caches them on its own `Tensor`. The view ops and the copy helpers that maintained those caches move with them — an argument is not a thing you take views of, and nothing outside a runtime called them. `init_external` and both factories lose their `manual_dep` / `version` parameters, which every caller passed as `false` / `0`. `PTO2TaskPayload` is no longer a friend, and the boundary header no longer includes `task_id.h`: it names no TaskId at all. `sizeof(ChipStorageTaskArgs)` falls from ~33.8 KB to 19464 B at the 256-tensor cap. `MAILBOX_SIZE` is unaffected — since hw-native-sys#1729 the frame is sized by the 144 B wire `Tensor`. `ChipTensor`, `PTO2TensorMapEntry` and `TensorCreateInfo` shared a byte-level layout so three copy paths could each be a single 64-byte `memcpy`, held by 21 hand-written `offsetof` assertions across four trees. Two of the three were shaped backwards to satisfy it: `TensorCreateInfo` carried `__pad0__` / `__pad2__` / `__pad_flags__` purely to occupy `ChipTensor::buffer` / `::owner_task_id` / `::address_space`, and the entry's `memcpy` wrote `ChipTensor::buffer.size` into a `PTO2TensorMapEntry *` field, its comment noting this was "harmless because `link_entry()` overwrites `next_in_bucket` immediately after". Each struct now assigns the fields it wants, by name. The optimization the `memcpy` existed for survives as intent: a canonically contiguous source has the derived pair recomputed from `shapes` rather than read across. `PTO2TensorMapEntry::copy_tensor_create_info` is deleted — it had no callers in any tree, and was the only reason `TensorCreateInfo` had to be punnable onto an entry. The entry keeps its own two-cache-line split and the assertions describing it; what goes is the claim that its bytes agree with a different struct's. A source that names one runtime's `Tensor` cannot be compiled under the other, and 57 scene-test sources were: a `host_build_graph` case naming the `tensormap_and_ringbuffer` case's kernels, and two `tensormap_and_ringbuffer` classes taking an hbg class's `CALLABLE` verbatim. That worked only because `ChipTensor` happened to be identical in both trees — the coupling this change removes. Each borrowing case now carries its own sources, 73 files, and names local paths. The two class-reuse cases rebase the inherited `CALLABLE` onto their own `kernels/` copy through a `_rebase_callable` helper, so the Python that drives them stays in one file while the C++ each runtime compiles is its own. `task_timing_slots` carries the runtime in a function default rather than a `@scene_test` decorator, so its sources move under `kernels/<runtime>/` and the helper builds the path from the runtime it is driving. The cost is real: `spmd_multiblock_mix`, `alternating_matmul_add`, `batch_paged_attention`, `paged_attention_unroll`, `benchmark_bgemm` and the task-timing kernels now exist once per runtime and the copies can drift. A case that asserts something about `host_build_graph` should not depend on a file the other runtime owns. `examples/` orchestration and kernel sources lose their `Generated by PyPTO IR Compiler` marker: they are this repository's sources, and this change edits them. 364 of those kernels declare a ptoas helper as `template <typename ChipTensor>`, shadowing the type with a parameter name. The parameter is `TensorT` now: a generated helper that means "any tensor with .data()" should not name one, and the shadowing is what made the rename ambiguous in the first place. `tensor_create_info.h`'s doc paragraph said `host_build_graph` has no initial-value fill "unlike the tensormap_and_ringbuffer copy of this header", which hw-native-sys#1981 made false by removing that fill; both copies now describe what is there. The copies were taken from sources whose spellings `main` has since retired — hw-native-sys#1980 finished that retirement repo-wide — so they carry the current ones. `test_hbg_sm_compaction` compared `GraphTensor` against `ChipTensor` to justify packing Graph boundaries into the payload's tensor slots, and read that pool as `ChipTensor`. The pool is the payload's, so both now name the runtime's `Tensor` — the assertion was true only while one type served both roles. Verification: full rebuild of both runtimes on both architectures, cpput 119/119, pyut 1908 passed / 14 skipped, a2a3sim 21 cases and a5sim 17 cases green. The sim sweeps build only the cases they run, so every orchestration source was also syntax-checked against its runtime's include path — 129 (source, runtime, arch) compiles, which is what caught a copy under host_build_graph/ still calling simpler::tmr::make_tensor_external. A second check reads which runtime *strings* a Python driver mentions rather than its @scene_test decorator; that is what caught task_timing_slots. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ChaoWao
added a commit
to ChaoWao/simpler-fork
that referenced
this pull request
Aug 25, 2026
…nsor An argument arrives at L2 as a `ChipTensor`, and the runtime then decides things about it: which task produced it, the version its OverlapMap keys on, whether dependency tracking is creator-only. Those decisions lived on `ChipTensor` itself, in `src/common/task_interface/`, so one type served the boundary and both runtimes' working state. `create_from_chip_args` recorded the mismatch as an assertion — `debug_assert(!t.manual_dep && t.version == 0)`, two fields that are meaningless on an argument. `docs/buffer-abi.md` recorded it too, calling `ChipTensor` "L2 leaf, internal" and saying "You never build a `ChipTensor`" while `ChipWorker.run` took a container of them and `nb::class_<ChipTensor>` exposed a `make()` factory. The fusion dates to hw-native-sys#1093, which needed the strided view hw-native-sys#808 had just given the runtime's tensor to reach the argument boundary, and got there by promoting the runtime-private header into `task_interface/` rather than adding strides to the 40 B `ContinuousTensor` it replaced. It cost 40 B → 128 B per wire tensor and a mailbox doubling for a capability that PR noted was not yet used: two months on, `make_tensor_arg` and `make_chip_tensor_arg` still reject non-contiguous tensors. with a cost argument that applies equally here; nobody undid it at L2. `ChipTensor` (72 B) is a task argument: a resolved buffer and a strided view. struct ChipTensor { PTOBufferHandle buffer; uint64_t start_offset; uint32_t shapes[MAX_TENSOR_DIMS]; uint32_t strides[MAX_TENSOR_DIMS]; uint32_t ndims; DataType dtype; AddressSpace address_space; }; `simpler::hbg::Tensor` and `simpler::tmr::Tensor` (128 B) are the same geometry plus `owner_task_id`, `version`, `manual_dep` and the two caches derived from the geometry. One copy each, shared across architectures, placed the way `src/common/host_build_graph/graph_cache.h` already established. `simpler::hbg` was an existing namespace. `Runtime::set_orch_args` is the single place one becomes the other. Both runtimes call it only from `host/runtime_maker.cpp`, so adoption happens on the host before any orchestration runs — including for `tensormap_and_ringbuffer`, whose AICPU executor now reads an already-adopted `simpler::tmr::EntryArgsStorage` instead of converting per run. From there inward nothing holds the argument form: orchestration, the payload, the TensorMap and the kernels all name their runtime's type. A `tensor.h` in each runtime's `runtime/` directory sits first on that runtime's include path, so existing `#include "tensor.h"` lines resolve to both types without an include sweep. That shim names the two headers by their path under `src/common`, which every platform target but `aicore` already had on its include path; the four `aicore` CMakeLists gain the line their `aicpu` and `host` siblings carry. `is_contiguous` and `extent_elem` become methods that compute from the geometry; a runtime that reads them per task caches them on its own `Tensor`. The view ops and the copy helpers that maintained those caches move with them — an argument is not a thing you take views of, and nothing outside a runtime called them. `init_external` and both factories lose their `manual_dep` / `version` parameters, which every caller passed as `false` / `0`. `PTO2TaskPayload` is no longer a friend, and the boundary header no longer includes `task_id.h`: it names no TaskId at all. `sizeof(ChipStorageTaskArgs)` falls from ~33.8 KB to 19464 B at the 256-tensor cap. `MAILBOX_SIZE` is unaffected — since hw-native-sys#1729 the frame is sized by the 144 B wire `Tensor`. `ChipTensor`, `PTO2TensorMapEntry` and `TensorCreateInfo` shared a byte-level layout so three copy paths could each be a single 64-byte `memcpy`, held by 21 hand-written `offsetof` assertions across four trees. Two of the three were shaped backwards to satisfy it: `TensorCreateInfo` carried `__pad0__` / `__pad2__` / `__pad_flags__` purely to occupy `ChipTensor::buffer` / `::owner_task_id` / `::address_space`, and the entry's `memcpy` wrote `ChipTensor::buffer.size` into a `PTO2TensorMapEntry *` field, its comment noting this was "harmless because `link_entry()` overwrites `next_in_bucket` immediately after". Each struct now assigns the fields it wants, by name. The optimization the `memcpy` existed for survives as intent: a canonically contiguous source has the derived pair recomputed from `shapes` rather than read across. `PTO2TensorMapEntry::copy_tensor_create_info` is deleted — it had no callers in any tree, and was the only reason `TensorCreateInfo` had to be punnable onto an entry. The entry keeps its own two-cache-line split and the assertions describing it; what goes is the claim that its bytes agree with a different struct's. A source that names one runtime's `Tensor` cannot be compiled under the other, and 57 scene-test sources were: a `host_build_graph` case naming the `tensormap_and_ringbuffer` case's kernels, and two `tensormap_and_ringbuffer` classes taking an hbg class's `CALLABLE` verbatim. That worked only because `ChipTensor` happened to be identical in both trees — the coupling this change removes. Each borrowing case now carries its own sources, 73 files, and names local paths. The two class-reuse cases rebase the inherited `CALLABLE` onto their own `kernels/` copy through a `_rebase_callable` helper, so the Python that drives them stays in one file while the C++ each runtime compiles is its own. `task_timing_slots` carries the runtime in a function default rather than a `@scene_test` decorator, so its sources move under `kernels/<runtime>/` and the helper builds the path from the runtime it is driving. The cost is real: `spmd_multiblock_mix`, `alternating_matmul_add`, `batch_paged_attention`, `paged_attention_unroll`, `benchmark_bgemm` and the task-timing kernels now exist once per runtime and the copies can drift. A case that asserts something about `host_build_graph` should not depend on a file the other runtime owns. `examples/` orchestration and kernel sources lose their `Generated by PyPTO IR Compiler` marker: they are this repository's sources, and this change edits them. 364 of those kernels declare a ptoas helper as `template <typename ChipTensor>`, shadowing the type with a parameter name. The parameter is `TensorT` now: a generated helper that means "any tensor with .data()" should not name one, and the shadowing is what made the rename ambiguous in the first place. ## Kernels name no runtime A kernel reads a payload element and its code is identical whichever orchestrator filled it, so naming a runtime there carries no information — and several kernels are compiled under both. `examples/a2a3/host_build_graph/deepseek_v4_flash_decode` re-points all 368 of the `tensormap_and_ringbuffer` case's incores at that case's directory; `test_task_timing_e2e` drives one AIV kernel under both. Each runtime's `runtime/tensor.h` therefore exports using TaskTensor = simpler::{hbg,tmr}::Tensor; and every kernel names `TaskTensor`. It resolves per translation unit, so it is one type per build, not a third type. The alternative was duplicating 13 MB of generated MoE kernels into the hbg tree, where the two copies would drift. The name is not `Tensor`: `buffer.h`'s L3+ wire `Tensor` is visible in every orchestration translation unit, and one spelling meaning two types by context is what this change exists to remove. Cross-runtime *orchestration* sources need no name — `const auto &a = orch_args.tensor(0).ref()` is enough, and only kernels must spell the type inside a `reinterpret_cast`. `tensor_create_info.h`'s doc paragraph said `host_build_graph` has no initial-value fill "unlike the tensormap_and_ringbuffer copy of this header", which hw-native-sys#1981 made false by removing that fill; both copies now describe what is there. The copies were taken from sources whose spellings `main` has since retired — hw-native-sys#1980 finished that retirement repo-wide — so they carry the current ones. `test_hbg_sm_compaction` compared `GraphTensor` against `ChipTensor` to justify packing Graph boundaries into the payload's tensor slots, and read that pool as `ChipTensor`. The pool is the payload's, so both now name the runtime's `Tensor` — the assertion was true only while one type served both roles. Verification: full rebuild of both runtimes on both architectures, cpput 119/119, pyut 1908 passed / 14 skipped, a2a3sim 21 cases and a5sim 17 cases green. The sim sweeps build only the cases they run, so every orchestration source was also syntax-checked against its runtime's include path — 129 (source, runtime, arch) compiles, which is what caught a copy under host_build_graph/ still calling simpler::tmr::make_tensor_external. A second check reads which runtime *strings* a Python driver mentions rather than its @scene_test decorator; that is what caught task_timing_slots. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ChaoWao
added a commit
that referenced
this pull request
Aug 25, 2026
…nsor (#1974) An argument arrives at L2 as a `ChipTensor`, and the runtime then decides things about it: which task produced it, the version its OverlapMap keys on, whether dependency tracking is creator-only. Those decisions lived on `ChipTensor` itself, in `src/common/task_interface/`, so one type served the boundary and both runtimes' working state. `create_from_chip_args` recorded the mismatch as an assertion — `debug_assert(!t.manual_dep && t.version == 0)`, two fields that are meaningless on an argument. `docs/buffer-abi.md` recorded it too, calling `ChipTensor` "L2 leaf, internal" and saying "You never build a `ChipTensor`" while `ChipWorker.run` took a container of them and `nb::class_<ChipTensor>` exposed a `make()` factory. The fusion dates to #1093, which needed the strided view #808 had just given the runtime's tensor to reach the argument boundary, and got there by promoting the runtime-private header into `task_interface/` rather than adding strides to the 40 B `ContinuousTensor` it replaced. It cost 40 B → 128 B per wire tensor and a mailbox doubling for a capability that PR noted was not yet used: two months on, `make_tensor_arg` and `make_chip_tensor_arg` still reject non-contiguous tensors. with a cost argument that applies equally here; nobody undid it at L2. `ChipTensor` (72 B) is a task argument: a resolved buffer and a strided view. struct ChipTensor { PTOBufferHandle buffer; uint64_t start_offset; uint32_t shapes[MAX_TENSOR_DIMS]; uint32_t strides[MAX_TENSOR_DIMS]; uint32_t ndims; DataType dtype; AddressSpace address_space; }; `simpler::hbg::Tensor` and `simpler::tmr::Tensor` (128 B) are the same geometry plus `owner_task_id`, `version`, `manual_dep` and the two caches derived from the geometry. One copy each, shared across architectures, placed the way `src/common/host_build_graph/graph_cache.h` already established. `simpler::hbg` was an existing namespace. `Runtime::set_orch_args` is the single place one becomes the other. Both runtimes call it only from `host/runtime_maker.cpp`, so adoption happens on the host before any orchestration runs — including for `tensormap_and_ringbuffer`, whose AICPU executor now reads an already-adopted `simpler::tmr::EntryArgsStorage` instead of converting per run. From there inward nothing holds the argument form: orchestration, the payload, the TensorMap and the kernels all name their runtime's type. A `tensor.h` in each runtime's `runtime/` directory sits first on that runtime's include path, so existing `#include "tensor.h"` lines resolve to both types without an include sweep. That shim names the two headers by their path under `src/common`, which every platform target but `aicore` already had on its include path; the four `aicore` CMakeLists gain the line their `aicpu` and `host` siblings carry. `is_contiguous` and `extent_elem` become methods that compute from the geometry; a runtime that reads them per task caches them on its own `Tensor`. The view ops and the copy helpers that maintained those caches move with them — an argument is not a thing you take views of, and nothing outside a runtime called them. `init_external` and both factories lose their `manual_dep` / `version` parameters, which every caller passed as `false` / `0`. `PTO2TaskPayload` is no longer a friend, and the boundary header no longer includes `task_id.h`: it names no TaskId at all. `sizeof(ChipStorageTaskArgs)` falls from ~33.8 KB to 19464 B at the 256-tensor cap. `MAILBOX_SIZE` is unaffected — since #1729 the frame is sized by the 144 B wire `Tensor`. `ChipTensor`, `PTO2TensorMapEntry` and `TensorCreateInfo` shared a byte-level layout so three copy paths could each be a single 64-byte `memcpy`, held by 21 hand-written `offsetof` assertions across four trees. Two of the three were shaped backwards to satisfy it: `TensorCreateInfo` carried `__pad0__` / `__pad2__` / `__pad_flags__` purely to occupy `ChipTensor::buffer` / `::owner_task_id` / `::address_space`, and the entry's `memcpy` wrote `ChipTensor::buffer.size` into a `PTO2TensorMapEntry *` field, its comment noting this was "harmless because `link_entry()` overwrites `next_in_bucket` immediately after". Each struct now assigns the fields it wants, by name. The optimization the `memcpy` existed for survives as intent: a canonically contiguous source has the derived pair recomputed from `shapes` rather than read across. `PTO2TensorMapEntry::copy_tensor_create_info` is deleted — it had no callers in any tree, and was the only reason `TensorCreateInfo` had to be punnable onto an entry. The entry keeps its own two-cache-line split and the assertions describing it; what goes is the claim that its bytes agree with a different struct's. A source that names one runtime's `Tensor` cannot be compiled under the other, and 57 scene-test sources were: a `host_build_graph` case naming the `tensormap_and_ringbuffer` case's kernels, and two `tensormap_and_ringbuffer` classes taking an hbg class's `CALLABLE` verbatim. That worked only because `ChipTensor` happened to be identical in both trees — the coupling this change removes. Each borrowing case now carries its own sources, 73 files, and names local paths. The two class-reuse cases rebase the inherited `CALLABLE` onto their own `kernels/` copy through a `_rebase_callable` helper, so the Python that drives them stays in one file while the C++ each runtime compiles is its own. `task_timing_slots` carries the runtime in a function default rather than a `@scene_test` decorator, so its sources move under `kernels/<runtime>/` and the helper builds the path from the runtime it is driving. The cost is real: `spmd_multiblock_mix`, `alternating_matmul_add`, `batch_paged_attention`, `paged_attention_unroll`, `benchmark_bgemm` and the task-timing kernels now exist once per runtime and the copies can drift. A case that asserts something about `host_build_graph` should not depend on a file the other runtime owns. `examples/` orchestration and kernel sources lose their `Generated by PyPTO IR Compiler` marker: they are this repository's sources, and this change edits them. 364 of those kernels declare a ptoas helper as `template <typename ChipTensor>`, shadowing the type with a parameter name. The parameter is `TensorT` now: a generated helper that means "any tensor with .data()" should not name one, and the shadowing is what made the rename ambiguous in the first place. ## Kernels name no runtime A kernel reads a payload element and its code is identical whichever orchestrator filled it, so naming a runtime there carries no information — and several kernels are compiled under both. `examples/a2a3/host_build_graph/deepseek_v4_flash_decode` re-points all 368 of the `tensormap_and_ringbuffer` case's incores at that case's directory; `test_task_timing_e2e` drives one AIV kernel under both. Each runtime's `runtime/tensor.h` therefore exports using TaskTensor = simpler::{hbg,tmr}::Tensor; and every kernel names `TaskTensor`. It resolves per translation unit, so it is one type per build, not a third type. The alternative was duplicating 13 MB of generated MoE kernels into the hbg tree, where the two copies would drift. The name is not `Tensor`: `buffer.h`'s L3+ wire `Tensor` is visible in every orchestration translation unit, and one spelling meaning two types by context is what this change exists to remove. Cross-runtime *orchestration* sources need no name — `const auto &a = orch_args.tensor(0).ref()` is enough, and only kernels must spell the type inside a `reinterpret_cast`. `tensor_create_info.h`'s doc paragraph said `host_build_graph` has no initial-value fill "unlike the tensormap_and_ringbuffer copy of this header", which #1981 made false by removing that fill; both copies now describe what is there. The copies were taken from sources whose spellings `main` has since retired — #1980 finished that retirement repo-wide — so they carry the current ones. `test_hbg_sm_compaction` compared `GraphTensor` against `ChipTensor` to justify packing Graph boundaries into the payload's tensor slots, and read that pool as `ChipTensor`. The pool is the payload's, so both now name the runtime's `Tensor` — the assertion was true only while one type served both roles. Verification: full rebuild of both runtimes on both architectures, cpput 119/119, pyut 1908 passed / 14 skipped, a2a3sim 21 cases and a5sim 17 cases green. The sim sweeps build only the cases they run, so every orchestration source was also syntax-checked against its runtime's include path — 129 (source, runtime, arch) compiles, which is what caught a copy under host_build_graph/ still calling simpler::tmr::make_tensor_external. A second check reads which runtime *strings* a Python driver mentions rather than its @scene_test decorator; that is what caught task_timing_slots.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
TaskArgs / wire / bindings previously carried the 40 B contiguous-only
ContinuousTensor. They now carry the runtime's unified 128 B stridedTensor, so a single tensor type spans host construction, the wire, and theruntime. Behavior is unchanged: this construction path stays contiguous
(row-major strides,
start_offset == 0), enforced at the entry points. Stridedviews are now expressible end-to-end but
make_tensor_argdoes not yet producethem.
Key changes
Tensor/PTOBufferHandle/make_tensor_external/TensorArgTypeinto
src/common/task_interface/tensor.h; movepto_task_id.hto common.Segment/OverlapStatus→pto_tensormap.h;TensorCreateInfo+ materialization (now free functions)→ new
tensor_create_info.h.always_assert/AssertionErrorinto sharedassert_compat.h; add ahost-side
assert_compat.cpplinked into the binding and the cpp UTs.child_memory(byte 43, cache line 1) +is_child_memory()/nbytes()/
data_as<T>()toTensor; make the default ctor public for POD/arraystorage.
TaskArgs/ChipStorageTaskArgs/TaskArgsViewuseTensor; migrateconsumers off
.data→buffer.addr(orchestrator, runtime_maker,aicpu_executor, remote_endpoint, remote_wire encode/decode, tensor_info).
make_tensor_argreturnsTensorand rejects non-contiguous torch tensors;from_tensor_argbecomes aconst Tensor&passthrough.tensor_arg.h; renameRUNTIME_MAX_TENSOR_DIMS→MAX_TENSOR_DIMS.ABI / sizing
MAILBOX_SIZEbumped16384 → 32768 (128 B ×
CHIP_MAX_TENSOR_ARGSalone exceeded the old mailbox);the capacity
static_assertis updated. All three programs (host / AICPU /AICore) rebuild together, so there is no cross-version wire concern.
Verification
static_asserts hold.test_task_interface: 98 passed.🤖 Generated with Claude Code