[GenerelSchwerz]

Overview

Share temporary Q8_1 activation images across compatible ordinary and routed MMVQ projections. Eligible RMS and F32 GLU producers can emit that image while retaining their original final F32 output. This removes a separate activation-quantization launch and reread where the allocated graph permits it. Existing matrix selection, reduction trees, quantization rounding and higher-priority matrix fusions remain intact.

PR116 is superseded. Ordinary and routed MMQ preparation remain in PR118 and PR119. These are independent review branches; their combined integration needs validation.

Coverage

• Ordinary MUL_MAT and quantized MUL_MAT_ID on their existing MMVQ paths. Routed eligibility uses get_mmvq_mmid_max_batch(type, cc) and the true token dimension. All 23 MMVQ weight formats use the standard Q8_1 activation image; compatible mixed formats, per-slot routed inputs and valid scalar strides are supported.
• Multiple live activation groups and arbitrarily many compatible readers. Existing-pool storage is packed by live intervals, including original stream joins. Input/ID identities and expert counts constrain routed reuse groups; input/ID writes, allocation aliases and implicit weight-padding writes bound reuse.
• Ready common branch inputs are prepared before the original fork event. Branch-produced inputs stay on their stream until its join. No new events or forced branch serialization. Event-backed graphs keep stream-safe reuse and the original quantizer; producer emission is declined.
• RMS-alone, RMS+MUL, RMS+MUL+ADD and RMS+SCALE producers can emit images, including singleton readers. Full-span canonical views and compatible reader groups can share emitted images. Later input-write epochs retain separate preparation. The activation image is independent of route IDs; emission can serve compatible ID-distinct readers.
• F32 ReGLU, GeGLU, SwiGLU, GeGLU-erf, GeGLU-quick, SwiGLU-OAI and SwiGLU-clamp can emit images. Separate, packed and packed-swapped inputs retain their original formulas, alpha/limit parameters and independent source row strides. Output and image row widths must be whole Q8_1 blocks; full-span reshapes are permitted. All original F32 output stores remain, including live outputs and extra floating readers.
• Strict direct ranges include synthetic NONE inputs. Unsafe input/output/weight aliases, partial views and overwritten activation epochs retain original preparation. Existing matrix+GLU fusion keeps priority; when it consumes a planned producer, the original quantizer lazily prepares the missing image. Higher-priority norm/RoPE cuts and live-intermediate rules remain intact.
• Images refresh on every graph execution/replay. This is temporary intra-graph reuse, not cross-token caching. Unsupported buffers/layouts/devices, FWHT, fusion-disabled execution and batches outside existing MMVQ selection retain original paths. There is no model whitelist.

This PR does not implement cross-token weight decode reuse, new paired matrix kernels, omission of dead F32 producer stores, MMQ producer images, direct emission from fused matrix producers, transfer/residency policy, an MTP implementation, the full HC pipeline or private-executor allocation/dispatch integration. Eligible nodes inside an existing MTP graph can benefit; model-serving gains remain unproven.

GLU producer evidence

RTX5070Ti / CUDA13.4 / SM120a. Own control is the previous PR117 head 55098781192d0231007d6b1b2ca3e2a87134f7ca, based on official upstream dcd387a412ca54e172a8d60eb71ef6753850c8ca. The new review leaf is six files, 152 added / 38 deleted lines, including 20 lines in the existing backend test file. The complete PR includes prior reuse/RMS work: ten files, 943 added / 47 deleted lines against its reference base.

Three rotated pairs per row, 2000 warmed captured-graph replays per run. Times cover the complete component graph; correctness, image inspection and sanitizer runs are separate from timing.

Q4_0 component	Control us	Candidate us	Less GPU time
Routed activation and two projections	8.189	6.148	24.93%
Packed activation and two routed projections	8.190	6.148	24.93%
Padded activation sources and two routed projections	8.189	6.148	24.93%
Per-slot routed, K8192 / 32 slots / T8	73.165	69.718	4.71%
Per-slot routed, K2560 / 128 slots / T8	85.051	81.276	4.44%
Ordinary, K8192 / T8	16.377	14.345	12.41%

The typical K2560 / 8-slot / T8 routed case is effectively flat (20.489 vs 20.470 us). Early ordinary, per-slot, existing paired-fusion, live-gate, RMS and K2592-tail controls are also flat. Fewer quantization launches do not imply a universal gain. Larger-volume improvements repeat in all three pairs.

Validation of this leaf:

• 82 normal-backend control/candidate component pairs / 328 refreshed datasets preserve full original GPU outputs, CPU references and uploaded input bytes. Coverage includes all 23 formats, all seven GLU formulas, packed/swapped/strided inputs, retained F32 outputs, full-span reshapes, seven readers, intervening independent work, writes, direct NONE-input alias declines and unchanged MMQ fallback. Three cases exercise actual upstream scheduler allocation, dispatch and capture, including existing paired-fusion priority and live-gate behavior. Persistent uploads are retained and checked after warmup/replay.
• 80/82 component pairs pass the unchanged CPU NMSE bound 5e-4. Broad random Q4_1 and Q5_1 cases reproduce the identical original failures: respectively 0.000579074244354 and 0.00231894806368 in both arms. These two are not counted as CPU-bound passes; no tolerance was relaxed. The changed producer is also checked directly through 91 complete original/candidate F32 and Q8_1 image byte pairs, including half scales/sums, codes and padding, using an external launch observer without product hooks.
• The existing backend fixture gains 36 cases covering all GLUs, packed/swapped inputs, ordinary/routed Q4_0/Q8_0 and retained GLU outputs. Both builds pass 510/510 cases with default execution, graphs disabled and fusion disabled: 3060 checks. Six memchecks, one racecheck and two syncchecks report zero errors/hazards with retained exact outputs/reference/input arrays.
• All 134 original unary/norm device functions retain identical SASS/resource records after normalizing only the generated nvcc translation-unit symbol token. Seven new GLU variants each use 24 registers and zero stack/local/shared memory. Core, norm and unary translation units compile for SM75/80/89/90: twelve checks, with no older-GPU runtime/performance claim.

Artifacts: moe-cache-tests/results/upstream-kernel-series-20261001/ACTIVATION-IMAGE-EMISSION-20261009. FINAL-SEALED-V1.json verifies 4943 source/build/log/payload bindings; FINAL-RESULTS-V1.md and FINAL-SOURCE-REVIEW-V1.md record results and exclusions. Failed external-fixture/parser probes are preserved and corrected separately, without product or ownership-check changes. No serving or Strata speed claim is made for this leaf.

Earlier routed reuse evidence

The preceding routed extension 55098781192d0231007d6b1b2ca3e2a87134f7ca was qualified against d1b2bd8d074aa6253e3123c5f19a5e9baed423bc; it changed host admission and the existing fixture, with quantizer/MMVQ/norm device objects unchanged. Three rotated pairs and 2000 captured replays measured 19.86% less component time for two shared routed projections, 13.74% for three per-slot projections, 12.71% with padded strides, 14.20% with seven readers and 11.10-12.44% for selected RMS producers. Distinct-input, ordinary, already-fused and T16 MMQ controls were flat; T16/T32 selection stayed unchanged.

That extension retained 50 paired normal graphs / 200 datasets, all 23 quant formats, mixed formats, input/ID writes, distinct ID values, in-place writes, live outputs and eight norm cuts. Full GPU/reference/input bytes matched and CPU NMSE stayed below 5e-4. All 474 then-existing backend cases passed in both builds across default/uncaptured/unfused operation. Eight focused memchecks and four changed-host older-SM compile checks passed.

Two additional actual two-stream fork/join fixtures retained independent MMVQ branches, exact outputs and event-backed emission decline. Quantizer counts fell from two to one; both memchecks passed. One component improved 3.55%, the norm-before-fork case was flat. Total routed-extension timing evidence was 60 rotated pairs / 120 calls. Artifacts remain under MULTITOKEN-WEIGHT-REUSE-20261008/ROUTED-ACTIVATION.

Prior evidence and serving limits

Earlier normalization qualification includes 18 boundary pairs, 234 emitted-image observations representing 37 fresh images, actual scheduler allocation, direct alias/partial-view/write-epoch declines and event-backed emission decline. One compound overwrite fixture exceeds CPU NMSE 5e-4 identically in both arms and remains baseline-limited, with no relaxed tolerance; the other 17 pairs pass. Artifacts remain under HYPERCONNECTION/Q8-EMISSION.

Earlier ordinary reuse evidence includes branch-local/common-input streams, overlapping/disjoint groups and post-join lifetimes. Some graphs-disabled synthetic cases took 0.4-2.4% more time; a small supplied-token model trial showed no additional gain over the previous extension. These historical results are not serving evidence for the latest updates.

Previous Qwen3.6-35B-A3B Q4_K_M serving tests used the same native MTP configuration in both builds: ABBA measured 100.010 vs 100.951 tok/s with overlapping ranges, hence inconclusive. Without MTP, one pair measured 64.525 vs 64.626 tok/s, also insufficient for a gain claim. Outputs matched within each mode and MTP acceptance matched 1345/2106. The roughly 1.6x MTP-versus-no-MTP difference is not a PR117 speedup.

Reproduction and requirements

Run test-backend-ops test -b CUDA0 -o MUL_MAT_REUSED_INPUT -j1, then repeat with GGML_CUDA_DISABLE_GRAPHS=1 and with GGML_CUDA_DISABLE_FUSION=1. Exact external commands, frozen weights, loaded DLL hashes and retained arrays are in the bound manifests. Use fresh output labels. Whole suites hold ordered shared build then GPU locks, workload-sized hard host-memory limits, zero swap allowance, finite timeouts and verified whole-process-tree teardown.

• Contributing guidelines and owner-fork policy read. Maintained-fork draft only; the owner handles maintenance and any upstream submission.
• AI disclosure: YES. Codex implemented/tested this change and prepared this description. Earlier stages used bounded independent reviews. The repository owner remains responsible for the submitted changes.