[GenerelSchwerz]

Overview

GDN alpha/beta gates currently require separate projections, pointwise kernels and, for packed banks, output copies. This PR recognizes closed graph cuts and writes the final gates in compatible projection kernels, shares rounded activation preparation when the original two GEMMs must remain, and combines postprocessing when producer fusion is unsuitable. Admission follows tensor contracts and existing CUDA selectors; there are no model-name rules or weight repacking.

Review against reference/upstream-kernels-20261001, official base dcd387a412ca54e172a8d60eb71ef6753850c8ca. This is an owner-requested review PR in the maintained fork. Related independent work: preparation reuse in #117/#118/#119, convolution/SiLU/QK in #121, and routed expert gate/up pairing in #122. This PR concerns recurrent attention gates, not MoE expert routing.

Coverage

Graph / original execution	Result and admission
Separate MMVF/MMVQ banks	Pair original dots/reducers and write softplus/bias/ssm_a and sigmoid outputs directly. F32/F16/BF16, all 21 ordinary vector quant formats and native MXFP4/NVFP4 retain their original representation and selector limits.
Optional scalar/head projection factors	Independent alpha/beta factors use the original rounded F32 multiplication before bias/sigmoid. Floating and quantized vector producers are covered.
Separate floating MMF banks	Pair compatible F32/F16/BF16 banks as virtual rows without repacking; retain original conversions, MMA and reducers. Both banks must select stock MMF, and the bank seam must align to the existing row tile. Scalar/head factors are supported.
Small BF16 cuBLAS projections	NVIDIA Ampere+ only, both original effective compute choices must be BF16. Round F32 activations to the original BF16 operands, then use a paired MMVF F32 reducer. This is an explicitly different summation policy; see numerical qualification below. At most two existing eight-column tiles, or sixteen columns per sequence plane. Existing vector/MMF selection has priority.
Longer BF16/F16 cuBLAS projections	Share the original rounded activation image, retain two original GEMMs and combine final postoperations when the closed cut permits it. Larger batches retain this path rather than the new paired reducer.
Packed flat/grouped banks	Remove eligible output copies; ordinary vector/MMF producers can write final gate outputs directly. Packed MMQ retains its original projector and combined postprocessing.
Column/batch/full repeated beta factors	Materialize the original beta projection and scale MUL, then combine final postprocessing. This also admits qualified mixed-dtype banks and already-computed compound alpha results; it does not pair arbitrary expression trees.

Logical cut closure, extra consumers, computed parameters, original skipped writes, physical ranges including NONE/leaf reads, padding, mutation, standard allocation dependencies and concurrent-stream exclusions remain guarded. Unsupported cuts execute the original graph. Private source-program allocation dependencies are separate integration work.

Isolated benefit

RTX 5070 Ti, SM120. Each figure is complete-component time against its stated frozen control, with actual optimized dispatch and complete downstream outputs checked. Rows are separate experiments and their gains cannot be added.

Component	Control	Less time
Initial Q4_0 K1024/H16/N1 and Q8_0 K2048/H32/N1	Official frozen base	33.23% / 49.93%
Native MXFP4/NVFP4 raw N4	Generic extension c03ed102f	25.11% / 24.96%
Packed Q8_0 K2560/H48/N1/group3	Native extension a1591844	15.69%
Packed floating MMF F32 N8	Packed vector extension c673b53	9.05%
Floating vector F32 N3 scalar / BF16 N1 head factors	Packed MMF extension 74cad68c	39.90% / 49.23%
Separate BF16 MMF K2560/H64 N2 / N8	Published b8df474c	42.67% / 37.48%
Rounded-input paired BF16 K2560/H48 N2 / N8	Published b8df474c	75.44% / 55.53%
Rounded-input paired BF16 K2560/H48 N9 / N16	Published b8df474c	49.03% / 33.30%
Materialized scaled-beta post K2560/H48 N2 / N8	Published b8df474c	2.27% / 4.94%

All reported pairs improved in every rotated repetition (four pairs; five for initial rounded N2/N8). Other small or flat outcomes remain recorded.

Two measured exclusions are deliberate. Unrestricted rounded-input tiling was 57.08% slower at N17 and 99.93%-242.81% slower at tested N24/N31/N32/N129, so the final matcher retains original GEMMs above sixteen columns. A packed ordinary-MMQ epilogue prototype was 11.16%/7.01% slower in whole-primary/split-fixup Q4 cases and added 26.29 MB of CUDA code; that prototype is not included.

Correctness and regressions

Ordinary vector/MMF pairing and materialized postprocessing preserve their original arithmetic and qualified full GPU outputs byte-for-byte. The new BF16 cuBLAS replacement preserves operand rounding, but cuBLAS summation order is opaque and its outputs are not claimed byte-identical. Its broad qualification covers eleven admitted numerical/layout groups and eleven original declines: midpoint rounding, cancellation, softplus thresholds, scalar/head factors, extreme factors, strided/batched inputs and odd heads. Twenty-two actual CPU graphs provide 396 per-output/update checks at the unchanged 5e-4 NMSE bound; admitted-case CPU maximum was 1.314e-11. An extreme head-factor fixture has a 0.84375 absolute difference from original CUDA on million-scale outputs, NMSE 8.827e-12. Full differences are retained; the smaller ordinary-case absolute error is not a universal bound.

Independent MMF qualification includes twenty admitted layouts, selector and alias declines, actual CPU graphs, existing backend checks and two clean memchecks. Scaled-beta post qualification covers fourteen layouts, exact-inplace admission, partial/gate overlap declines and a separately saved extra consumer, CPU references and two clean memchecks. Tiled rounded qualification adds N9/N16, larger rejected shapes and batched/strided planes with actual original rounded-input references.

The final integrated DLL passes an independent raw audit of 138 commands, 56 actual profiles, 96 complete three-update component payloads and thirty model output files. Sixteen new-path pairs match individually qualified outputs, canonical kernel inventories and full launch geometry exactly; ten existing-path pairs include original N17/N32 GEMM admission. Four scaled-input alias controls and two clean memchecks pass. Final old-path mean costs versus b8df474c were BF16 N1 0.793% more, Q4 N8 0.026% more, packed BF16 MMF 0.734% more, and materialized MMF 0.009% less, with mixed repetition signs. Final fixed-token model means at C1/C3/C4 were 0.019% less, 0.064% more and 0.071% less, with mixed signs and exact saved logits. These are regression checks, not new model gains.

Earlier matching Qwen3.5-0.8B Q4_0 fixed-token execution improved 2.10%/1.75% at three/four columns versus initial PR123, while one column used 0.14% more time. Later extension model comparisons are essentially flat with small losses retained. Runs use MTP disabled, 128 fixed steps, 32 warmups, per-process prewarm and four rotated pairs, saving full-vocabulary logits at steps 0, 1, 126, 127. These results do not establish free-generation throughput, native MTP benefit, or actual new MMF/rounded-producer dispatch in that model.

Cost, limits and evidence

The final CUDA library is 87,311,648 bytes, +4,860,552 bytes against b8df474c; separate MMF adds 768 producer variants. All final compiled resource records match the independently qualified implementations, including selected complete SASS checks. Existing BF16 GDN MMVF has eleven resource changes and a larger N1 instruction body; ordinary selected kernels remain instruction-identical. These are bounded static checks, not a zero-runtime-cost claim. Cached build times are not matched compile-cost evidence.

Native FP4 matrix activation paths, separate quantized MMQ producer pairing, non-aligned MMF seams, arbitrary dynamic/LoRA expression pairing, and shared preparation through scaled long-bank cuts are not newly optimized. A single-grid BF16 tiler could reduce repeated bank scans, but remains a source-only next step after measured serial-tiling regression. Existing execution is retained; it is not counted as equivalent optimized coverage. Other GPU generations, HIP/MUSA producer dispatch, native 35B/MTP, full perplexity/CI and the combined PR stack are unqualified. No universal model coverage or speedup is claimed.

Commands, frozen source/helper/DLL hashes, complete GPU/CPU payloads, SQLite profiles, resource/SASS dumps, rejected results and independent reviews are retained under /home/gencoolpc/moe-cache-tests/results/upstream-kernel-series-20261001, notably FLOATING-EXPERT/INTEGRATED-FINAL-SOURCE-V2 and INTEGRATED-FINAL-PROOF-V1. Each GPU suite holds the ordered shared build/GPU locks for its entire run; timing/model/profile work is serial. Immutable harness corrections preserve successful results and failures without repeating qualified runs or changing numeric policy.

Requirements

• Contributing guidelines reviewed; the repository owner's fork-only automation exception applies. The owner is responsible for review and maintenance of submitted changes.
• AI usage disclosure: YES. Codex implemented, reviewed and tested this owner-directed change; independent Codex subagents provided bounded source/evidence reviews. No upstream submission or merge is made.