[GenerelSchwerz]

Superseded by PR #117, which includes single-token and small multi-token adjacent MMVQ activation reuse in one matcher/helper implementation, with fresh combined correctness, dispatch and performance validation.

This standalone draft is closed without merging. Its branch and historical evidence remain available; active review belongs on #117.

AI usage: YES. Codex implemented and validated this owner-authorized fork change, with independent GPT-6.1 Sol High source review. The owner retains review, maintenance and any upstream submission responsibility.

[GenerelSchwerz]

Cross-quant and older-GPU follow-up for this PR. Measured official base dcd387a412ca54e172a8d60eb71ef6753850c8ca against unchanged candidate 8ea277299c7a08898629ff72a951dc9752cbd9d9, on Linux / RTX 5070 Ti / CUDA 13.4.59.

Correctness and method: all 23 weight formats supported by the MMVQ switch passed at the medium size. Both projection outputs were byte-identical between baseline and candidate, including after changing the shared input. All 328 unprofiled executions passed CPU comparison; worst NMSE was 1.5422e-5 versus the existing matrix tolerance 5e-4. These are synthetic kernel fixtures; IQ importance weights were uniform, not model calibration.

Timed the complete MUL_MAT, RESHAPE, MUL_MAT sequence at batch 1, including activation quantization. Each case used 16 warmups, 10000 asynchronous replays and a final synchronization, with four alternating baseline/candidate pairs. CPU reference work and fixture generation were outside timing. Shared build/GPU locks were held for each execution. Percentages below mean less component execution time, not increased model tok/s.

At k=1024, output rows 512+512, with default CUDA graphs/PDL:

Weight format	Baseline -> candidate, us	Time saved
Q4_0	6.1416 -> 4.0984	33.27%
Q8_0	6.1417 -> 4.0986	33.27%
Q4_K	6.1416 -> 4.0986	33.27%
Q6_K	6.1415 -> 4.1049	33.16%
IQ2_XXS	6.1466 -> 4.6091	25.02%
IQ2_S	6.1465 -> 5.7703	6.12%
Q2_K	6.1417 -> 6.1412	0.01%
Q3_K	6.1416 -> 6.1412	0.01%

All 23 medium-size formats

Weight format	Baseline us	Candidate us	Time saved
Q4_0	6.1416	4.0984	33.27%
Q4_1	6.1415	4.0984	33.27%
Q5_0	6.1418	4.0986	33.27%
Q5_1	6.1415	4.0985	33.27%
Q8_0	6.1417	4.0986	33.27%
Q2_K	6.1417	6.1412	0.01%
Q3_K	6.1416	6.1412	0.01%
Q4_K	6.1416	4.0986	33.27%
Q5_K	6.1417	4.0986	33.27%
Q6_K	6.1415	4.1049	33.16%
IQ2_XXS	6.1466	4.6091	25.02%
IQ2_XS	6.1416	4.0986	33.26%
IQ2_S	6.1465	5.7703	6.12%
IQ3_XXS	6.1444	4.0986	33.30%
IQ3_S	6.1417	4.0985	33.27%
IQ4_NL	6.1417	4.1002	33.24%
IQ4_XS	6.1417	4.0985	33.27%
IQ1_S	6.1413	4.0984	33.26%
IQ1_M	6.1418	4.0988	33.26%
MXFP4	6.1416	4.0986	33.27%
NVFP4	6.1416	4.0985	33.27%
Q1_0	6.1436	4.1070	33.15%
Q2_0	6.1414	4.0985	33.27%

Shape sensitivity: the six tested small cases (k=256, rows 64+96) were effectively unchanged, including a retained Q4_K mean 0.05% slower. At k=4096, rows 4096+1024, Q8_0/Q6_K/IQ3_XXS saved 8.51%/8.35%/14.13%; Q4_0/Q4_K/NVFP4 were effectively unchanged. No material repeatable regression appeared in this tested matrix. There is no universal all-format/all-shape gain.

Actual dispatch: eleven separate node-level profiles cover nine representative formats, plus Q4_K with PDL off and graphs off. Every comparison shows quantize_q8_1 launches 292 -> 146 across 146 graph evaluations. Other kernel counts, matrix-kernel names, grids, blocks, register counts and shared-memory configurations match exactly. Profile outputs also match; profiler timings are excluded from the speed claims. This is evidence that the shared path actually executes, including the Q2_K/Q3_K cases with no timing benefit.

Launch sensitivity: at the medium size, disabling PDL reduced savings for Q4_0/Q4_K/IQ3_XXS to 0.80%/0.27%/0.15%. Disabling graphs while retaining PDL gave 24.00%/23.63%/22.54%. PDL-off on Blackwell is a control, not older-GPU emulation. NVIDIA documents PDL from compute capability 9.0 onward, so the default 33% result must not be projected onto Turing/Ampere/Ada.

Architecture assessment: Turing has separate FP32/INT32 resources; Ampere GA10x and Ada have FP32-only capacity plus shared FP32/INT32 capacity. RTX Blackwell increases throughput for many integer instructions, but not every instruction. The CUDA native instruction table distinguishes arithmetic, shifts, shuffles and conversions. Packed INT8 dots are different from tensor-core INT8/FP4 MMA; this PR retains the existing MMVQ kernels and adds no tensor-core arithmetic.

The benefit mechanism is removal of a duplicate activation read, reduction, conversion, scratch write and launch. Counterarguments remain: different launch scheduling, faster quantization on another device, host matching overhead, or eviction of the shared activation by the first projection can offset savings. These are risks to measure, not observed older-GPU regressions. Native older-GPU testing is pending hardware availability. The inherited eligibility guard rejects capability <= 6.0; Pascal 6.1 can qualify (PASCAL=600, DP4A=610), so Pascal is not excluded wholesale.

Model/PR85 boundary: the prior stabilized Q4_0 model observation remains approximately +0.33% TG, with retained startup/serving variability. This sweep adds no all-quant model-level claim. #85 operates on quantized KV-cache staging in MMA FlashAttention; these quant names are projection weight formats. Combined PR85/PR116 performance remains untested.

Raw commands, cached fixture/output hashes, all samples, profiles, harness sources and binary identities are retained locally under /home/gencoolpc/moe-cache-tests/results/upstream-kernel-series-20261001: QUANT-CROSS-EVIDENCE.md, QUANT-CROSS-RESULTS.json, QUANT-CROSS-DISPATCH.json, QUANT-CROSS-MANIFEST.json and CUDA-ARCHITECTURE-NOTES.md. All 350 follow-up execution records have clean teardown. No source changes were needed.

AI assistance: Codex conducted this follow-up and prepared this comment at the repository owner's explicit request under the maintained-fork policy.