[Maxrio9]

Update (nvcc 13.0, rebased on main 37a4a84a): this branch now carries only the source and tests (see the receipts branch for the evidence: nvcc 13.0 run). Full CUDA contract under nvcc 13.0.88: 511 kernels pass gates 1-4, 0 fail, 340/340 controls caught, 213 limit problems, identical to untouched main under the same compiler; the new launches pass their live limit checks. The earlier numbers below are from the 13.3 run on 94b90c9d.

The 128-column/eight-warp affine MLP tile applies only to SM120/82 SMs, N=17408, K=5120, one K slice and 1–16 rows. Other chips, shapes, GEMV and grouped projections retain their dispatch. Arithmetic and serving features stay unchanged. Fixes #1700. Attention is separate in #1731; no GDN fusion.

Draft: numerical qualification completed; the global backend gate fails only on unchanged main limit-record gaps.

The full unfiltered CUDA integration run reports 465 kernels passing gates1–4, zero numerical failures, eight declared known findings;322/322 controls caught, zero missed;223 limit-record problems. A limits-only control with unmodified main production kernels/registry reproduces the identical168 missing static records and55 live mismatches. The three new launches (MLP128 and both MT2 forms) all pass their own live limit checks. Full backend qualification therefore remains red; the PR stays draft pending maintainer review of that unchanged baseline.

Complete numerical run and main-only limit diagnosis contains the exact SUMMARY, full/partial logs, source/binary/build hashes, baseline control and every failure. One combined integration binary contains the two independently benchmarked changes plus a disclosed five-file test-only fixture-lifetime fix; math/shapes/seeds/controls/bounds stay unchanged. This does not claim an unmodified-source, single-contribution-only or globally passing backend run.

Above-noise check requested by the maintainer

Six independent fresh-calibration stock/candidate/stock groups: 18 native processes, canonical greedy code/prose, seven seeded 256-token repeats plus warmup for each prompt and boot. Same single-contribution binary on main 94b90c9d, override TF_CUDA_TEST_MLP128=0/1/0. Every output fingerprint matches.

• Greedy geometric mean gain: +0.825%.
• Observed six-group range: +0.707% to +0.900%; all six positive.
• Approximate 95% Student-t interval on the six group log ratios: +0.754% to +0.896% (df=5).
• Exact two-sided sign test: p=0.03125. Requests/tokens are not treated as independent samples.

Predeclared protocol, all paired cells/baseline drift, raw outputs/logs, telemetry and analysis. One incomplete group was interrupted by a failed desktop-status read and restarted in full; partial outputs are retained. No timing-based exclusions or optional stopping. The interval assumes independent approximately normal group log gains and does not establish a universal workload/GPU benefit.

Separate independent current-main bracket: greedy +0.754%, community +0.172%, all tokens/features match. Community includes a negative code/T1 cell. Original-source clean brackets and historical private zero-reserve measurements remain separate; no pooled samples. The public code branch retains its original 01e25d3a ancestry; rebased receipts identify their exact 94b90c9d source and binary hashes.

Complete receipt includes replacement registration against stock bytes, independent FP64 contract cases, live/compiled launch records, host chip/shape fallback controls, four clean sanitizer runs and the completed 60-request cold supplement. Host graph: 285/285 steps, 108 fresh tests with cached remainder using a disclosed common test-fixture repair; inherited mock-fixture crashes reproduced on untouched main remain attached. This is not an unmodified-source host pass or full GPU qualification.

RTX PRO 4500 Blackwell (SM120/82 SMs), 200 W; Ryzen 5900X, 31.24 GiB physical RAM / 24 GiB test budget; Linux, driver 615.78.08, Zig 0.17.0, nvcc 13.3.73. Native Zig/CUDA. Qwen3.8-27B affine4/group64 revision 22d8d538154e0e5b6f8dcb1fcc74a60a0798104c; BF16 DFlash2 revision 50307d4c4cde6860d4eee73e2547cd786fe8e8a4. Configured 131072 context, vision/tools/schema/reasoning support, FP8 KV, int8 prefill, 8 GiB host cache, parallel 1, upstream 0.5 GiB reserve. Build label 1.1.6 is recipe compatibility metadata. No physical 5090/PRO 6000 qualification; independent override instructions remain in the existing PR comment.

• Repeated alternating main/candidate evidence with matching tokens and between-group uncertainty.
• Replacement/launch/host dispatch and sanitizer evidence attached with scope limits.
• Cold supplement, inherited failures and negative controls retained.
• Complete unfiltered numerical run and publish final SUMMARY, including all failures.
• Resolve unchanged main compiler-record gaps / obtain maintainer scope decision before marking ready.
• Maintainer release/cross-platform/previous-release and standard-server gates where applicable.

[Maxrio9]

Testing request: RTX 5090 and RTX PRO 6000 Blackwell owners

Could someone with either card test this improvement independently? Please specify the PRO 6000 edition (Workstation, Max-Q or Server) and power limit. Use only PR #1730; do not combine it with the companion PR.

A startup-only experimental override is now included:

Setting	Effect
TF_CUDA_TEST_MLP128=0	Original kernel path: baseline
TF_CUDA_TEST_MLP128=1	Candidate path on SM120, including other SM counts
Unset	Existing automatic policy: candidate only on SM120/82 SMs

Other architectures cannot be forced on. Model/shape, precision, KV, context and graph guards remain in place. For attention, wider windows are still selected only at 8192<=cached<65536; the override does not force every attention shape. The flags are read at startup, so restart the server between arms. Exporting the flag only in the benchmark client does nothing.

Commands below assume Linux/Bash.

1. Isolated checkout and one build

git clone --filter=blob:none --no-checkout https://github.com/ashhart/TensorFold.git tensorfold-pr1730-test
cd tensorfold-pr1730-test
git fetch origin pull/1730/head
git checkout --detach FETCH_HEAD
git rev-parse HEAD
zig build native -Dsm=120 -Dnvcc="$(command -v nvcc)" -Dversion=pr1730-test -j4

Use Zig 0.17.0 and preferably the same CUDA 13.0 or 13.3 toolchain for all arms; record full versions. Reuse your qualified grammar/media library setup. Use the same TensorFold/Qwen3.8-27B-MLX-4bit revision 22d8d538154e0e5b6f8dcb1fcc74a60a0798104c and BF16 z-lab/Qwen3.8-27B-DFlash2 revision 50307d4c4cde6860d4eee73e2547cd786fe8e8a4 throughout.

2. Server terminal: baseline, candidate, baseline again

Set MODEL_DIR and DRAFTER_DIR to your existing checkpoint directories. Stop other GPU workloads during each measured arm. Start with 0, then repeat with 1, then 0 again. Set ARM respectively to baseline, candidate, baseline-repeat; each invocation gets fresh calibration storage.

unset TF_CUDA_TEST_MLP128 TF_CUDA_TEST_ATTN_MT2
MODEL_DIR=/path/to/Qwen3.8-27B-MLX-4bit
DRAFTER_DIR=/path/to/Qwen3.8-27B-DFlash2
ARM=baseline
TF_CUDA_TEST_MLP128=0 TENSORFOLD_MEMORY_RESERVE_GIB=0.5 HOME="$(mktemp -d)"   ./zig-out/native/bin/tensorfold-native serve "$MODEL_DIR"   --drafter "$DRAFTER_DIR" --name bench --host 127.0.0.1 --port 18080   --context 131072 --parallel 1 --kv-cache fp8 --prompt-precision int8   --vision --prompt-cache-gib 8 --max-tokens 2048   >"$ARM-server.log" 2>&1

Confirm the log says experimental MLP128 override=0, enabled=false for baseline and override=1, enabled=true for candidate. An enabled device flag permits the candidate only on its eligible shapes; it does not mean every launch uses it. Keep the same power settings, context, vision, caches and precision in all arms. If this configuration cannot fit, report that rather than silently reducing features in one arm.

3. Client terminal: identical sequence for every arm

Wait until /health is ready and warming is false. Set ARM to match the server terminal, then run:

ARM=baseline
RECEIPT=docs/receipts/rtx-pro-4500-affine-tile-20261030
python3 tools/bench_openai.py http://127.0.0.1:18080 bench   --tokens 256 --reps 5 --temperatures 0,1 --output "$ARM-short.json"
python3 "$RECEIPT/testing/replay_long.py" http://127.0.0.1:18080   "$RECEIPT/testing/fixture-8192.json" --model bench --repetitions 3   --output "$ARM-long8k.json"
python3 "$RECEIPT/testing/replay_long.py" http://127.0.0.1:18080   "$RECEIPT/testing/fixture-32768.json" --model bench --repetitions 3   --output "$ARM-long32k.json"

The long replay runs both draft=true and draft=false, checks repeated hashes, and asserts drafted/plain token equality. After all three arms, also check baseline/candidate hashes:

python3 - <<'CHECK'
import json
for size in ['8k', '32k']:
    reference = None
    for arm in ['baseline', 'candidate', 'baseline-repeat']:
        samples = json.load(open(f'{arm}-long{size}.json'))['samples']
        hashes = {(x['drafted'], x['repeat']): x['response']['tensorfold']['token_sha256'] for x in samples}
        if reference is None:
            reference = hashes
        assert hashes == reference, (size, arm, 'token mismatch')
print('Long-fixture hashes match across all three arms.')
CHECK

Please attach

GPU model/edition and power limit; driver, nvcc, Zig, OS and Git SHA; checkpoint revisions and unchanged serving settings; all nine benchmark JSON files and the three server logs. Include the short benchmark's evidence hashes, not just headline tok/s. Report short, 8K, 32K, plain and drafted results separately, including regressions. For the long runs use median server decode tok/s; compare candidate against the geometric mean of the two baseline medians. Report prefill/TTFT separately, and keep cached-token counts visible.

This is a focused performance/exactness request, not a request to run an hours-long full suite. The PR remains draft: full backend contract SUMMARY qualification and the other documented release gates are still outstanding. No GPU run of this newly added override has been performed locally; pure override tests and targeted host checks passed. The original 4500 receipts remain pinned to their original measured commits. Please report failures without disabling hash checks or changing precision.

[ashhart]

Thanks @Maxrio9. The six alternating groups answer our noise question: +0.825% with every token equal, and the replacement case checks the new tile's bytes against the 64-column tile. On the limit records: the 223 problems are main's records checked against nvcc 13.3, and your main-only control shows they aren't yours. The release fatbins are built with nvcc 13.0, so that's the run that gates this. Could you post the full CUDA contract SUMMARY built with 13.0 (your receipts show you have it), and file the MLP128 base record from that compile rather than 13.3's? We'll add main's missing 13.3 set ourselves as a records-only PR. Two smaller things: please keep the receipts on your branch with a short README and hashes in the PR (193 files is a lot for the repo), and either drop TF_CUDA_TEST_MLP128 from the landing commits or refuse values other than 0 and 1 by name. The test-fixture fix you mention would be welcome as its own PR.

[Maxrio9]

Thanks. Everything you asked for, in order:

nvcc 13.0 SUMMARY (current main 37a4a84a, nvcc 13.0.88, SM120/82 SMs). One combined test binary (the fixture fix from #1845 + this PR + #1731), plus an untouched-main control built the same way:

SUMMARY 511 kernels: 511 pass gates 1-4, 0 fail, 8 known findings; 340 controls: 340 red as planted, 0 missed; 213 limit problems; gates 5503142 ms, limits 1037963 ms

MLP128 record. Compiled with 13.0 it is 54 registers, 0 static + 26,880 dynamic shared bytes, 256 threads, which equals the base record already in this PR, so nothing changed. It passes its live limit check (PASS limits live qmm_group MLP128 ... (recorded 54, 0)).

The 213 limit problems are still there under 13.0, and they are main's. The control (untouched main + a 5-line limits-only entry point, same compiler) reports LIMITS_SUMMARY: 213 problems with the same 168 missing static records and 45 live mismatches. The sorted failure lists are identical apart from the combined build registering 1387 launches against main's 1384, the three new ones, which pass. They are in qwen4_*, nvfp4_experts_tile and the older qwen35 attention kernels. So 13.0 doesn't turn the backend gate green for me; if your CI reaches a clean result with a different entry point or flags, tell me which and I'll run that.

Receipts. Moved to the receipts/pro4500-affine-tile branch on my fork, with a short README and SHA-256 list for this run; the PR branch now carries only the change (15 files). I didn't force-push, so the old receipt commits are still in this branch's history. If you squash-merge that is moot; otherwise tell me and I'll rewrite the branch.

TF_CUDA_TEST_MLP128. Kept, but it now accepts only 0 or 1 and refuses anything else by name (TF_CUDA_TEST_MLP128 must be 0 or 1, not '<value>', error InvalidTestOverride), with a unit test. It shares sm120_test_override.zig with #1731.

Test-fixture fix. Own PR: #1845.

Scope, plainly: the run was on a combined binary, with a graphical desktop active (4 GiB reserve) rather than headless; the contract is deterministic and has no timing gates, but it isn't a headless receipt. No served benchmark was rerun on the final commit; the +0.825% result was measured on 94b90c9d with the same kernel.

[ashhart]

Thanks @Maxrio9 for the full receipts. This conflicts with main now. Could you rebase onto current main and mark it ready when you're happy with it? Then we'll check 7b97871 and 024d84b against the 1 Nov asks and land it with #1731.

[Maxrio9]

@ashhart this is ready for your check. I merged main into the branch rather than rebasing (a rebase would need a force-push), so the conflict in pending_133.zig is resolved, keeping both records. Head is now 3601368f, merged with main at c7952f39. Main moves quickly, so if it conflicts again by the time you get here, please say and I'll merge again.

The lean check reports 424 problems, the same as main's, so nothing new. The zig, zig-windows, desktop, desktop-linux, appimage-linux and fmt jobs passed on the previous head (49ca2bb3). The zig-macos-dev-tools job on that head shows as cancelled after about 2.5 hours on a macOS runner, not as a test failure, so it has no result. If you re-run it, that completes the set. I have no re-run rights on a fork.

It is still a draft; I'll mark it ready for review on your word. It lands with #1731.

[ashhart]

Thanks @Maxrio9. Everything from 1 Nov is in. TF_CUDA_TEST_MLP128 refuses anything but 0 or 1 by name, with a test, and the MLP128 record matches the 13.0 compile. We've taken it out of draft and are re-running the cancelled CI. Once it's green we'll squash it and land it together with #1731. Both add the same sm120_test_override.zig byte for byte, so they won't conflict.

[ashhart]

Thank you @Maxrio9. This is merged into main with your commits under your own name, so you're listed as a TensorFold contributor. It ships in the next release, credited to you in the changelog and the release notes.

[ashhart]

Thanks @Maxrio9! This is in TensorFold 1.2.3, out now: https://github.com/ashhart/TensorFold/releases/tag/v1.2.3