Repository navigation
STRATA_QFUSE=1 changes the verified text and lowers MTP acceptance (not repeatable; RTX 3080, sm_86) #3948
Description
Summary
With STRATA_QFUSE=1, the verified output text changes when MTP speculative decoding is on, and it does not repeat between two engines with the same settings. Draft acceptance also drops. With STRATA_QFUSE unset, the same setup is stable. This looks like a race or a stale buffer in the fused q8_1 quantise path, not a summation-order difference.
Setup
- RTX 3080 10 GB (sm_86), CUDA 13.0, Linux.
- Strata from a fork based on upstream
477ac13c(2026-10-16). The QFUSE code is unchanged from upstream. - Qwen3.8-Flash-Next, Q2_0 experts, MTP drafts (
--spec 6,--spec-min-p 0.7;STRATA_SPEC_DEPTH=1for this test), int8 KV. - Each arm starts a fresh engine and runs 40 chat prompts at temperature 0. We dump the top-20 logprobs of the first 200 tokens of each reply.
Result
| pair | replies identical (of 40) | mean KL | first divergence (median token position) |
|---|---|---|---|
| QFUSE off vs QFUSE off | 16 | 0.043 | 7 |
| QFUSE on vs QFUSE on | 6 | 0.108 | 3 |
| QFUSE off vs QFUSE on (two pairs) | 5 / 5 | 0.161 / 0.153 | 2 / 3 |
- Drafts accepted/offered: 0.799 / 0.791 with QFUSE off, and 0.795 / 0.755 with QFUSE on. With QFUSE on, far fewer drafts were offered: about 2,000-2,400 against about 4,170.
- Earlier runs with QFUSE on gave about 1.67 tokens per verify window, against about 2.8 with it off. Chat speed varied from 12 to 142 tok/s between replies.
- Decision-style requests that do not draft gave byte-identical results with QFUSE on and off.
Where we think it is
We read the code only and did not run gr_parity. With qcnt_ set, the hc read writes the q8_1 image of mixed into xq_ (attention half) and nat_xq_ (expert half), and the expert half then skips its own quantize_q8_1_rows (verify.cpp around lines 1318-1322, 1894 and 2045-2049). The per-group "last block" counter in fused_gr.cu (around lines 218-250) is our first suspect. There are two candidates:
- (a) A counter is not reset when a captured graph replays with a different T, so the last block quantises before every partial has landed.
- (b) A window of more than one group (G = 2) uses
nat_xq_offsets that the counter does not cover.
The next check would be gr_parity with STRATA_QFUSE=1 at T = 1 to 8 and with two groups, comparing xq_ and nat_xq_ against the separate quantise.
We keep STRATA_QFUSE off. We are happy to run a test build or a parity case on sm_86 if that helps.
Activity
github-actions commentedon Oct 24, 2026
Thanks for the issue. To keep the tracker readable, every item needs an area prefix in the title and the parts of the issue form filled in. This one is missing:
Edit the issue (title and description) and this note goes away on its own. If it is still missing parts in 5 days it will be closed, which is not a rejection: fix it and comment
/reopen.Details: CONTRIBUTING.md
tests: STRATA_QFUSE parity for a window of two groups and for the GDN…
QFUSE: the fused q8_1 producers divide the way the separate quantiser…
Merge f-027-gh-eng-d (#3602 #3948 #3638 #3702 #3797 #2409 #3844: audi…
STRATA_QFUSE=1: a one-token window did not self-commit, but commit() …
Merge f-027-gh-eng: GitHub engine items for 0.2.7
Niko1221 commentedon Oct 27, 2026
Thank you. Fixed in 0.2.7: STRATA_QFUSE parity for a window of two groups and for the GDN q8_1 destination. We could check it with unit tests and builds, but not on your hardware or with your workload. Please try 0.2.7 and tell us how it goes.
Release: https://github.com/Niko1221/Strata/releases/tag/v0.2.7
integ: restore the fork's fused_gr HIP perf line on top of 0.2.7
integ: fused_gr q32 tail divides with q81_div like every other produc…
sycl: re-migrate the port to upstream 0.1.54 (from its last re-migrat…
F-369: v0.2.7 perf/parity port set (upstream d18d8ba + 7d1016f)