[bong-water-water-bong]

Re-evaluated 2026-11-01 12:25 AST, after #764 merged (d3d2667a6) — kept open, and this PR is now the belt-and-braces

#764 is on main. It refuses a load before any backend starts when GTT used now + estimate > GTT total - 16 GiB
(= 104 GiB of 120, GGML_HRX_GTT_RESERVE_MB), and its estimate is good: GGUF-header based, no weights read, and
it does include the drafter (-md is added as draft <weights+kv+state>, app/mem_admit.h:312), plus mmproj, KV,
recurrent state and a 512 MiB compute buffer.

It would not have refused this arm, and that is a measurement, not an opinion. The engine's own estimator on the
shape that faulted (tools/mem_budget.py model, main):

Qwen3.8-27B-UD-Q4_K_XL.gguf  arch=qwen35  ctx=20480  slots=1
  weights          16.34 GiB
  KV + state        5.08 GiB
  compute           0.50 GiB
  program cache     4.00 GiB
  extras            1.12 GiB   [the DFlash2 q4k drafter]
  NEED             27.05 GiB
verdict: FITS

27.05 GiB against a 104 GiB ceiling — and the shipped judge is even smaller (it carries the program cache as a
warning, not a term): 23.04 GiB. There is no box state in which #764 refuses this arm, so it remains admissible and
the 11:13 fault would still happen. #764's wall is at 104 GiB; this fault happened at a load four times below it.

The two issues guard different things and neither subsumes the other:

	#764 (merged)	this PR
what it sees	the size of the load, at launch	the lever combination, before a load
failure it prevents	landing past 104 GiB and faulting on allocation	a small load that faults at the first decode step because drafting is on at a long context
depends on box state	yes (GTT used now)	no
depends on the estimate being right	yes	no

This is also why #764 cannot absorb it by construction: the arm was a lever interaction, not a magnitude. The
18,758-token prefill completed clean at 252.63 tok/s and the fault came at the first decode step with
[spec] failed to measure draft model memory: failed to create llama_context from model already in the load log — i.e.
the piece that grew at decode could not be sized at load. A pre-launch ceiling check has no way to see that.

So: kept open, diff unchanged (2 files, +32/-2). What it adds on top of #764 is that a candidate costing a queue slot instead of
a reset does not depend on a number being right.

When this can be closed (and the PR closed with it)

Either of these, measured:

1. the drafter-at-long-context arm runs clean on this box with serve: refuse a load that would land past the GTT ceiling (engine #680) #764 + fork MoE streaming: Qwen3.8-Flash-Next streamed and measured; serve maps per-layer embeddings #166's allocation-time backstop in place
   (and stays clean over repeats), or
2. llama.cpp can size a DFlash/MTP drafter at load — no [spec] failed to measure draft model memory warning — so the
   decode-time growth of the arm is inside the estimate that serve: refuse a load that would land past the GTT ceiling (engine #680) #764 uses.

Until then the adapter refusal in this PR stays, and it is the cheap half of the box rule: it fails before a load, so it cannot
contribute to a reset.

---

The box rule from the 2026-11-01 12:13 all-clear: no long-context run near the GTT ceiling until #764's guardrail is merged. This is the adapter half of that rule for the lever that actually faulted.

Why

The arm in flight at the 11:13 reset was replay_warm_recipe = HL_DFLASH=1 HL_MTP_MAX=6 at HL_CTX=20480 / HL_LONGCTX=18800 on Qwen3.8-27B UD-Q4_K_XL: a clean 18,758-token prefill at 252.63 tok/s, then HSA_STATUS_ERROR_MEMORY_FAULT at the first decode step, hrx-system … system_event.c:122: ABORTED; AMDGPU memory access fault, llama_decode ret=-3; the box then went to a MODE2 reset ~3.5 min later. At load the same run had warned it could not size the drafter ([spec] failed to measure draft model memory: failed to create llama_context from model). Full logs are in the lane state file.

So the short-prompt drafter arm (+13.8 % geomean, gate PASS) stays, and the drafter at long context is refused until the #764 admission guardrail can refuse a bad load before it faults.

What changed

tools/hyperloom/custom/benchmark.py, two small pieces:

• dflash_on() — one definition of "the DFlash drafter is requested" (HL_DFLASH truthy and not 0/off/false), used by serve_lever_args() where it was an inline expression before.
• longctx_drafter_conflict() — HL_LONGCTX>0 and drafting on. main() raises SystemExit on it before the model or the drafter is loaded, so the tuner scores that candidate zero with the reason instead of measuring a shape the box rule forbids.

tests/hyperloom_benchmark_test.py covers the truth table (long+dflash → refused, long alone → allowed, dflash alone → allowed) and passes:

hyperloom_benchmark_test: OK     (python3 tests/hyperloom_benchmark_test.py, rc=0)

No grid change, no lever semantics change, nothing GPU-side: the candidate that is refused fails before a load, which is the point — it costs a queue slot, not a reset.

[github-actions]

PR Reviewer Guide 🔍

(Review updated until commit 48dd962)

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶 764 - Partially compliant Compliant requirements: Implementation of the GTT ceiling check before load Inclusion of drafter in the estimation Consistent behavior with ggml-hrx fork Documentation updates in docs/serve.md Tests added in tests/mem_admit_test.cpp and hyperloom_benchmark_test.py Non-compliant requirements: None Requires further human verification: Verification that the specific fault scenario (Qwen3.8-27B at 18.8k context) is properly handled Confirmation that the box rule is correctly applied in all relevant scenarios

⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Box Rule Enforcement The new box rule enforcement in longctx_drafter_conflict() function may be overly restrictive. It prevents any long-context measurement with drafter enabled, even when the specific combination might not actually cause a fault. This could prevent valid experiments from running, especially if the engine#764 guardrail is not yet merged but the box rule is already in place. def longctx_drafter_conflict(): """The box rule (2026-11-01 12:13): no long-context measurement may draft. The drafter at long context is the arm that faulted the HRX graph on this box (27B at 18.8k, the 11:13 MODE2 reset), so it is refused until engine#764's admission guardrail is merged. A refused candidate scores zero and says why, rather than loading a drafter nobody may measure. """ return env_int("HL_LONGCTX", 0) > 0 and dflash_on() Early Exit Logic The early exit logic in main() function that raises SystemExit based on longctx_drafter_conflict() could be improved to provide more context about why the experiment is being refused, potentially including the specific GTT ceiling values and the estimated memory usage. def main(): started = time.time() # Box rule first: nothing else in this run may even resolve if it asks for the forbidden shape. if longctx_drafter_conflict(): raise SystemExit( "refusing HL_DFLASH with HL_LONGCTX>0: the drafter at long context is the arm that faulted the " "HRX graph on this box (27B, 18.8k, 2026-11-01 11:13 reset) and the box rule forbids it until " "engine#764's refusal guardrail is merged")

[bong-water-water-bong]

VERDICT: READY

Ran it, and the guard behaves:

$ python3 tests/hyperloom_benchmark_test.py
hyperloom_benchmark_test: OK                                        rc 0

$ HL_LONGCTX=18800 HL_DFLASH=1 python3 tools/hyperloom/custom/benchmark.py
refusing HL_DFLASH with HL_LONGCTX>0: the drafter at long context is the arm that faulted the HRX graph …
                                                                    exit 1

$ HL_LONGCTX=18800 … (no HL_DFLASH)      longctx_drafter_conflict() -> False
$ HL_DFLASH=0                           dflash_on()               -> False
$ RESULT_DIR unset in the refused run   no directory created     (the refusal is before anything resolves)

So: the refusal exits 1 with the reason, it fires before RESULT_DIR, before any binary is located and before any
model is loaded, and it does not refuse a long-context run that simply has no drafter. dflash_on() also fixes a real
inconsistency — HL_DFLASH=0 used to be truthy in the caller (if dflash and dflash not in ("0","off","false") was
already right, but the same string was read twice, in two places, by two rules; now there is one predicate).

Two strings are now stale, and they are what a user will read first. Both say the guard is temporary "until
engine#764's refusal guardrail is merged" — #764 merged at 12:20 AST (d3d2667a), and the PR's own re-evaluation says
the opposite of what the message implies: #764 would not have refused this arm (27.05 GiB need against a 104 GiB
ceiling), so the guard stays for its own reason, not for a merge that has happened.

1. the SystemExit message in main();
2. the longctx_drafter_conflict() docstring ("refused until engine#764's admission guardrail is merged").

The accurate wording is the one already in your body: the arm faulted at the first decode step with the DFlash2 drafter
and #764's load-time ceiling cannot see it, so the refusal stands until the arm runs clean with #764 + fork #166's
allocation-time backstop, or until llama.cpp can size a DFlash/MTP drafter at load.

One scope question, not a blocker: the docstring says "no long-context measurement may draft", but only
HL_DFLASH is refused — HL_MTP_MAX passes, and MTP is on by default in this workload (levers.json has mtp_off
as the lever that turns it off, plus mtp_3/6/8). If MTP drafting at 18.8k is also out, that is one more clause; if
the fault really was the external DFlash drafter (extras 1.12 GiB in your estimate, and
[spec] failed to measure draft model memory is DFlash's warning), then the docstring should say DFlash rather than
"may draft".

I am not withholding the merge on any of that: the behaviour is correct and conservative, and a candidate refused here
costs a queue slot instead of a reset. Fix the two strings in a follow-up. tprs-ready applied.

[github-actions]

Persistent review updated to latest commit 827758a

[bong-water-water-bong]

VERDICT: READY

Confirmation at 827758a9: Merge branch 'main' into 1bit/hyperloom-longctx-nodraft, both author commits unchanged
(8d25319, 857efff), own diff still 2 files +32 −2. Main's 1d42bad brought no change to these two files, and the
behaviour I ran at the previous head stands (refusal exits 1 before RESULT_DIR/binaries/model; long-context without a
drafter passes the guard; HL_DFLASH=0 is off).

The two stale strings are still there in this head — both say the refusal stands "until engine#764's refusal guardrail
is merged", and #764 merged at 12:20 AST (d3d2667a). The reason to keep the refusal is the one in this PR's own
re-evaluation (#764's ceiling is 104 GiB and the faulting arm needed 27 GiB, so it would not have been refused), not a
merge that has happened. Text-only; tprs-ready stays.

[github-actions]

Persistent review updated to latest commit f0e9c7f

[bong-water-water-bong]

VERDICT: READY

Confirmation at f0e9c7f4: Merge branch 'main' bringing #766's gitlink; both author commits (8d25319, 857efff)
unchanged, so the guard I ran still reads the same (refusal exits 1 before RESULT_DIR/binaries/model; long-context
without a drafter passes; HL_DFLASH=0 is off).

The two stale strings are still in this head — the SystemExit message and longctx_drafter_conflict()'s docstring both
say the refusal stands "until engine#764's refusal guardrail is merged", and #764 merged at 12:20 AST. Text-only; label
stays.

[bong-water-water-bong]

VERDICT: READY

Confirmation at fbd76f54: merge-of-main, both author commits (8d25319, 857efff) unchanged, diff still 2 files
+32 −2. The guard behaves as I ran it (refusal exits 1 before RESULT_DIR/binaries/model; long-context without a
drafter passes; HL_DFLASH=0 off). The two stale "until engine#764 is merged" strings are still in this head —
text-only, and #764 merged at 12:20 AST. Label stays.

[github-actions]

Persistent review updated to latest commit 359ec20

[bong-water-water-bong]

VERDICT: READY

Confirmation at 359ec20d: Merge branch 'main' (bringing #768's init.sh change into the branch), both author commits
(8d25319, 857efff) unchanged, so the guard is what I ran — refusal exits 1 before RESULT_DIR/binaries/model,
long-context without a drafter passes, HL_DFLASH=0 is off.

The two stale strings are still in this head (message and docstring both say "until engine#764 is merged"; #764 merged at
12:20 AST). Text-only; tprs-ready stays.

[bong-water-water-bong]

VERDICT: READY

Confirmation at 2b5cf6d0 after update-branch: their own files are byte-identical to the head I verdicted — verified with
git diff --stat <previous head> 2b5cf6d0 -- <this PR's paths> → empty, the merge brought #768's and #775's squashes
(7f4e98a, e55096b) and nothing else. Content as reviewed (hyperloom), build green. tprs-ready stays.

[github-actions]

Persistent review updated to latest commit c4f4bb6

[bong-water-water-bong]

VERDICT: READY

Confirmation at c4f4bb67: their own paths are byte-identical to the head I verdicted
(git diff --stat 2b5cf6d0 c4f4bb67 -- tools/hyperloom tests/hyperloom_benchmark_test.py → empty), so the guard is
still the one I ran (refusal exits 1 before RESULT_DIR/binaries/model; long-context without a drafter passes;
HL_DFLASH=0 off). The two stale "until #764 is merged" strings are unchanged — text-only. tprs-ready stays.

[bong-water-water-bong]

VERDICT: READY

Confirmation at dbf5704a after the waiter's rebase onto main (90c998a = #776 merged): this PR's own paths are
byte-identical to the head I verdicted — git diff --stat <previous head> dbf5704a -- <its paths> → empty. Content as
reviewed, build green. tprs-ready stays.

[bong-water-water-bong]

VERDICT: READY

Confirmation at 48dd962dae398810290682890e2a19f070a444e4, update-branched and green. This PR's own paths are byte-identical to the content I
reviewed (git diff --stat against the last head I verdicted → empty), so the guard still behaves as I ran it: the
refusal exits 1 before RESULT_DIR/binaries/model, long-context without a drafter passes, HL_DFLASH=0 is off.
tprs-ready stays.

[github-actions]

Persistent review updated to latest commit 48dd962