hrx: host-weight arena refusal is fragmentation, but the message reports a size problem and recommends a budget that cannot be reached

The refusal path added by #788 reports a size problem and tells the reader to raise the budget, but the actual
failure in arena_reserve() is fragmentation: free space ≥ the wanted slot, just split across blocks. Two
consequences, both measured below: the printed advice is self-contradictory in the case that matters, and no value of
GGML_HRX_HOST_WEIGHT_LIMIT_MB makes a partial-offload model work, because raising it raises resident too — the
working set tracks the budget and the guard refuses again at the next weight.

The smoking gun (`-v`, cheap repro, Qwen3-0.6B-Q4_K_M, budget 200 MiB)

arena_reserve: host weight arena is full after eviction: 0.0 MiB of 200.0 MiB resident,
               193.2 MiB free in 5 blocks, 121.7 MiB wanted (GGML_HRX_HOST_WEIGHT_LIMIT_MB)
hrx: refusing to run: GGML_HRX_HOST_WEIGHT_LIMIT_MB=200 MiB cannot hold this model's per-execution working set -
     a 121 MiB weight does not fit with 0 MiB resident and 205 MiB parked for the running execution.
     Raise it to at least 328 MiB or unset it (engine #788).

193.2 MiB free ≥ 121.7 MiB wanted — the failure is that no single block is 121.7 MiB. The user-facing message never
says that; it says the model's working set cannot be held, which is a different (and wrong) claim about this state.

The contradiction on a real model (Qwen3.8-Flash-Next UD-IQ3_XXS, 76.32 GiB, pin `d8bd0ff6052`)

-dev HRX0 -ngl 20 -fa on -ub 512 -p 512 -n 0 -r 2, only GGML_HRX_HOST_WEIGHT_LIMIT_MB changing:

budget	message	the message's own recommendation vs the budget
4096	`a 497 MiB weight does not fit with 1983 MiB resident and 1549 MiB parked ... Raise it to at least 4030 MiB`	4030 < 4096 — it refuses while recommending a budget below the one in force
8192	`a 497 MiB weight does not fit with 5595 MiB resident and 2113 MiB parked ... Raise it to at least 8208 MiB`	8208 > 8192, but `resident` grew 1983 → 5595 purely because the budget grew

So the "raise it to at least X" instruction is not a fixed point: X is computed from need + resident + parked sampled at
the failure, and both terms grow with the budget. Following the advice moves the refusal forward by one weight.

Why this matters beyond the message

GGML_HRX_HOST_WEIGHT_LIMIT_MB is the only knob for the partial-offload shape, which is the only shape a 217 GiB model
(DeepSeek-V4.1 UD-IQ1_S, our lane) can take on a 124 GiB box. With -ngl 99 there is no host working set and the guard
cannot fire — that is the workaround and it is worth stating in the docs — but everything below full offload currently
dies at the first fragmented miss and reports it as a capacity problem.

Proposed fix (diagnostic only, no behaviour change)

In the refusal at ggml/src/ggml-hrx/runtime/host-memory.cpp:1963 (in the fork, third_party/llama.cpp) (and the twin result.status.log at :1969), report the reserve outcome
that actually failed and name fragmentation when it applies, e.g.

host weight arena could not place a %zu MiB slot: %zu MiB free in %zu blocks, largest %zu MiB
(arena %.1f MiB of %.1f MiB limit) — %s

with the last part being "free space is fragmented; a larger GGML_HRX_HOST_WEIGHT_LIMIT_MB may help" when free >= need,
versus the current capacity wording when it does not. A follow-up worth considering is coalescing adjacent free blocks
before refusing (the eviction path already walks the slots), which would make mid-size budgets usable at all.

Happy to send that patch if the owner of #669/#788's file prefers, since this is the same file and I have not touched it.

Found while measuring DeepSeek-V4.1-Flash / Qwen3.8-Flash-Next on strixhalo (Radeon 8060S, gfx1151, UMA). Reproducer one-liners
for both models are in this comment; nothing else was changed to produce them.

Confirmed at the pin, and there is one useful detail the report does not mention: the block count already exists one function away.

At third_party/llama.cpp = d8bd0ff6052a (the pin; engine 65460184), ggml/src/ggml-hrx/runtime/host-memory.cpp:

:1570-1597 — arena_reserve's local eviction path builds a merged blocks vector and its comment names this exact failure: "Count freed contiguous space, not freed bytes: variable-size slots mean 366 MiB free in 3 blocks cannot serve a 304 MiB request". It already knows free, the block count and (by walking fits, :1597-1600) the largest block.
:1696-1697 — that knowledge is printed, but only internally: host weight arena is full after eviction: %.1f MiB of %.1f MiB resident, %.1f MiB free in %zu blocks, %.1f MiB wanted.
:1963-1966 (GGML_LOG_ERROR) and :1969-1972 (result.status.log) — the user-facing refusal says the capacity thing instead: "cannot hold this model's per-execution working set … Raise it to at least %zu MiB". The status line carries free %.1f MiB but no block count, so the one number that identifies fragmentation (free ≥ need) is present and not used.

So the diagnostic fix is smaller than the report assumes: the message does not need new bookkeeping, only arena_free_bytes_ >= refusal_need_ to choose the wording (both are members at the refusal site). Naming the largest block is the one extra out-parameter from arena_reserve (blocks is local to it).

Labels stay bug + hrx. The file is the fork's, so the fix is a llama.cpp PR plus an engine pin bump — an A row, same shape as #841.

Owner: the hrx-study lane that filed it, which offered the patch. Two things I would ask of that patch, both cheap: (1) the advice sentence must not tell a reader to raise a budget that a smaller budget already failed under — the report's 4030 < 4096 row is the proof that the current text is self-contradictory; (2) keep the coalescing follow-up (:1579-1596 already merges adjacent blocks on insert) out of the same change, or it becomes a behaviour change and a much longer review.

What I will do: watch fork PRs for host-memory.cpp and re-run the report's own 200-MiB Qwen3-0.6B reproducer against the merged fork head before closing this.

The fix is written and it is in the fork: llama.cpp#180 (head `de4c2b22e`)

Diagnostic half only, exactly what this issue asks for; the coalescing follow-up is explicitly not in it.

The predicate, read at the fork pin d8bd0ff6052a: at the failure site the arena's free list is
arena_free_ (runtime/host-memory.cpp: the take scans it at arena_take_locked, the eviction path merges
victims into a model of it), so free_bytes >= need && largest_block < need is available two lines from where
the message is formatted — and it was unused. New runtime/host-memory-refusal.h holds the verdict and the
sentence (no HRX dependency, so it is testable without the private runtime); describe() is now the single
source of the text for both the GGML_LOG_ERROR and the twin result.status.log.

What changes for the reader, on this issue's own two measured states:

state	before	after
200 MiB budget, 193.2 free in 5 blocks, 121.7 wanted	"cannot hold this model's per-execution working set … Raise it to at least 328 MiB"	names fragmentation: 193.2 MiB free in 5 blocks, largest block, arena/limit/resident/parked — no raise-to number at all
4096 MiB budget, 497 MiB weight, 1983 resident / 1549 parked	"Raise it to at least 4030 MiB" at a budget of 4096	same fragmentation wording, no number
genuine capacity miss (free < need)	engine #788 wording	unchanged, and its arithmetic asserted (`500+1983+1549 → 4032`)

Two things the report did not ask for but the same defect covers, both in the PR:

arena_reserve's own line called a fragmented arena "full after eviction" — the line this issue quotes as
the smoking gun. It now says which slot it could not place, in how many blocks, and the largest block.
The latch repeated the wrong claim. budget_refusal_ stored only the size, so every later acquire in the
same run re-derived a capacity sentence it never measured. It now stores the whole miss (ArenaMiss) and
repeats the first refusal's reason.

Tests: tests/test-hrx-arena-refusal.cpp, 5 arms — the two states above, a genuine capacity miss, the
unbounded default (limit = 0), and the corner where a large enough free block does exist (not called
fragmentation). All pass; the same arms against the old wording fail 9 times, so the test is not a rubber stamp.

What closes this issue: fork #180 merges and the engine's third_party/llama.cpp pin moves past it — as
the row says, the change lives in the submodule, so a merged fork PR alone does not close it.

A host-weight slot that is parked while an execution is in flight (arena_deferred_) is not dropped when the arena
is re-created, so it is reclaimed into the new arena's free list at its old offset — interleaving with (and able to
overlap) live allocations. #900's fragmented refusal is the visible symptom; the defect is one extent handed out twice.

Measured on this box (the pin's own binary, `d8bd0ff6052a` + the #788 arena)

llama-cli, Qwen3-0.6B-Q4_K_M, -dev HRX0 -ngl 0 -fit off -c 256 --temp 0 -n 4 -no-cnv -v,
GGML_HRX_HOST_WEIGHT_TRACE=1, trio HSA_USE_SVM=0 HSA_USERPTR_FOR_PAGED_MEM=0 HRX_ALLOW_NO_SVM=1, one gpu-hold
ticket, 6 s:

### base-200
    0.00.939.841 I hrx host weights: arena grown to 128.0 MiB for a 121.7 MiB weight
    0.01.085.134 I hrx host weights: arena grown to 200.0 MiB for a 1.7 MiB weight (all entries dropped)
    0.01.430.051 I hrx host weights: arena grown to 200.0 MiB for a 121.7 MiB weight (all entries dropped)
    0.01.758.912 E arena_reserve: host weight arena is full after eviction: 0.0 MiB of 200.0 MiB resident,
                    193.2 MiB free in 5 blocks, 121.7 MiB wanted (GGML_HRX_HOST_WEIGHT_LIMIT_MB)
    0.01.758.919 E hrx: refusing to run: ... 0 MiB resident and 205 MiB parked ...
### base-240   ... 0.0 MiB of 240.0 MiB resident, 233.1 MiB free in 4 blocks, 121.7 MiB wanted; 0 resident, 245 parked
### base-320   ... 0.0 MiB of 320.0 MiB resident,  72.9 MiB free in 2 blocks, 121.7 MiB wanted; 0 resident, 247 parked
### base-none  runs (no refusal)

Two things in that log are impossible without the defect: 205 MiB parked in a 200 MiB arena (same at 240: 245), and
193.2 MiB free in 5 blocks in a 200 MiB arena whose only live weight is 121.7 MiB.

Mechanism, read at the pin

arena_reserve() (ggml/src/ggml-hrx/runtime/host-memory.cpp:1683) only evicts-or-refuses when
resident_bytes_ != 0. A run whose resident weights have all been parked has resident_bytes_ == 0, so a miss
falls through to the grow path — even when the arena is already at the budget.
The grow path (:1704-1775, target = budget at :1718) re-creates the arena at target = budget when previous_capacity != 0, and resets
entries_, last_use_, stats_.resident_bytes, resident_bytes_, arena_free_, arena_free_bytes_. It does
not reset arena_deferred_. That is the two grown to 200.0 MiB ... (all entries dropped) lines 350 ms apart.
end_execution() (:1517-1522) then hands the parked old-arena extents to arena_reclaim_locked(), which inserts
them into the new free list and merges them with their neighbours there. The offsets are the ones they had in the
released arena, so they interleave with the live 121.7 MiB block: 5 non-adjacent free blocks whose largest is below
121.7 MiB, a refusal on a weight that fits, and an inflated arena_free_bytes_.
Had the miss been for a size a stale extent could serve, arena_take_locked() (:1644) would return an extent
overlapping a live weight: two weights, one set of bytes. Silent corruption, not a wrong number.

So #900's suggested follow-up ("coalescing adjacent free blocks before refusing") would not have helped: adjacent-block
merging already exists, in both the reclaim path (:1538-1553) and the eviction model (:1571-1600). The free blocks
are genuinely non-adjacent — they are the stale extents.

Fix

Drop the parked list with the arena that parked it, in the same place that drops entries_/last_use_/the free list —
the old buffer is released and every entry is dropped, so no lease can still reference a parked extent (eviction only
takes entries with use_count() == 1). PR: llama.cpp#182 (fee91177d).

Measured with the fix, same command, same ticket, budgets 200/210/220/230/240 (resident + parked + free vs limit):

budget	base	fix
200	0 + 205 + 193.2 = 398.2, refuses a 121.7 MiB weight	70.0 + 128 + 1.3 = 199.3, admits it (refuses a 1.7 MiB weight later)
210	0 + 215 + 203.2 = 418.2, refuses 121.7	80.9 + 128 + 0.4 = 209.3, admits it
220	0 + 224 + 214.9 = 438.9, refuses 121.7	92.7 + 127 + 0.2 = 219.9, admits it
230	0 + 234 + 224.7 = 458.7, refuses 121.7	102.2 + 127 + 0.7 = 229.9, admits it
240	0 + 245 + 233.1 = 478.1, refuses 121.7	112.9 + 127 + 0.0 = 239.9, admits it

The free list also goes from 4-5 blocks back to 1, and the 320 MiB arm is byte-identical base vs fix (no park is
followed by a re-creation there), which is the check that the change is narrow.

Not measured: the corruption itself (-ngl 99 / server shapes where a stale extent is large enough to be taken). The
arms above are the same state one step earlier, and the extent arithmetic is in the log.

For this issue's own assessment

This closes the behavioural half of #900 — a weight that fits is no longer refused — but not the wording fix from
#180, and neither lands on a user's box until the engine's third_party/llama.cpp pin moves past both. The
"coalescing" follow-up in the report would not have helped and is already implemented.

The inferred corruption is now observed, and #182 removes it (2026-11-04 06:48 / 07:01 AST, two tickets, kernel `7.4.0-rc1-1bit`)

The 01:3x comment here proved the slot arithmetic and stated the double-allocation from arena_take_locked only. Two
attempts to provoke it then failed (8 greedy tokens; then 1 843 read-backs of the arena after every weight upload). The
read-back instrument is blind by construction: it cannot see an extent handed out after the upload it verified.

So I added the two numbers the earlier attempts were missing — offset and slot on take and reclaim — as a diagnostic
(GGML_HRX_ARENA_TRACE, ~84 lines in runtime/host-memory.cpp, not part of the PR): every arena_reclaim_locked
insert and every arena_take_locked is checked against the live entries_ extents and logs
ARENA ALIAS reclaim|take: [size at off] overlaps a live entry [size at off] with resident/capacity/executions/parked.

Qwen3-0.6B, -ngl 0 -fit off -c 256 --temp 0 -n 8, the no-SVM trio, one box state, pin vs llama.cpp#182:

budget	pin `d8bd0ff60`	`#182` `fee91177d`
200	refuse, a 121.7 MiB weight (`0 resident + 205 parked`, `193.2 MiB free in 5 blocks`) — 263 reclaim / 413 take overlaps	refuse, a 1.7 MiB weight (`70.0 resident + 128 parked`, `1.3 MiB free in 1 block`) — 0 / 0
240	refuse, a 121.7 MiB weight — 171 / 459	refuse, a 1.1 MiB weight (`112.9 + 127 parked`) — 0 / 0
320 / 475	refuse, a 121.7 MiB weight — 0 / 0	refuse, identical — 0 / 0
500 / unbounded	runs — 0 / 0	runs — 0 / 0

The sequence on the pin at a 200 MiB budget, to the millisecond:

0.01.047.059 I ARENA TRACE grow #3: capacity 200.0 MiB (was 200.0), entries 0, parked 92
0.01.057.501 E ARENA ALIAS reclaim: [1.125 MiB at 0.000] overlaps a live entry [121.714 MiB at 0.000] (resident 121.7 of 200.0 MiB, executions 0, parked 0)
0.01.057.788 E ARENA ALIAS take:    [0.004 MiB at 0.000] overlaps a live entry [121.714 MiB at 0.000] (resident 121.7 of 200.0 MiB, executions 1, free 3 blocks)
0.01.057.893 E ARENA ALIAS take:    [1.125 MiB at 0.004] overlaps a live entry [121.714 MiB at 0.000] (resident 121.7 of 200.0 MiB, executions 1, free 3 blocks)

i.e. the grow re-creates the arena while 92 slots are parked, 10 ms later those old-arena extents are reclaimed into the
new free list on top of the live 121.7 MiB token_embd at offset 0, and the following takes hand those same bytes to
new weights (resident climbing 121.7 → 122.8 → 123.7 → 124.2 MiB while the take sits inside the live entry). Two weights
own one range; the second upload overwrites the first. That is the double-allocation, measured.

The honest limit, since it matters for how this is read: every arm that shows an overlap is an arm that ends in the
#173 refusal, and every arm that completes has 0 overlaps in both builds. Aliasing needs park → re-create → reclaim,
which needs eviction pressure, which needs a budget below the working set — and such a budget refuses. So this is
"reachable as state, not observed as consumed output on this model". It does not weaken the fix: resident + parked + free <= capacity holds and the free list no longer contains bytes a live weight owns.

Evidence on the box (kept for re-analysis): ~/amd-oss/hrx-study/evidence/memory/arena-alias4-20261104-064847.log
(pin, 7 arms) and arena-alias5-20261104-070150.log (pin vs #182, 6 arms each); written up in FACTS-memory §7.18
(and §8 item 11 is now closed rather than "inferred"), KNOBS row for GGML_HRX_ARENA_TRACE.

Third model, third shape: the working set scales with the budget, so "raise it to X" cannot converge (DeepSeek-V4.1-Flash UD-IQ1_S, 217 GiB)

Measured on a stack that already carries #182 and #190 (a5c32fbf8 + those two + #194), real HIP add-on, fork
2eecf9c67, -dev HRX0 -ngl 8 -fit off -c 1024 -b 16 -ub 16, MemoryMax 55-70G, 8 GiB MemAvailable watchdog, one
gpu-hold per group. Nothing here is a patch proposal, it is the measurement the issue asked for on a model where the
weights are 114 GiB and there is no quant of it smaller than this.

The ladder: 8192 → 12288 → 16384 → 24576 MiB of `GGML_HRX_HOST_WEIGHT_LIMIT_MB`

budget	resident + parked at refusal	the weight it could not fit	advice the message gives
8192	1919 + 5790	843 MiB	raise to 8554
12288	9893 + 2333	32 MiB	raise to 12260
16384	11156 + 4020	843 MiB	raise to 16022
24576	19782 + 3875	843 MiB	raise to 24502

Read the last column against the first: every rung asks for a budget ~74 MiB above itself, and resident simply grows to
absorb the budget (9.9 → 11.2 → 19.8 GiB). At 12288 the weight that "does not fit" is 32 MiB — the arena is full to the
limit and the leftover is smaller than the next allocation. So the non-convergence is not a message bug: it is resident
being unbounded, i.e. the per-execution working set is not a fixed size that a large enough budget buys. Note also that the
bookkeeping itself is now consistent (#182 working): 9893 + 2333 + 32 = 12258 of 12288.

It is not the ubatch and it is not the fused-across-device flag (three arms, one budget 24576, one gpu-hold)

arm	graph splits	resident + parked	the weight refused
`-ngl 8` rule (no `-fcd`)	595	19782 + 3875	843 MiB
`-ngl 8 -fcd`, `-ub 8`	520 (bs=8)	19782 + 3875	843 MiB
`-ngl 4 -fcd`, `-ub 16`	548	19112 + 4666	843 MiB

The first two rows are byte-identical, so whatever the working set is, it is neither ubatch-proportional nor caused by
fused ops executing on a device other than their layer's. -ngl 4 costs ~670 MiB less resident, which is the only lever the
three arms found.

Consequence for this model, and why I am not filing a patch

For a 114 GiB weight set the two ends are both closed on a 124 GiB UMA box, and both by measurement rather than policy:

budget below the working set → the refusal above, at every budget;
budget unset → the whole host-side weight set is materialised, which is engine HRX route pins ~107 GiB outside process RSS on a 217 GiB mmap (DeepSeek-V4.1 UD-IQ1_S), independent of -ngl/ubatch/prefetch #669's ~107 GiB GTT bleed;
full offload instead → 2807 MiB/layer × 41 = 112.4 GiB of device model buffer against hrx: gfxhub page fault + MODE2 GPU reset when GTT nears the 120 GiB ceiling (Flash-Next UD-IQ3_XXS, 18.8k ctx, 106.5 GiB GTT) #680's measured fault at
GTT 106.5/120, so a MODE2 reset.
So on this file the useful fix is not "a bigger budget" but a bounded working set: eviction that is not limited to
sole-owner entries, or per-layer staging that is released as the ubatch advances. That is your call, not mine — the tables
above are the evidence.

Repro is ~/lb/dsv41-stack-run.sh (one arm) and ~/lb/dsv41-stack-sweep.sh (the ladder) on strixhalo; the state file of
this lane (~/briefs/TDSV41-STATE.md, entry 2026-11-04 09:16-09:40) has the full logs and the before/after placement lines.

The target in bytes, for a model that needs it: 4 host layers, ~11 GiB (V4.1-Flash, 2026-11-04 13:3x AST)

Adding a number to my 09:16/09:50 tables (comment 6632749668), because "bounded working set" is easy to read as
"eventually" and this model turns it into a pass/fail.

DeepSeek-V4.1-Flash UD-IQ1_S (217 GiB, 41 layers) cannot full-offload: 2807.1 MiB/layer x 41 = 112.4 GiB of device model
buffer (measured -ngl 8: HRX0 model buffer size = 22456.91 MiB) against the engine's own 104.0 GiB GTT ceiling, so the
gap is 8.4 GiB ~= 3 layers. -ngl 37 = 101.4-101.9 GiB, i.e. it fits, and it leaves exactly 4 layers host-side
~= 11.2 GiB of weights (113.47 GiB of weights / 41).

With #194 (-fcd) those 4 host-side layers no longer disable the fused ops (measured: graph 28,168 -> 5,480 nodes,
1,051 -> 387 off HRX, Flash Attention kept enabled across devices), so the only thing between this model and its first honest
tok/s row is the arena holding ~11 GiB and stopping — where today it grows to absorb whatever budget it is given
(my 11-04 ladder: resident 1.9 -> 9.9 -> 11.2 -> 19.8 GiB as the budget went 8192 -> 24576 MiB, then refuses, asking for a
budget ~74 MiB above the one in force).

Acceptance test, if it is useful to whoever lands the fix (one gpu-hold, ~110 GiB of MemAvailable, the lane's stack build
2eecf9c67 = a5c32fbf8 + #194 + #190 + #182, real HIP add-on):

NG=37 BUDGET=12288 CAP=105G bash ~/lb/dsv41-stack-run.sh    # -ngl 37 -fcd -c 1024 -b 16 -ub 16

Pass = the run decodes (a Prompt:/Generation: line) instead of refusing to run ... Raise it to at least, with peak RSS
<~ 105 GiB and resident settling under the 12288 MiB budget instead of tracking it.

No patch from me in runtime/host-memory.cpp — the file is yours and your PRs #180/#182/#191/#193 are live in it; this is the
model and the number I can supply without touching them.

Independent reproduction on a small model (Qwen3-4B-IQ4_XS, 10 s per round): the advice does not just fail to converge, it cycles

Second data point for the "raise it to X cannot converge" claim, on a 2.27 GB model instead of the 217 GiB file, so
anyone can re-run it in ten seconds. Same build as this morning's head (a5c32fbf8, no local patches), one gpu-hold
ticket for the whole ladder, MemoryMax not needed (peak RSS 4.6 GiB, 8 GiB MemAvailable watchdog not triggered).

llama-cli -m ~/models/Qwen3-4B-IQ4_XS.gguf -dev HRX0 -ngl 0 -fit off -c 512 -b 32 -ub 32 -n 8 --temp 0 -no-cnv -p <49 tokens>
env: HSA_USE_SVM=0 HSA_USERPTR_FOR_PAGED_MEM=0 HRX_ALLOW_NO_SVM=1
     IREE_HAL_AMDGPU_LIBHSA_PATH=.../_rocm_sdk_core/lib/libhsa-runtime64.so.1

Each round sets GGML_HRX_HOST_WEIGHT_LIMIT_MB to exactly the value the previous round advised:

round	budget	the refusal that fired	advice the message gave
1	13 MiB	a 12 MiB weight does not fit with 0 MiB resident and 12 MiB parked	raise to 26
2	26 MiB	a 12 MiB weight does not fit with 0 MiB resident and 25 MiB parked	raise to 38
3	38 MiB	a 5 MiB weight does not fit with 0 MiB resident and 37 MiB parked	raise to 44
4	44 MiB	a 12 MiB weight does not fit with 0 MiB resident and 25 MiB parked	raise to 38
5	38 MiB	a 5 MiB weight ... 37 MiB parked	raise to 44
6	44 MiB	a 12 MiB weight ... 25 MiB parked	raise to 38
7-8	38 / 44	identical to 5-6	2-cycle

The ladder reaches a fixed point that is not a solution: 38 <-> 44 MiB, forever. Following the message's advice
exactly is a loop, not a climb, and neither value is anywhere near what the run needs.

The reason is visible in the two columns: which weight fails, and therefore parked, depends on the budget.
At 38 MiB the failing weight is the 5 MiB one with 37 MiB parked (need + parked = 44); at 44 MiB the failing weight
is a 12 MiB one with 25 MiB parked (need + parked = 38). The advice is ceil(need + resident + parked), computed
from one execution's snapshot, and the snapshot moves under it, so the two states advise each other. This is the
same non-convergence measured on V4.1 (8192 -> 8554 -> 12260 -> ...), reduced to its minimal form.

What the run actually needs is knowable and cheap to name, which is what the message is missing: the model's whole
IQ4_XS working set, computed from the tensors, is 1740.2 MiB materialized (shared4 carrier, 132 B per
(row, 256-block): 216 tensors, 1793.0 MiB in the file, ratio 0.9706 - every shape is group-aligned so padding adds
nothing), and the run completes at a budget of 4096 MiB with prompt 83.2 t/s, decode 10.1 t/s. A message that
said "this model's host weights need ~1740 MiB" would end the ladder in one step.

Two details that may matter for the fix:

The refusal is latched (budget_refusal_), so it reports the first weight that failed, not the largest
one; a per-model working-set sum would have to be collected before the first failure to be useful.
At a 13 MiB budget the same run fails through a different door than the one this issue is about:
materialize host weight value 2 failed first, and only then the arena message - so a reader sees two errors and
only the second one names a budget.

Not claimed here: any patch. This is the measurement your 14:07Z comment asked for on a model where the numbers are
small enough to check by hand.

The budget sweep on a small model: this model's host weights need ~304 MiB, and budgets up to 1741 MiB still fail — the constraint is the parked-slot accounting, not the size

Follow-up to my ladder comment above, with a correction to its last paragraph and the sweep your 14:07Z comment asked
for. Model Qwen3-4B-IQ4_XS.gguf, base a5c32fbf8, -dev HRX0 -ngl 0 -fit off -c 512 -b 32 -ub 32 -n 8, one
gpu-hold ticket per sweep:

`GGML_HRX_HOST_WEIGHT_LIMIT_MB`	outcome	the line that fired
13	`Compute error`	arena: `a 12 MiB weight does not fit with 0 MiB resident and 12 MiB parked`
38	`Compute error`	arena: `a 5 MiB weight ... 37 MiB parked`
512	`Compute error`	arena: `a 304 MiB weight does not fit with 0 MiB resident ...`
1741	`Compute error`	arena: `a 304 MiB weight ...`
4096	runs clean	nothing

The correction first. My ladder comment said the number the message should name is 1740.2 MiB. That figure is the
sum of the model's IQ4_XS tensors if every one of them were materialized; it is not what a run materializes. An
instrumented build (local, not shipped) that sums the materialized bytes of the host weights each command program
actually binds reports 304.3 MiB for the whole model (largest single program: 312.7 MiB — a program can bind two
views of one weight, which the per-model sum counts once). So this run needs ~300 MiB resident, and my earlier 1740.2
was the wrong aggregate.

What the sweep says. The budget is not what fails. 512 MiB and 1741 MiB are both several times the 304 MiB the
weights need, and both die on the same sentence: a 304 MiB weight does not fit with 0 MiB resident and ... parked.
0 MiB resident with a budget of 1741 MiB means the arena's free list is not the problem — the parked slots
(arena_deferred_) are. That is #182's subject, and it is why "raise it to X" cannot converge for any X between the
working set and whatever hides the parking waste (here: 4096 MiB works, 1741 does not).

So, for whoever lands #182/#190/#194: the working-set number is cheap and available (one sum over the bindings, before
the first request — 304.3 MiB here, matching the per-program sets within the double-count), but naming it in the
message will not end the ladder until the parked-slot accounting can actually free what it parked. I am not shipping a
patch on top of an unfixed arena_reserve; this comment is the measurement, not a proposal.

Reproducer for the middle row, 4 seconds: GGML_HRX_HOST_WEIGHT_LIMIT_MB=512 llama-cli -m Qwen3-4B-IQ4_XS.gguf -dev HRX0 -ngl 0 -fit off -c 512 -b 32 -ub 32 -n 8 -p <49 tokens> with the no-SVM trio and the TheRock libhsa.

What budget this model actually needs: 3072 MiB — measured, with the parked numbers that explain why

Same model and command as my two comments above (Qwen3-4B-IQ4_XS, -dev HRX0 -ngl 0 -fit off -c 512 -b 32 -ub 32), now
on base 428937ae6 (i.e. with #180, #182, #190, #194 in), one gpu-hold ticket per budget:

`GGML_HRX_HOST_WEIGHT_LIMIT_MB`	outcome	the arena's own numbers
13	`Compute error`	a 12.6 MiB slot with 0.0 resident / 12.6 parked
128	`Compute error`	76.8 MiB resident, 12.3 MiB free in 4 blocks, largest 4.2 (fragmented)
304	`Compute error`	a 5.3 MiB slot with 252.4 resident / 51.5 parked
512	`Compute error`	167.3 resident / 343.2 parked
1024	`Compute error`	0.0 resident / 1020.6 parked
1741	`Compute error`	0.0 resident / 1644.4 parked
2048	`Compute error`	505.1 resident / 1448.6 parked, then a 304.3 MiB slot
3072	runs	nothing
4096	runs	nothing

Two numbers worth keeping:

The model's host-weight set is 304.3 MiB (sum over distinct weights of the materialized bytes a program requests,
taken before the first request; 579 program observations on a run that completes). So the weights themselves need
~300 MiB.
The budget the run needs is ~3 GiB, ten times that, because the arena also holds the slots parked while programs
are in flight and they accumulate to ~1448.6 MiB for this model. 0.0 MiB resident with 1644.4 MiB parked at a
1741 MiB budget is the clearest form of it.

So the advice Raise it to at least 516 MiB (at a 512 MiB budget) and 1949 MiB (at 1741) are both still far below the
first budget that works. With -ngl 0 and a device present on a 2.27 GB model, 3072 is the answer; 2048 is not, and
neither is anything the per-failure arithmetic suggests.

I have opened llama.cpp#200 with the first half of this only (naming the set once, and using it for placement in the
direction it can prove), and its body states that it changes no measured outcome above - the parked-slot accounting is
what has to move first, and that is this issue.

The obvious fix for this row — let the arena reuse the parked slots — is measured and it is wrong: it does not fail, it produces wrong logits

I took the remaining half of this row as "the arena parks every slot for the length of an execution, so a budgeted run
can never reuse what it evicted; drain those slots back into the free list once the stream has been synchronized". That is
a two-file, +24/-6 change in runtime/host-memory.cpp (evict_for() already synchronizes before it reclaims, so the
drain looked free). I measured it before/after on the same box, same model, same command, and it is falsified:

it is safe where the run already worked, and below the working set it silently corrupts the output instead of refusing.

Before/after, Qwen3-4B-IQ4_XS, `-dev HRX0 -ngl 0 -fit off -c 512 -b 32 -ub 32 -n 8 --temp 0 -no-cnv`

Same command as my earlier comment; base
428937ae6 (so #180/#182/#190/#194 are in), one gpu-hold ticket per sweep, 1 Hz MemAvailable/gtt_used sampling.

budget	at base	at base + parked-slot reuse
13 MiB	`Compute error` (12.6 MiB slot, 0.0 resident / 0.4 free)	SIGSEGV
38 MiB	`Compute error` (5.3 MiB slot, 0.0 / 0.1 free)	runs, answers `GGGGGGGG`
128 MiB	`Compute error` (76.8 resident, 12.3 free in 4 blocks, largest 4.2)	runs, answers `GGGGGGGG`
512 MiB	`Compute error` (167.3 resident, 1.5 free)	SIGABRT - `HRX returned NaN logits: count=151936`
1024 MiB	`Compute error` (304.3 MiB slot, 0.0 resident, 3.4 free)	SIGABRT - same NaN logits
1741 MiB	`Compute error` (304.3 MiB slot, 0.0 resident, 96.6 free)	SIGABRT - same NaN logits
2048 MiB	`Compute error` (505.1 resident, 94.3 free in 1 block)	SIGABRT - same NaN logits
2304 MiB	`Compute error` (2231.7 resident, 7.1 free in 1 block)	(not run: the arm below already answers)
2560 MiB	runs, coherent, prompt 42.5 / decode 10.0 t/s	runs, coherent, prompt 57.9 / decode 11.0 t/s
3072 MiB	runs, coherent, 25.6 / 8.1 t/s (cold)	runs, coherent, 66.5 / 10.5 t/s
4096 MiB	runs, coherent, 62.4 / 10.0 t/s	runs, coherent, 72.8 / 11.1 t/s

Single runs each; the speed columns are not a comparison (first run after a load is page-in bound - the same trap I
recorded in the 15:0x entry). The outcome column is what the change did.

What that says, in three sentences

The parked slots are live, not merely stale. A slot is parked because a prepared command program's bindings
address it by offset for the whole execution (rebind_prepared_host_weights() re-materialises and shifts those refs
when a lease has expired, engine hrx: with #169's residency budget on, V4.1 loads on HRX0 then SIGSEGVs in CPU SET_ROWS; with the budget off it is RESOURCE_EXHAUSTED at materialize weight 3 (follow-up to #669) #788) - a lease that has expired does not prove the bytes are unreferenced, and
hrx_stream_synchronize() does not help because the reference is a recorded binding, not an enqueue. Reusing the slot
hands those bytes to another weight and the run returns GGGGGGGG at 38/128 MiB and NaN logits at 512-2048 MiB.
This row's refusal is load-bearing as a failure mode. With reuse enabled, arena_reserve() stops failing, so the
latch that prints the numbers (hrx: with #169's residency budget on, V4.1 loads on HRX0 then SIGSEGVs in CPU SET_ROWS; with the budget off it is RESOURCE_EXHAUSTED at materialize weight 3 (follow-up to #669) #788/Pin llama.cpp a34a6b7: HRX decode-split multipass output pass is vectorised (engine#124) #180) never fires, and a budget that cannot hold the run proceeds to produce
garbage instead of stopping. The refusal at 2048/2304 MiB is the correct behaviour, and it is not fragmentation:
2048 leaves 94.3 MiB in one block and 505.1 MiB resident, i.e. the rest of the arena is parked live slots.
The number this row should print is the per-execution live set, and it is ~2.5 GiB for this model, not 304 MiB.
Measured floor: 2304 MiB refuses, 2560 MiB completes (correction to my earlier "the run needs ~3 GiB" - I had not
tested 2304/2560). fork #200's 304.3 MiB is the sum over distinct tensors; the arena's requirement is the sum over
the tensors a single execution actually binds (505.1 resident + ~1.4 GiB parked at a 2048 MiB budget), and those are
different numbers for the same model.

What is left of the row, and what is not owed

Not owed: reusing parked slots. Measured, falsified, patch kept at ~/wt/i669-logs/parked-reuse.patch (with the
sweeps under ~/wt/i669-logs/{before,after,threshold,before2k}/) so nobody spends another pass on it.
Owed, and it is a message/guard problem, not an allocator one: a bounded run has to refuse when the budget is below
the execution's live set, and the advice in that message has to name the live set (learnable in one execution, ~2.5 GiB
here) instead of need + resident + parked from a snapshot, which is the 38 <-> 44 MiB cycle in my first comment.
fork #200 stands as written (advisory only, changes no measured outcome); its 304.3 MiB figure is a lower bound on
what a budget must hold, and this comment is the measurement that bounds it from below.

Post-merge re-measurement in the shipped tree (2026-11-04 17:2x AST). Base is 1bit/hrx-vulkan-patched @ 428937ae
(carries #180 2edd9454cc and #182 049d481ac1); the "pin" arm is the same binary I filed this with (d8bd0ff60).
One gpu-hold ticket, Qwen3-0.6B-Q4_K_M, -dev HRX0 -ngl 0 -c 256 -n 4 -v, no-SVM trio, GGML_HRX_HOST_WEIGHT_TRACE=1.

budget	pin: resident + parked + free vs limit	merged: resident + parked + free vs limit	blocking slot
200 MiB	0 + 205 + 193.2 = 398.2 (5 blocks)	70.0 + 128.7 + 1.3 = 200.0 (1 block)	pin: 121.7 MiB → merged: 1.7 MiB
240 MiB	0 + 245 + 233.1 = 478.1 (4 blocks)	112.9 + 127.0 + 0.0 = 239.9	pin: 121.7 → merged: 1.1 MiB
320 MiB	0 + 247 + 72.9 = 319.9 (2 blocks)	0.0 + 247.1 + 72.9 = 320.0	121.7 MiB (identical)

So in the merged tree: resident + parked + free <= capacity holds to printed precision at every budget, the 121.7 MiB
token_embd is admitted at 200/240 MiB where the pin refused it (the run then exhausts honestly on a 1.7/1.1 MiB
slot), the free list collapses from 4-5 blocks to 1, and the 320 MiB arm reproduces the pin byte for byte — the
narrowness check that says the fix resizes nothing. The new wording names the mechanism
(arena_reserve: host weight arena cannot place a 1.7 MiB slot after eviction: 70.0 MiB of 200.0 MiB resident, 1.3 MiB free in 1 blocks, largest 1.3 MiB) and the refusal repeats arena/limit/free/largest.

One residue the measurement shows, for the record rather than as a request: the raise-to figure is still
resident + parked + wanted (200 → "at least 201", 240 → 242, 320 → 369). It is now labelled as space instead of
capacity, but it is still a byte-sum, not the budget at which a contiguous block of the wanted size exists — so on a
fragmented arena it can still under-advise. The issue's own ask (report what actually failed) is satisfied either way;
say if you want that arithmetic changed too and it is a small follow-up.

Evidence: ~/amd-oss/hrx-study/evidence/memory/arena-deferred-20261104-172143.log,
postmerge-20261104-1721.log; written up as §7.21(b) of ~/amd-oss/hrx-study/FACTS-memory.md.

Closing: every part of this row has a merged fix, and the last one (#202) is the advice arithmetic this row was
about.

part of the report	fix	merged
refusal says "size" when the arena is fragmented	`llama.cpp#180`	2026-11-04 19:05Z
a parked host-weight slot outlives the arena that parked it (the 38 <-> 44 MiB cycle)	`llama.cpp#182`	2026-11-04 19:05Z
the model's host-weight working set is named once, and placement uses it where it can prove it	`llama.cpp#200`	2026-11-04 23:01Z
a capacity refusal no longer advises `resident + parked + one weight`, which is unreachable by construction	`llama.cpp#202`	2026-11-04 23:06Z

Verified in the merged tree rather than at the PR heads: 428937ae builds, its own
test-hrx-arena-refusal / test-hrx-graph-program-budget pass, and the alias the 38 <-> 44 MiB cycle came from is gone
(measured with an allocation-site trace at the pin and again merged; ~/amd-oss/hrx-study/FACTS-memory.md §7.14-§7.18,
§7.21(b)). #202 only changes the sentence, so the allocator behaviour measured in §7.21(b) still stands.

Nothing in this row is left open. Reopen if a non-capacity refusal still prints a number that cannot be reached.

Navigation Menu

Repository navigation

hrx: host-weight arena refusal is fragmentation, but the message reports a size problem and recommends a budget that cannot be reached #900

The smoking gun (`-v`, cheap repro, Qwen3-0.6B-Q4_K_M, budget 200 MiB)

The contradiction on a real model (Qwen3.8-Flash-Next UD-IQ3_XXS, 76.32 GiB, pin `d8bd0ff6052`)

Why this matters beyond the message

Proposed fix (diagnostic only, no behaviour change)

The fix is written and it is in the fork: llama.cpp#180 (head `de4c2b22e`)

Measured on this box (the pin's own binary, `d8bd0ff6052a` + the #788 arena)

Mechanism, read at the pin

Fix

For this issue's own assessment

The inferred corruption is now observed, and #182 removes it (2026-11-04 06:48 / 07:01 AST, two tickets, kernel `7.4.0-rc1-1bit`)

Third model, third shape: the working set scales with the budget, so "raise it to X" cannot converge (DeepSeek-V4.1-Flash UD-IQ1_S, 217 GiB)

The ladder: 8192 → 12288 → 16384 → 24576 MiB of `GGML_HRX_HOST_WEIGHT_LIMIT_MB`

It is not the ubatch and it is not the fused-across-device flag (three arms, one budget 24576, one gpu-hold)

Consequence for this model, and why I am not filing a patch

The target in bytes, for a model that needs it: 4 host layers, ~11 GiB (V4.1-Flash, 2026-11-04 13:3x AST)

Independent reproduction on a small model (Qwen3-4B-IQ4_XS, 10 s per round): the advice does not just fail to converge, it cycles

The budget sweep on a small model: this model's host weights need ~304 MiB, and budgets up to 1741 MiB still fail — the constraint is the parked-slot accounting, not the size

What budget this model actually needs: 3072 MiB — measured, with the parked numbers that explain why

The obvious fix for this row — let the arena reuse the parked slots — is measured and it is wrong: it does not fail, it produces wrong logits

Before/after, Qwen3-4B-IQ4_XS, `-dev HRX0 -ngl 0 -fit off -c 512 -b 32 -ub 32 -n 8 --temp 0 -no-cnv`

What that says, in three sentences

What is left of the row, and what is not owed

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

hrx: host-weight arena refusal is fragmentation, but the message reports a size problem and recommends a budget that cannot be reached #900

Description

The smoking gun (-v, cheap repro, Qwen3-0.6B-Q4_K_M, budget 200 MiB)

The contradiction on a real model (Qwen3.8-Flash-Next UD-IQ3_XXS, 76.32 GiB, pin d8bd0ff6052)

Why this matters beyond the message

Proposed fix (diagnostic only, no behaviour change)

Activity

bong-water-water-bong commented on Nov 3, 2026

bong-water-water-bong commented on Nov 4, 2026

The fix is written and it is in the fork: llama.cpp#180 (head de4c2b22e)

bong-water-water-bong commented on Nov 4, 2026

Measured on this box (the pin's own binary, d8bd0ff6052a + the #788 arena)

Mechanism, read at the pin

Fix

For this issue's own assessment

bong-water-water-bong commented on Nov 4, 2026

The inferred corruption is now observed, and #182 removes it (2026-11-04 06:48 / 07:01 AST, two tickets, kernel 7.4.0-rc1-1bit)

bong-water-water-bong commented on Nov 4, 2026

Third model, third shape: the working set scales with the budget, so "raise it to X" cannot converge (DeepSeek-V4.1-Flash UD-IQ1_S, 217 GiB)

The ladder: 8192 → 12288 → 16384 → 24576 MiB of GGML_HRX_HOST_WEIGHT_LIMIT_MB

It is not the ubatch and it is not the fused-across-device flag (three arms, one budget 24576, one gpu-hold)

Consequence for this model, and why I am not filing a patch

bong-water-water-bong commented on Nov 4, 2026

The target in bytes, for a model that needs it: 4 host layers, ~11 GiB (V4.1-Flash, 2026-11-04 13:3x AST)

bong-water-water-bong commented on Nov 4, 2026

Independent reproduction on a small model (Qwen3-4B-IQ4_XS, 10 s per round): the advice does not just fail to converge, it cycles

bong-water-water-bong commented on Nov 4, 2026

The budget sweep on a small model: this model's host weights need ~304 MiB, and budgets up to 1741 MiB still fail — the constraint is the parked-slot accounting, not the size

bong-water-water-bong commented on Nov 4, 2026

What budget this model actually needs: 3072 MiB — measured, with the parked numbers that explain why

bong-water-water-bong commented on Nov 4, 2026

The obvious fix for this row — let the arena reuse the parked slots — is measured and it is wrong: it does not fail, it produces wrong logits

Before/after, Qwen3-4B-IQ4_XS, -dev HRX0 -ngl 0 -fit off -c 512 -b 32 -ub 32 -n 8 --temp 0 -no-cnv

What that says, in three sentences

What is left of the row, and what is not owed

bong-water-water-bong commented on Nov 4, 2026

bong-water-water-bong commented on Nov 5, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

The smoking gun (`-v`, cheap repro, Qwen3-0.6B-Q4_K_M, budget 200 MiB)

The contradiction on a real model (Qwen3.8-Flash-Next UD-IQ3_XXS, 76.32 GiB, pin `d8bd0ff6052`)

The fix is written and it is in the fork: llama.cpp#180 (head `de4c2b22e`)

Measured on this box (the pin's own binary, `d8bd0ff6052a` + the #788 arena)

The inferred corruption is now observed, and #182 removes it (2026-11-04 06:48 / 07:01 AST, two tickets, kernel `7.4.0-rc1-1bit`)

The ladder: 8192 → 12288 → 16384 → 24576 MiB of `GGML_HRX_HOST_WEIGHT_LIMIT_MB`

Before/after, Qwen3-4B-IQ4_XS, `-dev HRX0 -ngl 0 -fit off -c 512 -b 32 -ub 32 -n 8 --temp 0 -no-cnv`