Feature Request: support DeepSeek V4.1 Flash with bounded SSD expert caching

Prerequisites

I am running the latest repository base used for this work: 0007bc6d3536bcd8c98c4e3eb18c5751b6822267.
I carefully followed the repository README and the Strix-specific contribution requirements.
I searched halo-box/strix-llama.cpp, halo-box/llama.cpp, and ggml-org/llama.cpp for DeepSeek V4.1, expert-cache, lazy-MoE, and SSD-streaming issues and pull requests; no matching work was found.
I reviewed the available repository documentation and existing DeepSeek V4, lazy tensor, and disk-backed n-gram implementations.

Feature Description

Support the published calibrated DeepSeek V4.1 Flash Q2 GGUF directly on a headless 128 GB Strix Halo without rewriting or requantizing the artifact.

Target artifact:

Path on the validation host: /mnt/models/deepseek-v41/DeepSeek-V4.1-Flash-Q2.gguf
Size: 365713686528 bytes
SHA-256: 1ce6a8f8806205c13330d7ca287bd198331dc5ca35ccc5d8a9a92a188a6f6f42
general.architecture=deepseek41
Source revision: df42c109f1defefcbfcedbe7d905718a12266e40
Quantization: IQ2_XXS gate/up, Q2_K down, Q8_0 attention/shared/head

The implementation must cover:

V4.1 metadata, tensor mapping, graph semantics, source-layer sharing, and output handling
Two disk-backed Engram tables with 264-byte rows: 256 native E4M3 values plus 8 E8M0 scales
Bounded SSD-backed routed-expert caching and native-quantized ROCm staging
Bounded prefill and decode execution using the existing IQ2_XXS, Q2_K, and MoE kernels
Runtime host-memory admission plus an independent watchdog
32,768 context validation first, progressing to 131,072 only after correctness and memory stability

The published GGUF must remain unchanged if technically possible. Engram and expert payloads must stay on NVMe when not resident. The implementation must not silently degrade quantization.

Motivation

The published Q2 GGUF is about 365.7 GB and its ordinary main weights are about 151.77 GiB, so conventional full loading cannot fit within the target host's 124 GiB usable RAM. Strix Halo uses unified CPU/GPU memory, which means model weights, ROCm buffers, graph scratch, context state, the OS, and filesystem cache all compete for the same physical memory.

The target acceptance policy is:

total host RAM below 120 GiB at all times
automatic cache sizing and graceful stop at a 116 GiB soft threshold
zero configured or used swap during validation
no evaluation I/O on rotational /mnt/bigspace
no host ROCm package changes

The existing TENSOR_READ_LAZY path is not sufficient: it keeps marked tensors CPU-mapped, relies on page faults/page-cache behavior, and cannot provide useful-speed ROCm execution with an explicitly bounded expert residency set.

Possible Implementation

Use a dependent, one-concern-per-PR stack:

DeepSeek V4.1 architecture and Engram
- Register deepseek41 separately from deepseek4.
- Add V4.1 metadata for compressor/indexer source-layer arrays and Engram layout.
- Reuse current DeepSeek V4 graph code only where semantics are identical.
- Add V4.1 source-layer indirection and output behavior.
- Decode Engram rows directly from the unchanged GGUF using bounded reads and reference-compatible BF16 rounding.
- Add synthetic/model-free architecture, hash, decode, malformed-row, and graph tests.
Bounded routed-expert store and cache core
- Exclude routed expert extents from ordinary model mapping/allocation.
- Read contiguous native-quantized expert slices with aligned direct I/O.
- Bound cache slots/bytes, bounce buffers, and in-flight entries.
- Add deterministic eviction, statistics, and explicit I/O errors.
Runtime memory admission and watchdog
- Account for non-routed weights, expert cache, Engram staging, ROCm upload buffers, graph/scheduler scratch, KV/recurrent state, and context.
- Auto-fit the expert cache to a projected total host use no higher than 116 GiB.
- Refuse startup if mandatory allocations exceed the budget or swap is enabled.
- Add an independent watchdog that measures host-wide use as MemTotal - MemAvailable and terminates below the 120 GiB hard ceiling.
ROCm cached-expert decode
- Split routing/selection from expert execution where needed.
- Load and upload only selected expert slices.
- Reuse existing gfx1151 quantized kernels without requantization.
- Prevent eviction while ROCm work is in flight.
Bounded prefill and final validation
- Chunk prefill so selected expert working sets cannot exceed the cache/staging budget.
- Add conservative measured prefetch and cache telemetry.
- Validate against /home/papa/src/ds4-v41 at bd66c402070042bf0a79ad6ece8242de4c93680c using exact token IDs, Engram row IDs, expert routing IDs, complete logits, top-1 results, and deterministic multi-step decode over all four repository correctness corpora.
- Build and run the merge-base and candidate separately in the same hardware session before performance claims.

No full-model load will be attempted while another user is active on the validation host.

AI usage disclosure: AGENT-AUTHORED. GitHub Copilot inspected the repository and reference implementation, drafted this issue, and will implement the proposed stack. All real-hardware verification and any unverified areas will be reported explicitly.

Implementation investigation found that DeepSeek V4.1 cannot safely share the current DeepSeek V4 runtime cache:

current DeepSeek V4 compressed state is fixed to 4x CSA and 128x HCA paths
V4.1 uses ratios 0, 1, and 2
V4.1 shares compressor state through KV source layers [2, 8, 14, 20]
V4.1 shares index selection through source layers [2, 8, 14, 20, 24, 28, 32, 36]
V4.1 adds candidate block propagation from layer 20
V4.1 uses the last hyper-connection pre-weights for output collapse instead of the DeepSeek V4 output_hc_* tensor family

To avoid landing a false deepseek4 alias, the stack is refined as follows:

GGUF schema contract: register deepseek41, exact metadata keys, published tensor names, and Engram tensor names, with C++ and Python model-free tests. Runtime remains explicitly unsupported.
Compressed attention graph: strict hparams, ratio 0/1/2 state, source ownership, candidate propagation, and output collapse.
Disk-backed Engram: row decoding, token-history hashing, bounded reads, and graph injection.
Bounded routed-expert store and cache core.
Runtime host-memory admission and independent watchdog.
ROCm cached-expert decode.
Bounded prefill, telemetry, and final host validation.

The unchanged published GGUF remains the target. No model payload has been loaded, and no Strix runtime test will be attempted while the host is in use.

Upstream prior art update (2026-09-12):

model : add DeepSeek V4.1 (DeepseekV41ForCausalLM) ggml-org/llama.cpp#28696 now provides a draft DeepSeek V4.1 converter/schema, but explicitly has no runtime. Its tensor/metadata differences are being compared against the existing published GGUF rather than used to replace that artifact.
llama : stream MoE routed experts from disk ggml-org/llama.cpp#25294 now provides a general SSD-streamed MoE cache with direct I/O, expert-ID remapping, bounded slots, and wave-partitioned prefill. The expert-cache layer will port/reuse compatible work instead of independently recreating it, then add the stricter Strix host-memory admission, file-bound validation, and correctness gates required here.
llama: GPU-resident LRU cache for host-offloaded MoE expert weights ggml-org/llama.cpp#27861 provides additional prior art for asynchronous cache publication and in-flight slot safety.

The review stack remains necessary: V4.1 graph/state and Engram semantics are still absent upstream, and the target is the already-published calibrated Q2 GGUF under a total host-memory ceiling rather than a newly converted replacement.

Correctness caveat for the expert-streaming layer: follow-up testing reported on ggml-org#25294 retracts its general bit-exact parity claim. Long greedy generations can diverge when wave-partitioned prefill changes floating-point accumulation order, even though each configuration is internally deterministic. Its automatic cache sizing can also choose fewer than the 3 x top-k slots required by the wave graph and then abort during graph construction.

This project will therefore reuse the bounded direct-I/O/cache/remap foundation, but will not enable wave-partitioned execution by default until it passes the repository correctness flow. Initial execution must keep a single expert GEMM path by fitting/adapting ubatch to cache capacity, and minimum capacity must be rejected during admission rather than through a graph-time abort.

Published-GGUF header-only capacity result (no payload mmap/load): 1,046 tensors split into 188.833 GiB of two disk-only Engram tables, 142.383 GiB of 120 routed-expert tensors, and 9.376 GiB of all other tensor extents. The model has 384 experts x 40 layers. One cache slot across every layer is 398,131,200 bytes (~0.37079 GiB): gate 3,041,280 + up 3,041,280 + down 3,870,720 bytes per layer/expert.

The pinned ds4 V4.1 graph formula estimates ~7.884 GiB at 32,768 context and ~8.951 GiB at 131,072. Consequently, 256 slots/layer plus dense weights and 131K graph state is already ~113.25 GiB before process baseline, direct-I/O buffers, and safety margin, so 256 cannot be the automatic setting under the 116 GiB soft threshold. A 224-slot configuration is ~101.38 GiB before those remaining categories. Final admission will derive values from the actual metadata and candidate allocation breakdown, then round down to complete slots.

DeepSeek V4.1 Engram graph integration

Reference: ds4 revision bd66c402070042bf0a79ad6ece8242de4c93680c.

Placement

Engram applies before attention on layers 1 and 14. It modifies the four hyper-connection residual branches, then the ordinary attention hyper-connection mixer runs.

For each token and active Engram layer:

Hash the token stream using the per-sequence history and optional text mask.
Read 24 rows x 256 values from the layer's disk-only table.
Multiply those rows by blk.L.engram_kv.weight.
Interpret the result as four key vectors plus one shared value vector, each width 5120.
For each of the four hyper-connection residual branches, compute a normalized gate and add the gated shared value.
Round the updated residual to BF16 at the same boundary as the reference.

The reference gate for branch h is:

dot = sum_i residual[h,i] * q_norm[h,i] * k_norm[h,i] * key[h,i]
dot *= rsqrt(mean(residual[h,:]^2) + eps)
dot *= rsqrt(mean(key[h,:]^2) + eps)
dot *= rsqrt(width)
gate = sigmoid(copysign(sqrt(max(abs(dot), 1e-6)), dot))
residual[h,i] = bf16(residual[h,i] + gate * bf16(value[i]))

eps is the model RMS epsilon. Image/non-text positions are masked: hashing inserts the dead/pad history value and the graph must not modify residual rows for those positions.

Decode lifecycle

Copy history before hashing.
Read both layers' 24-row sets before GPU execution.
Upload the first row pack only before layer 1 and the second before layer 14.
Commit the copied history only after the entire token evaluation succeeds.
A failure must not advance history or position.
Drain in-flight work before overwriting shared Engram staging used by a later layer.

Prefill lifecycle

Hash the whole logical token chunk once into [token][2][24].
Preserve history across chunk boundaries and commit it only after successful completion.
For large chunks, overlap disk reads with intervening layers, but keep the buffer bounded.
Layer 1 can consume table 0 while table 1 prefetch proceeds during layers 2-13.
A batch reader uses a row stride of 48 IDs to select the 24 IDs for one layer.
If prefetch is disabled or fails, use the bounded synchronous batch path; never map or warm the full table.
Keep staging ownership explicit so a later cache upload cannot overwrite bytes referenced by in-flight backend work.

State operations

Sequence reset, copy, checkpoint, restore, and rollback must include the four-token Engram history. Batched sequences require independent history. Invalid tokens and failed evaluations must leave history unchanged.

Correctness probes

exact row IDs before graph execution
masked-token residual remains byte-identical
layer 1 and layer 14 use the correct 24-ID half
decode and chunked-prefill history produce identical IDs
graph gate matches a scalar double-precision reference within the established BF16 boundary
complete prefill/decode logits compared with ds4 after attention, Engram, and expert paths are integrated

Stack progress

Two more model-free layers are published:

Attention/state: deepseek41 : add compressed attention runtime ajaxdude/llama.cpp#3, commit 70985986933c92cfdeb4fbb0d66ec0e755d1128f, stacked on schema commit e6a694b412c12aeab903f695d9100824d5debb80.
- Exact 40-entry metadata validation.
- Ratio 0/1/2 state, source ownership, candidate propagation, V4.1 output primitives, and parameterized memory accounting.
- The real model remains gated before payload allocation because disk-backed Engram and routed-expert execution are not integrated.
- Independent local validation passed test-deepseek41-runtime, test-deepseek41-schema, and test-llama-archs.
Bounded expert-store core: llama : add bounded routed expert store ajaxdude/llama.cpp#4, commit b44aac9b11b9dd8fe66a250f38cffc04949e3e3f, independently based on the schema layer.
- Registers routed tensors as external extents without GGML/backend storage.
- Preserves IQ2_XXS and Q2_K bytes and validates published merged-expert geometry.
- Adds fixed byte/slot budgets, deterministic LRU, atomic request-union planning, original-ID to slot remaps, and move-only pinned leases for later asynchronous upload.
- The non-bit-exact upstream wave-prefill execution was intentionally not ported.
- Independent local validation passed test-expert-store, test-deepseek41-schema, test-llama-archs, and test-gguf.

The next stacked session is integrating the completed Engram core and exact ds4 gate/history semantics on top of the attention branch. Expert execution and in-process admission remain separate later layers. No published model payload has been mapped or executed.

Runtime stack progress

Expert-store core: llama : add bounded routed expert store ajaxdude/llama.cpp#4 at a8364460999b94ac2ef52cf28bd1d0d4c925eef2.
- Production defaults require direct I/O and fail closed.
- Buffered mode is explicit and does not claim bounded host memory.
- Linux final-EOF O_DIRECT and Windows fail-closed regressions pass.
- Independent local tests and focused review found no remaining core issue.
Engram runtime: deepseek41 : add disk-backed Engram runtime ajaxdude/llama.cpp#5 at 4ac4d4a8433dc5862256874db1fd4257a6a90cf9.
- Disk-only tables require direct I/O.
- Exact hashing/decoding, layers 1/14 graph injection, transactional history, masked-row preservation, signed-zero gate semantics, and local-CPU callback pinning are implemented.
- Independent local tests and focused review found no remaining integration issue.

The next local-only layer is integrating external routed tensors, original-ID to cache-slot remapping, native IQ2_XXS/Q2_K upload, and lease lifetime through backend completion. It preserves the original single expert-reduction order and rejects an over-capacity selected union. Wave-prefill and automatic host-memory sizing remain later layers.

All work on the Strix host is paused by user directive. No SSH, remote build/test, model access, inference, copy, or configuration work will occur until explicitly resumed.

Stack repair after master sync

Current master moved to 654803517, creating overlap in the DeepSeek41 schema files.

Schema PR deepseek-v41 : add bounded runtime and correctness tracing #49 was repaired with a normal merge at ea7793dba3497a95ab2516e7bf36f15846b5bd4b.
- The only textual conflict was test registration.
- The resulting delta against master remains the seven intended schema/test files.
- C++ schema/model-generation and Python schema tests pass.
- Ubuntu and Windows CI pass.
Watchdog PR scripts : add Strix host-memory watchdog ajaxdude/llama.cpp#1: 2e27f6ed38607af78db7cb0c926f9b8c152af524.
Engram core PR engram : add bounded disk table and hash core ajaxdude/llama.cpp#2: 89b27c351d866e2c9bdb06e60f6568a4cf05883d.
Attention PR deepseek41 : add compressed attention runtime ajaxdude/llama.cpp#3: 9adba3bef8d83246fff770b56f8b985ca406e647.
Expert-store PR llama : add bounded routed expert store ajaxdude/llama.cpp#4: b7e4055d118c6116f9df40ca9880de94406ed8f4.
Engram runtime PR deepseek41 : add disk-backed Engram runtime ajaxdude/llama.cpp#5: e094c50c508c7395315ad6bff5e573a9876e2dd8.
Expert execution PR deepseek41 : add bounded routed expert execution ajaxdude/llama.cpp#6 is merging the updated PR reasoning-budget: add multi-point budget mechanism (intro, soft warnings, grace, usage telemetry) #5 head next.

The next functional layer is scheduler-backed full DeepSeek41 graph/state activation. Host-memory admission and the ds4 correctness harness are proceeding in parallel and will merge that graph layer before publication.

No heavy Strix validation has started. The live preflight still blocks it because /proc/swaps contains the configured swap file, an unrelated DS4 coding workload is active, and the external watchdog is not running. The active model/evaluation path must be NVMe-only, with 116 GiB soft and strictly below 120 GiB hard host-memory enforcement.

Schema warning fix and second stack repair

A clean rebuild exposed an exhaustive-switch warning because the new LLM_ARCH_DEEPSEEK41 enum was not classified by llama_model_rope_type(). The published model uses normal RoPE, so schema PR #49 now includes the one-line classification at 6a473b7e942a2cc54f1e5bb2b2d5deacd247b888. This does not enable the gated runtime.

Validation at the corrected schema head:

test-deepseek41-schema: passed
test-generate-models: passed
test-llama-archs -a deepseek41: expected schema-only skips
Python schema tests: 3 passed
rebuild has no unhandled DeepSeek41 switch warning
git diff --check: passed

Dependent branches were updated with normal merges and retain required authorship trailers:

watchdog PR scripts : add Strix host-memory watchdog ajaxdude/llama.cpp#1: 0ca941a17cd45a5f0f601071264f939f637a138c
Engram core PR engram : add bounded disk table and hash core ajaxdude/llama.cpp#2: 7ef7f03a56a56a4a2370265ee80505c664fa17cf
attention PR deepseek41 : add compressed attention runtime ajaxdude/llama.cpp#3: 580986ee4a86ad2ec5a73d367c8f220e06377da8
expert-store PR llama : add bounded routed expert store ajaxdude/llama.cpp#4: 50e1b27d3a1287ec0b2c9bf19016317ff801d313
Engram runtime PR deepseek41 : add disk-backed Engram runtime ajaxdude/llama.cpp#5: d57dd1c5ad668eb8c88fae5600efd8bfc51cf5c3

Expert execution PR ajaxdude#6 has fixed the reviewed remap-sentinel, Grovemoe lookup-ID, and asynchronous lease-release blockers and is currently merging the new PR #5 head and rerunning its suite. Full-graph work has started locally on the reviewed expert fixes but will not publish until it incorporates the final #6 owner head.

Heavy Strix validation remains blocked and has not started: the unrelated xmrig/DS4 workload is active and /swapfile remains configured. No process or host configuration was changed.

Final expert execution and dependent integration

Expert execution is now finalized in draft PR ajaxdude#6 at c4bab9a74c37ded9ef5b92eda4e81d3ed2ed797f.

The final review fixes include:

safe in-range slot IDs for failed remaps, with the runtime error surfaced after synchronization
original Grovemoe routing IDs preserved separately from chunk lookup IDs
scheduler synchronization before releasing expert leases on graph failure
backend buffer selection probed with the native IQ2_XXS/Q2_K MUL_MAT_ID shape
Windows shared-library test registration guarded consistently

Focused schema, Engram, expert, runtime, store, parser, Grovemoe, qwen4exp, and diff checks pass. Ubuntu and Windows CI are still running for PR #6.

Dependent local integration state:

full graph/state is active on exact base c4bab9a74, including the DeepSeek41 capability-gate audit and transactional scheduler memory
memory admission merged the final expert head normally and is at aa6220555; focused DeepSeek and watchdog tests pass
correctness harness merged the final expert head normally and is at b1ffc5f97; final executable-graph trace wiring is in progress

Heavy Strix validation has not started. Read-only preflight still shows the unrelated DS4 worker and mining services active, /swapfile configured with zero use, and no watchdog. No process or host configuration was changed.

Full graph, correctness, and admission review status

The runtime stack has advanced through published full-graph and correctness layers:

Expert execution deepseek41 : add bounded routed expert execution ajaxdude/llama.cpp#6 remains at c4bab9a74c37ded9ef5b92eda4e81d3ed2ed797f; mergeable/CLEAN with labeler, Ubuntu, and Windows passing.
Full graph/state deepseek41 : activate scheduler-backed full graph ajaxdude/llama.cpp#7 is finalized at 081c549451b92c01fe832ffb0bdd9ad90370e21a; mergeable/CLEAN with labeler, Ubuntu, and Windows passing. Independent review found and the branch fixed compressed-cache sentinel OOB access, non-F32 SCALE nodes, and on-device state restore. Targeted re-review passes all three fixes. The final 081c commit reuses the already-tested packed Engram row-ID helper without changing semantics.
Correctness harness deepseek41 : add cross-runtime correctness trace harness ajaxdude/llama.cpp#8 is restacked on exact 081c at 5dc4dc74ebdfba34a74afe9e743baf7b70f078d2. Its reserved dsv41.trace.* namespace now fails closed on malformed components and wrong layers. Independent re-review passes. Local validation is 6/6 focused CTests and 17/17 Python trace tests; Ubuntu and Windows CI pass. The base-to-head binary diff SHA-256 is 691875ad00f96e91ce42aae18b242975dae7ff85aa8e5075007b15a3724b57f9.
Canonical watchdog scripts : add Strix host-memory watchdog ajaxdude/llama.cpp#1 is updated at 8acdb46f83251b531b896658d5c913ba8edecf4a with SIGHUP forwarding and process-group cleanup coverage.
Memory admission deepseek41 : admit unified host memory before allocation ajaxdude/llama.cpp#9 is currently published at d0a161602f96ae7fb90d5e702f010f198ce71c6a, with corrected graph and local 14/14 CTests plus 20/20 watchdog tests. It is not final: independent review found three remaining fail-closed gaps now being fixed - common apps still defaulting to inadmissible ubatch=512, second-context rejection occurring after large allocations, and expert replacement peak accounting missing the incoming selected-expert payload plus direct-I/O bounce buffer. The local branch has also merged canonical watchdog head 8acdb46f8; a replacement PR Sync halo-box/llama.cpp: --ngram-on-disk and 31 upstream commits #9 head will be pushed after fixes and reruns.

An isolated integration session is active from final correctness and is waiting for the corrected admission head before the complete rebuild and low-level suite.

Real Strix validation remains blocked and has not started. The unchanged published GGUF was reverified on NVMe at 365,713,686,528 bytes with SHA-256 1ce6a8f8806205c13330d7ca287bd198331dc5ca35ccc5d8a9a92a188a6f6f42. The unrelated DS4 quantizer and mining services remain active, /swapfile remains configured with zero use, and no process or host configuration has been changed.

Corrected admission head and integrated validation

Memory admission ajaxdude#9 now replaces the superseded head with exact commit a127c1999f45ff0937cda1b808958d37f7a5a6d1. It is based on final graph 081c549451b92c01fe832ffb0bdd9ad90370e21a and contains canonical watchdog 8acdb46f83251b531b896658d5c913ba8edecf4a.

The replacement closes all three admission blockers:

implicit common CLI/server DeepSeek41 ubatch resolves to 32 after architecture discovery, while explicit -ub values are preserved, including the fit path;
the sole runtime context is acquired immediately after parameter validation and before expensive allocations, with exception rollback and exact-once destruction release;
peak accounting includes the incoming selected-expert payload min(384, 6*n_ubatch) * 9,953,280 plus the conservative 3,874,816-byte aligned direct-I/O bounce buffer.

Owner validation passed 14/14 focused and adjacent CTests, 20/20 watchdog tests, adjacent qwen4exp/Grovemoe architecture checks, builds, git diff --check, and ASCII scan. Independent review of 04dd21c8..a127c1999 found no significant issues and verified the production paths and exact threshold regressions. Hosted PR #9 Ubuntu/Windows CI is running; self-hosted Python/lint remains queued because no [self-hosted, fast] runner is available.

The integration branch merged the replacement normally and is now at 8e600a6b6e6c5564c5b51a96d25a5542ab6fb5b5. Ancestry checks pass for final correctness 5dc4dc74, graph 081c54945, watchdog 8acdb46f8, and admission a127c1999. The complete rebuild and focused/full low-level validation are in progress. The previously observed CPU q8_0 MUL_MAT m=128,n=4,k=5120 failure remains unsuppressed and was reproduced on immutable correctness base 5dc4dc74, so it is classified as pre-existing rather than an admission regression.

Real Strix validation remains blocked and no host state has changed. Read-only preflight at 2026-09-12 22:02 PT showed the DS4 worker/quantizer, xmrig, p2pool, and monerod still active; /swapfile remains configured at 32 GiB with zero use; no watchdog is running. /mnt/models and /home remain NVMe/SSD, while /mnt/bigspace is rotational and is used only by the unrelated DS4 job. The published GGUF path and size remain unchanged.

Newly explicit oracle-exporter dependency

The committed correctness harness and guarded runbook are ready, but the pinned ds4 checkout at bd66c402070042bf0a79ad6ece8242de4c93680c does not contain the required externally reviewed trace exporter. This is now an explicit blocking dependency rather than an assumption.

The required executable must leave the canonical ds4 checkout unmodified and implement the fixed harness interface:

DS4_EXPORTER --model MODEL --prompt-file PROMPT --output TRACE_DIR --context N --decode-steps 8 --prefill-chunk N --memory-audit FILE --swap-audit FILE --watchdog-audit FILE

It must emit dsv41-trace v1 with runtime ds4, the pinned ds4 revision, its own executable SHA, exact prompt tokens/bytes, every required Engram/routing/attention/candidate component, complete prefill and decode f32 logits, and greedy token IDs. run_ds4.py already validates this contract and rejects incomplete or unaudited output.

Duplicate searches in halo-box/strix-llama.cpp, ajaxdude/llama.cpp, and ggml-org/llama.cpp found no existing exporter PR or issue. A read-only cross-repo investigation is now active in the configured ds4gguf project to determine the smallest standalone implementation and correct repository ownership. No code or host state has changed. Hardware correctness remains blocked until both this exporter and the safe idle Strix window exist.

Correctness runbook blockers found before hardware execution

The first concrete guarded runbook exposed two fail-closed defects before any Strix execution, so correctness PR ajaxdude#8 head 5dc4dc74ebdfba34a74afe9e743baf7b70f078d2 is superseded for real-host use pending correction:

Its example matrix uses ubatch=512, 8 expert slots, and 4096 MiB. Final admission requires min(384, 6*n_ubatch) slots, so that setting requires all 384 slots and more than 142 GiB of cache alone. The initial matrix must use an admitted ubatch (normally 32), at least 192 slots, and a metadata/admission-derived cache byte budget.
The proposed lease procedure identifies a standalone heartbeat helper rather than the canonical memory watchdog. That can satisfy PID/start/cmdline/heartbeat checks without enforcing the 116/118 GiB thresholds. The harness must bind the lease and audit evidence to the actual strix_memory_watchdog.py process group, configured thresholds, and child exporter, and reject heartbeat-only leases.

The correctness owner is auditing committed scripts, tests, and documentation and will publish a replacement head if these flaws are present in the branch. Integration is retained but paused from publication. No model or host state was changed.

Oracle platform constraint

Cross-repo research confirmed that the pinned oracle revision antirez/ds4@bd66c402070042bf0a79ad6ece8242de4c93680c has no trace exporter and cannot execute DeepSeek V4.1 on Strix. Its V4.1 graph is Metal-only and the engine rejects non-Metal V4.1 inference; CPU can perform model-free inspection only, while its ROCm backend does not implement this graph.

A compliant exporter therefore requires a small separately reviewed DS4 source overlay built in a disposable checkout, leaving /home/papa/src/ds4-v41 clean. The recommended owner is the independent ds4gguf project, while strix-llama.cpp retains only the launcher/schema/comparator. The overlay can hook existing Engram hashes, router tensors, selected attention/candidate tensors, complete logits, and greedy IDs without changing model math, but exact layer-20 prefill candidate-block order needs one trace-only retention buffer because the canonical graph otherwise keeps only an unordered mask after each token.

This removes the assumption that the oracle can be produced on the Strix host. The remaining options are:

build the exporter and run the pinned DS4 oracle on a suitable 128 GB+ Apple Metal host, then transfer the immutable content-addressed trace bundle for comparison on Strix; or
treat a DS4 V4.1 ROCm port as a separate major prerequisite.

Implementation planning for the independent exporter is proceeding, but no code has been changed yet. The current task has no suitable Metal oracle host configured, so real ds4 trace generation remains an explicit environment blocker even after the exporter exists. No Strix state was changed.

Acceptance sequencing update (2026-09-12): the available Apple Metal host is a Mac17,2 with 16 GiB RAM and cannot execute the 340.6 GiB pinned DS4 V4.1 oracle. This is no longer a blocker to guarded Strix bring-up.

The complete pinned-DS4 dsv41-trace comparison remains unchanged as an external verification gate requiring a separately authorized 128 GiB+ Apple Metal host. It will not be silently weakened or reported as passed.

After the final watchdog/correctness heads are integrated and the Strix host is idle, the unchanged published Q2 GGUF will instead proceed through a separately labeled Strix acceptance lane: immutable published official compact probability/logit anchors; exact schema/tensor/Engram/routing fixtures from pinned DS4 source/tests/exporter with hashes; deterministic repeated multi-step self-consistency; immutable base-vs-integrated regression; guarded host-wide memory, zero-swap, NVMe-only, and watchdog evidence; then sequential 32K, 64K, 98K, and 131072 context gates. The unavailable full Metal cross-runtime result will remain explicitly listed as external verification, not conflated with Strix acceptance.

Written by GPT-5.6 Sol.

Safety update: watchdog 59833018814c0883f995848912cd5a52889c5303 and dependent admission 265f3df3055cbcdbf10a545d8d7c234ddd495fab are frozen for hardware use after independent review found four high-severity fail-closed defects: forgeable launch/artifact identity, orphaned protected processes after hard watchdog death, artifact I/O before emergency enforcement, and cleanup audit failure aborting escalation/reaping. Corrected replacements and regressions are in progress. No Strix mutation or swapoff may begin on these heads.

Written by GPT-5.6 Sol.

Watchdog rereview update: c4598ee747fdb811b474c0b41bdb38eeda3bbc0c closes three of four prior HIGH findings, but one HIGH cleanup failure remains, along with graceful-pulse and lease-version issues. Admission, correctness publication, integration, and the authorized Strix pause/swapoff remain frozen pending another corrected pushed SHA and fresh independent closure. DS4 is now idle; no host mutation has occurred.

Written by GPT-5.6 Sol.

Watchdog 0071e8f21701b74b3cc4dfe5c1131950ce338143 closes the six previously tracked findings but remains rejected after a new HIGH lifecycle defect was reproduced: guardian control failure during cleanup can bypass SIGKILL escalation/reaping and leave the payload alive. Admission, correctness publication, integration, and Strix mutation remain frozen pending another corrected SHA and fresh independent review.

Written by GPT-5.6 Sol.

Watchdog cd97b50b664bd170819b4225ae52cad97d762ada now kills and reaps correctly and preserves all earlier safety closures, but remains blocked by one medium evidence-classification defect: final artifact failure can overwrite the primary guardian-control failure. Admission, correctness publication, integration, and Strix mutation remain frozen pending the next exact SHA and independent review.

Written by GPT-5.6 Sol.

Status update (2026-09-16): stack revived directly on this repo as #70 + #71

The ajaxdude/llama.cpp-fork development described in the comments above was consolidated and re-landed directly against halo-box/strix-llama.cpp. Earlier PRs from this lineage (#45, #49, #52, #59, #60) were closed for stacking/hygiene reasons (not correctness) and are superseded by:

deepseek41 : add core DeepSeek V4.1 runtime (arch, loader, Engram, expert store, memory admission) #70 — "deepseek41 : add core DeepSeek V4.1 runtime" (base master, 7 commits): architecture registration, loader, model graph, Engram, expert store, memory admission, host-memory watchdog script. Consolidates the schema/Engram, expert-store, and memory-admission/watchdog layers described above into one reviewable PR.
trace : add DeepSeek V4.1 cross-runtime correctness harness #71 — "trace : add DeepSeek V4.1 cross-runtime correctness harness" (stacked on deepseek41 : add core DeepSeek V4.1 runtime (arch, loader, Engram, expert store, memory admission) #70, 3 commits): tools/deepseek-v41-trace/, the dependent server-child-protocol fixes, and the Python trace test.
Registered as a native GitHub Stack (Misc: Use argparse ggml-org/llama.cpp#72). `@dzannotti's review requested on both.
CI: all GitHub-hosted checks (ubuntu/windows/UI) passing on both; self-hosted checks (Lint/EditorConfig/metal/etc.) are currently queued behind a repo-wide runner backlog unrelated to this content.

On the oracle cross-validation gate this issue specifies: the pinned antirez/ds4 checkout still has no trace exporter and remains Metal-only for V4.1, as already noted above — that full bit-exact hardware comparison has not been run. #71's harness instead validates schema/manifest/provenance/containment/cross-runtime alignment at the llama.cpp level, and separately references kyuz0's antirez/ds4#1036 as external validation context (informative, not a substitute for the hardware-gated run this issue originally calls for).

This issue was closed because it has been inactive for 14 days since being marked as stale.

Navigation Menu

Repository navigation

Feature Request: support DeepSeek V4.1 Flash with bounded SSD expert caching #48

Prerequisites

Feature Description

Motivation

Possible Implementation

DeepSeek V4.1 Engram graph integration

Placement

Decode lifecycle

Prefill lifecycle

State operations

Correctness probes

Stack progress

Runtime stack progress

Stack repair after master sync

Schema warning fix and second stack repair

Final expert execution and dependent integration

Full graph, correctness, and admission review status

Corrected admission head and integrated validation

Newly explicit oracle-exporter dependency

Correctness runbook blockers found before hardware execution

Oracle platform constraint

Status update (2026-09-16): stack revived directly on this repo as #70 + #71

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Feature Request: support DeepSeek V4.1 Flash with bounded SSD expert caching #48

Description

Prerequisites

Feature Description

Motivation

Possible Implementation

Activity

ajaxdude commented on Sep 12, 2026

ajaxdude commented on Sep 12, 2026

ajaxdude commented on Sep 12, 2026

ajaxdude commented on Sep 12, 2026

ajaxdude commented on Sep 12, 2026

DeepSeek V4.1 Engram graph integration

Placement

Decode lifecycle

Prefill lifecycle

State operations

Correctness probes

ajaxdude commented on Sep 12, 2026

Stack progress

ajaxdude commented on Sep 12, 2026

Runtime stack progress

ajaxdude commented on Sep 13, 2026

Stack repair after master sync

ajaxdude commented on Sep 13, 2026

Schema warning fix and second stack repair

ajaxdude commented on Sep 13, 2026

Final expert execution and dependent integration

ajaxdude commented on Sep 13, 2026

Full graph, correctness, and admission review status

ajaxdude commented on Sep 13, 2026

Corrected admission head and integrated validation

ajaxdude commented on Sep 13, 2026

Newly explicit oracle-exporter dependency

ajaxdude commented on Sep 13, 2026

Correctness runbook blockers found before hardware execution

ajaxdude commented on Sep 13, 2026

Oracle platform constraint

ajaxdude commented on Sep 13, 2026

ajaxdude commented on Sep 13, 2026

ajaxdude commented on Sep 13, 2026

ajaxdude commented on Sep 13, 2026

ajaxdude commented on Sep 13, 2026

ajaxdude commented on Sep 17, 2026

Status update (2026-09-16): stack revived directly on this repo as #70 + #71

github-actions commented on Oct 31, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions