Asofwar · Oct 7, 2026 · Oct 7, 2026 · Oct 8, 2026 · Oct 9, 2026 · Oct 10, 2026
diff --git a/CMakeLists.txt b/CMakeLists.txt
@@ -910,6 +910,14 @@ if(STRATA_ENABLE_CUDA OR STRATA_ENABLE_HIP)
   add_test(NAME route_resident_parity COMMAND route_resident_parity)
   set_tests_properties(route_resident_parity PROPERTIES TIMEOUT 120)
 
+  if(STRATA_ENABLE_CUDA)
+    # the fast router (router_top10's default since it runs on CUDA too) bitwise against the portable kernel
+    add_executable(router_fast tests/cuda/router_fast.cpp)
+    target_include_directories(router_fast PRIVATE ${_strata_gpu_include_directories})
+    target_link_libraries(router_fast PRIVATE strata_kernels ${_strata_gpu_runtime_target})
+    add_test(NAME router_fast COMMAND router_fast)
+  endif()
+
   add_executable(router_top10_parity src/kernels/router_top10_parity.cpp)
   target_include_directories(router_top10_parity PRIVATE ${_strata_gpu_include_directories})
   target_link_libraries(router_top10_parity PRIVATE strata_kernels ${_strata_gpu_runtime_target})
@@ -1007,6 +1015,10 @@ if(STRATA_ENABLE_CUDA OR STRATA_ENABLE_HIP)
   target_include_directories(kv_hybrid_parity PRIVATE ${_strata_gpu_include_directories})
   target_link_libraries(kv_hybrid_parity PRIVATE strata_kernels ${_strata_gpu_runtime_target})
   add_test(NAME kv_hybrid_parity COMMAND kv_hybrid_parity)
+  add_executable(attn_splits_parity src/kernels/attn_splits_parity.cpp)
+  target_include_directories(attn_splits_parity PRIVATE ${_strata_gpu_include_directories})
+  target_link_libraries(attn_splits_parity PRIVATE strata_kernels ${_strata_gpu_runtime_target})
+  add_test(NAME attn_splits_parity COMMAND attn_splits_parity)
   # PR #21: Q4_0 KV with Walsh-Hadamard rotation vs a host reference (GPU, synthetic, no model).
   add_executable(kv_q4_parity src/kernels/kv_q4_parity.cpp)
   target_include_directories(kv_q4_parity PRIVATE ${_strata_gpu_include_directories})

diff --git a/README.md b/README.md
@@ -118,6 +118,7 @@ sizes are faster. Larger sizes are a bit smarter.
 
 | Your RAM | Take | Why |
 | --- | --- | --- |
+| **16-24 GB** | **[Qwen3.6-35B-A3B](docs/QWEN36.md)** | a smaller model of the same family; runs on 8-12 GB cards (NVIDIA) |
 | **32 GB** | **Coder** | it fits 32 GB, and it is made for code (with a 24 GB card, Q2_0 and IQ2_XS run too) |
 | **48 GB** | **IQ2_XS** (or Q2_0, the fastest) | the larger sizes do not fit |
 | **64 GB** | **IQ2_XS** (recommended), or IQ3_XXS / IQ3_S | every size fits; IQ3_S is the best and the slowest |
@@ -135,6 +136,10 @@ sizes are faster. Larger sizes are a bit smarter.
   model. But Strata reads most of it from the SSD while it answers, so it writes only 7-8.5 tokens/s on a 64 GB PC.
 - **[OrcaRouter's Uncensored IQ3_XXS](docs/MODELS.md#orcarouter-uncensored-iq3_xxs):** you set it up by hand. It is
   not in the installer's menu.
+- **[Qwen3.6-35B-A3B](docs/QWEN36.md):** a second, smaller model (35B, 3B active) whose experts take 12-14 GB of RAM:
+  105-127 tokens/s on a 12 GB card, 75-89 with an 8 GB card's VRAM (emulated). NVIDIA, one GPU, no images yet.
+  Its coding-agent fine-tune [Ornith-1.5-35B-A3B](docs/QWEN36.md#ornith-15-35b-a3b) installs the same way
+  (`--family ornith`).
 
 Sizes, downloads and what fits where: [docs/MODELS.md](docs/MODELS.md). To add another model later, run
 `SETUP.bat` (Linux: `./setup.sh --setup`).

diff --git a/bench/results/2026-10-13-rtx5080-ornith-attn-splits/A/README.md b/bench/results/2026-10-13-rtx5080-ornith-attn-splits/A/README.md
@@ -0,0 +1,61 @@
+# Community benchmark on NVIDIA GeForce RTX 5080
+
+Measured on 2026-10-13 with `tools/strata_bench.py` (suite 1, 3 runs per test, medians with ranges). Speeds are the engine's own timings; first-token times are measured by the client over HTTP. Speed only: no correctness check was run.
+
+## Hardware and software
+
+- GPU and VRAM: NVIDIA GeForce RTX 5080, 16 GiB VRAM
+- CPU: Intel(R) Core(TM) i9-14900K (24 cores, 32 threads); installed RAM: 47.7 GiB; storage: not measured
+- OS: Windows 11 (10.0.28000); GPU driver: 617.42; CUDA: not read
+- Strata: project 0.1.43.1, commit f1e33fb9; engine 0.1.43.1
+- Background workloads and power limits: not recorded
+
+## Model and configuration
+
+- Model and quantization: ornith-1.5-35b-a3b-abliterated-iq4_xs
+- Context: 262144 tokens; parallel slots: 1
+- Settings: the install's own configuration, unchanged; thinking off for the suite; temperature 0
+
+```text
+strata.exe --pack ornith-abl-iq4_xs --native Huihui-Ornith-1.5-35B-A3B-abliterated.i1-IQ4_XS.gguf --expert-profile expert-profile-qwen36.bin --expert-cache auto --prefill auto --spec 4 --mtp Huihui-Ornith-1.5-35B-A3B-abliterated.i1-IQ4_XS.gguf --mtp-draft-vocab draft_vocab_cyrillic.bin --max-context 262144 --kv int8 --expert-profile-save expert-profile-ornith-learned.bin --pcie-frac 0.00 --spec-min-p 0.50 --pool-workers 7 --adapt-every 1 --adapt-swaps 160 --adapt-decay 0.97
+```
+
+## Method
+
+Run by `python tools/strata_bench.py --long` against the HTTP API of a Strata server (started by the benchmark from the install configuration); model loading is not timed. One warm-up chat and a 1K prompt are discarded first. Every prompt starts with a unique tag, so the conversation cache cannot help the fresh-prompt tests.
+
+- short chat: the three questions of `tools/ab_engine.py`, up to 100 new tokens, 3 runs
+- 4K / 16K prompts: generated word lists sized by a calibration request, 24 new tokens, 3 fresh prompts each
+- agent session: system prompt and four tools, then 5 turns that each add about 1500 words of a project file and continue from the model's own earlier answers (the way agent clients resend a conversation), 64 new tokens per turn, 3 sessions; the cache numbers come from the answer's usage (`cached_tokens`) and timings (`prompt_n`)
+- concurrency (when run): N requests sent at the same moment, total tokens over the wall time of the burst, latency over all requests
+
+## Results
+
+| Configuration | Actual prompt tokens | Reused tokens | Generated tokens | Runs | Prompt tok/s median and range | Decode tok/s median and range | TTFT seconds median and range |
+| --- | ---: | ---: | ---: | ---: | --- | --- | --- |
+| short chat | 46 (43-51) | 0 | 100 (100-100) | 9 | not measured | 244.2 (206.7-265.7) | 0.14 (0.12-0.17) |
+| 4K prompt, fresh | 4019 (4019-4019) | 0 (0-0) | 24 cap | 3 | 5522.9 (5497.2-5782.7) | not measured | 0.75 (0.71-0.76) |
+| 16K prompt, fresh | 16058 (16058-16058) | 0 (0-0) | 24 cap | 3 | 5983.1 (5975.1-6078.2) | not measured | 2.71 (2.67-2.71) |
+| 32K prompt, fresh | 32049 (32049-32049) | 0 (0-0) | 24 cap | 3 | 4848.8 (4841.5-4887.5) | not measured | 6.65 (6.59-6.66) |
+| agent session, follow-up turns | 14559 (14551-14579) at the last turn | see below | up to 64 | 3 | see below | 226.2 (206.9-282.5) | 0.75 (0.48-0.85) |
+
+Ranges are the lowest and highest of the runs; `Runs` counts requests that returned a token.
+
+### Agent session (conversation cache)
+
+Cache hit over the follow-up turns: 68.2 % (68.2-68.2); earlier-turn tokens read again per session: 264 (257-284) (earlier turns that the cache did not hold; the 13721 (13720-13721) new tokens of the files are read once); first turn (nothing cached) first token: 0.49 s (0.47-0.49).
+
+| Turn (run 1) | Prompt tokens | Reused | Read | Prompt tok/s | First token (s) | Decode tok/s |
+| ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| 0 | 609 | 0 | 609 | 1344.4 | 0.473 | 175.7 |
+| 1 | 3779 | 602 | 3177 | 5318.0 | 0.612 | 208.5 |
+| 2 | 5308 | 3772 | 1536 | 2965.3 | 0.541 | 217.4 |
+| 3 | 8740 | 5301 | 3439 | 4251.5 | 0.834 | 230.0 |
+| 4 | 11647 | 8733 | 2914 | 4038.8 | 0.74 | 213.4 |
+| 5 | 14559 | 11640 | 2919 | 3575.0 | 0.854 | 206.9 |
+
+Per-request data: `results.json`. Engine log lines from the start: `engine-log-excerpt.txt` (when this run started the server).
+
+## Correctness and limitations
+
+Not measured: this is a speed suite. The conversation uses the model's own replies, so token counts differ a little between runs. Machine details come from the server, `nvidia-smi`/`rocm-smi` and the OS; background load and power limits are not recorded. Total time of the suite: 54 s.
diff --git a/bench/results/2026-10-13-rtx5080-ornith-attn-splits/A/engine-log-excerpt.txt b/bench/results/2026-10-13-rtx5080-ornith-attn-splits/A/engine-log-excerpt.txt
@@ -0,0 +1,5 @@
+[strata] loading the experts into RAM (about 19 GB) and locking part of them for the GPU.
+         Please wait and don't close this window; the browser opens when it is ready.
+[strata] filling the GPU's expert cache (5362 experts, 8.35 GiB of VRAM) ...
+[strata] almost ready ...
+ready: http://127.0.0.1:8096/v1  (OpenAI: /v1/chat/completions, Anthropic: /v1/messages, context 262144 tokens)
diff --git a/bench/results/2026-10-13-rtx5080-ornith-attn-splits/A/engine-start.txt b/bench/results/2026-10-13-rtx5080-ornith-attn-splits/A/engine-start.txt
@@ -0,0 +1,5 @@
+strata mtp: draft layer loaded from the model file, 704 MiB of VRAM (experts 408, dense 19), files read in 0.61 s (696 MiB/s)
+strata generate: expert cache auto: +393 MiB kept for the dense attention at 262144 tokens of context
+strata generate: expert cache 5362 slots, 8.35 GiB of VRAM; policy is
+strata verify: window up to 6 tokens, 441.8 MiB of device buffers
+strata serve: 0 MiB of VRAM free with everything loaded - LOW: requests may stall; add --vram-reserve-mib 1212 to the config's args (or lower --max-context)