[Engine] DeepSeek V4.1 engine (strata-dsv41) is refused on an RTX 4090: the prebuilt packages carry sm_86/sm_120 only, and the docs do not say RTX 40 is unsupported #3935

Activity

1. Benderyu commented on Oct 24, 2026

   

   Benderyu

   on Oct 24, 2026

   Author

   More actions

   Summary

   The DeepSeek-V4.1 engine is shipped as its own packages (strata-dsv41-0.1.49-...), and every package in the signed feed carries sm_86, sm_120 (or gfx1200, gfx1201) — no sm_89. On an RTX 4090 (sm_89) setup.py --family deepseek41 stops at step 4:

   [X]  DeepSeek's engine is not available for this computer yet
        Supported graphics cards: NVIDIA RTX 30 and RTX 50 series, and the Radeon AI PRO R9700
   

   Nothing in docs/DEEPSEEK41.md says RTX 40 is unsupported; the runtime list is the only place it is excluded, so people cannot know before downloading 257 GB.

   I built strata-dsv41 from source for sm_89 and it runs: the rest of this report is that build and its measurements, in case it helps decide whether to ship an sm_89 variant (or to document the source-build path).

   Prebuilt packages (feed sequence 111)

   package	archs
   strata-dsv41-0.1.49-windows-x86_64-cuda-cuda13.tar.gz	sm_86, sm_120, compute_120
   strata-dsv41-0.1.49-linux-x86_64-cuda-cuda13.tar.gz	sm_86, sm_120, compute_120
   strata-dsv41-0.1.49-linux-x86_64-hip-rx9070.tar.gz	gfx1200, gfx1201

   compute_120 cannot JIT down to sm_89, so there is no fallback.

   A source build for sm_89 works

   packaging/build_dsv41_windows.bat already takes DSV41_ARCH; its comments mention ARCH="86-real;89-real;89-virtual" for a CUDA 12.9 package. On this PC (nvidia driver 617.42, CUDA 13.2 toolkit, MSVC 14.44, cmake 4.4.3, ninja):

   set BUILD_JOBS=24
   set DSV41_ARCH=86-real;89-real
   set CUDA_ROOT=C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v13.2
   set STRATA_GGML_DIR=<checkout>\third_party\llama.cpp
   call packaging\build_dsv41_windows.bat <build dir> <checkout>
   

   • builds in a few minutes, no errors, strata-dsv41.exe 55.3 MB;
   • cuobjdump --list-elf → sm_86, sm_89;
   • strata-dsv41.exe --version → strata-dsv41 0.1.49.1 build c33fa7ba6;
   • placed at engine\strata-dsv41.exe (or STRATA_DSV41_EXE), setup.py --family deepseek41 --model IST-DAS-2.00bpw --gguf-dir <model folder> accepts it and writes the config.

   Model running on the RTX 4090

   GPU: NVIDIA GeForce RTX 4090 48 GB, driver 617.42, Windows 11, i9-13900K, 128 GB DDR5
   model: DeepSeek-V4.1-Flash IST-DAS 2.00bpw (IQ2_XXS + Engram + DSpark), 257 GB
   args: --max-context 262144 --ssd-cache 108 --ssd-policy lfu --dspark on --dspark-conf 0.65
   

   Engine log:

   ds4: RAM store: up to 10513 slots of 10.3 MB (108.0 GB), policy 1
   ds4: CUDA0 layers [0, 42] + head: weights 6.61 GB, experts 38.99 GB (4027 of 15744 in VRAM), caches 0.24 GB, 566 MiB free
   ds4: 11717 routed experts on the CPU lane (101.0 GB, in place in the model's files), 24 threads
   

   Measured (1 request at a time, 64-token answers, timings from the OpenAI endpoint):

   	result
   decode, median of 5	40.2 tok/s (samples 16.4 / 29.5 / 44.3 / 40.2 / 45.7)
   prompt, 1.2K tokens	141 tok/s
   prompt, 4.6K tokens	615 tok/s
   262,144-token context KV	0.24 GB of VRAM

   Two tuning notes that may be useful to others on a 48 GB card: the --ssd-cache RAM store (108 GB here) is what makes long prompts fast, and --dspark auto decides "off" on this box but --dspark on roughly doubles decode here (16–22 → 40 tok/s); the auto heuristic ("the verify rounds read 3.3x a token's bytes and gain nothing") does not hold once the experts are cached in RAM. --dspark-conf 0.65 measured better than 0.80 (30.6) or 0.50 (38.3).

   Also hit #3927

   While packing, pack_gguf.py picked up models\mmproj-Qwen3.8-Flash-Next-BF16.gguf (a Qwen vision tower in the shared models folder) and stopped. The documented workaround (rename it temporarily) got past it; #3927 covers that separately.

   Ask

   1. Is sm_89 (RTX 40) intentionally unsupported for this engine? If yes, please state it in docs/DEEPSEEK41.md next to the supported list, so nobody downloads 257 GB for a card that cannot run it.
   2. If not, would you ship an sm_89 variant (the main engine already ships 89, and the build above needed one more -gencode)? Happy to test a prebuilt package on this 4090 and report.
   3. If a local build is intended, please document the recipe (the DSV41_ARCH line above) and confirm that an unsigned local package via --dsv41-package is supported the same way as a built engine\strata-dsv41.exe.
2. Niko1221

   added 3 commits that reference this issue on Oct 27, 2026

   DeepSeek engine packages: add sm_89 (RTX 40) to the CUDA builds (#3935)

   ...

   4a759f8

   Merge f-027-gh-eng-g (#3935 sm_89 in the DeepSeek CUDA package builds, …

   ...

   40c9dae

   Merge f-027-gh-eng: GitHub engine items for 0.2.7

   ...

   6ca60e9
3. Niko1221 commented on Oct 27, 2026

   

   Niko1221

   on Oct 27, 2026

   Owner

   More actions

   Thank you. Fixed in 0.2.7: The DeepSeek CUDA packages are built for sm_86, sm_89 and sm_120 (plus PTX 120), and an RTX 40 card takes the package that holds its own sm_89 image; an sm_86-only package still fits. We have no Ada card; your 40.2 tok/s on an RTX 4090 48 GB (2.00 set, --dspark on) is quoted as yours in docs/DEEPSEEK41.md. Could you try the 0.2.7 package and tell us how it runs?

   Release: https://github.com/Niko1221/Strata/releases/tag/v0.2.7
4. Benderyu commented on Oct 28, 2026

   

   Benderyu

   on Oct 28, 2026

   Last edited by Benderyu

   Author

   More actions

   Thanks — the sm_89 package works on an RTX 4090. Here is what I measured with the 0.2.7 release.

   What I ran

   • Package: strata-dsv41-0.1.52-windows-x86_64-cuda-cuda13.tar.gz (sha256 f1a3ae88…; archs sm_86, sm_89, sm_120, compute_120), installed by hand as engine\strata-dsv41.exe.
   • strata-dsv41.exe --version = strata-dsv41 0.1.52 build a42cc3d7d; cuobjdump --list-elf lists sm_86, sm_89 and sm_120, so the card loads its own sm_89 image.
   • Machine: RTX 4090 48 GB (sm_89), driver 617.42, Windows 11, i9-13900K, 128 GB DDR5, NVMe.
   • Model: ISTA 2.00 bpw (IQ2_XXS + Engram + DSpark), pack rebuilt with the Unsloth mmproj-BF16.gguf (projector deepseek41v), context 262144.
   • Settings: --dspark on --dspark-conf 0.65 --dspark-block 3 --experts 5, --mmap-experts auto (the engine copies all 11,837 CPU-lane experts, 101.8 GB), 24 CPU threads, one request at a time.

   It loads in ~40–50 s with no errors and stays stable across restarts. Vision also works: a 336×336 test image (red background, white circle, blue square) is described correctly through the OpenAI API, at +~186 prompt tokens per image and 47.3/48 GB of VRAM (3,907 experts in VRAM with the tower loaded).

   Short benchmark (5 warm runs each)

   Setting	64-token decode (median)	256-token decode (median)	4K prompt	32K prompt
   0.2.7 defaults	45.3 tok/s	40.5 tok/s	960 tok/s	~1,947 tok/s
   DS4_DEFAULTS=026	48.1 tok/s	39.3 tok/s	1,037 tok/s	not measured

   The two sets are inside run-to-run noise on this card, so I see no Ada regression from the 0.2.7 defaults.

   Real agent session (what it feels like in use)

   From the engine's own log over a long coding session (786 requests, 452,544 generated tokens):

   • 110 generations of 800+ tokens: median 32.5 tok/s, P90 40.9, max 47.8.
   • Longest single generation: 26,541 tokens at 44.6 tok/s; other long ones: 5,466 tokens at 43.1, 2,300 at 45.8, 2,300+ at 45–46.
   • Prompt reuse is typically 96–100% (the client reuses the cached prefix), so prefill is effectively free and the perceived speed is the decode rate above.
   • Weighted over all 786 requests (including short tool turns and thinking) the average is 33.3 tok/s.

   So the "40.2 tok/s" in docs/DEEPSEEK41.md reproduces and long code generations sit at 40–47 tok/s on this card. My first few samples after a fresh load were 25–31 tok/s; that is cold-start, not a default-set difference — after one warm-up both default sets land at ~40 tok/s for 256-token answers.

   Nothing crashes; the difference between the two default sets is noise. Happy to bisect single switches (DS4_RING_MB, DS4_ATTN_SPLIT, DS4_COPY_PIN, DS4_DSPARK_MAX) if you want a per-switch table, but I would not expect a large one.
5. Niko1221 commented on Oct 29, 2026

   

   Niko1221

   on Oct 29, 2026

   Owner

   More actions

   Thank you for trying the package and posting the numbers. Closing: an RTX 4090 runs the 0.2.7 DeepSeek package.
6. Niko1221

   closed this as completedon Oct 29, 2026