Repository navigation
Description
Strata - Qwen3.8-Flash-Next on a normal PC (a GPU + system RAM + CPU)
=== Step 1: checking your PC ===
[ok] GPU: NVIDIA GeForce RTX 3070 Laptop GPU, 8.0 GB VRAM, compute capability 8.6, driver 616.92
[!] less than 12 GB of VRAM: Strata will run, but most experts stay on the CPU and it will be slow
[ok] RAM: 32 GB
[ok] CPU: 11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz (AVX-512)
=== Step 2: your choices ===
-
Qwen3.8-Flash-Next Qwen; GSQ-RCO quants by ISTA-DASLab - the original model
-
Swift 1.5 UkisAI's fine-tune of Qwen3.8-Flash-Next - thinks much shorter (-63% thinking tokens, 1.8x sooner answers by its authors' numbers)
-
Qwen3.8-Flash-Next Coder ISTA-DASLab's coding version - half the experts (code, tools, images kept): needs ~32 GB of RAM, faster; weaker outside coding
Which model? [1]: 1
[ok] model: Qwen3.8-Flash-Next -
Q2_0 2-bit, the fastest; download 66 GB, uses ~34 GB of RAM <- needs 48 GB RAM, you have 32
-
IQ2_XS 2-bit i-quant, a little better quality, close in speed; download 68 GB, uses ~36 GB of RAM <- needs 48 GB RAM, you have 32
-
IQ3_XXS 3-bit i-quant, better quality, slower (more CPU work per token); download 76 GB, uses ~43 GB of RAM <- needs 60 GB RAM, you have 32
-
IQ3_S 3.5-bit i-quant, the best quality (matches the full model), the slowest; needs a 64 GB PC with little else running; download 84 GB, uses ~50 GB of RAM <- needs 62 GB RAM, you have 32
Which size? [1]: 1
[ok] low-RAM mode: Q2_0's experts (34 GB) are read from the model folder through the OS file cache instead of a copy in RAM (32 GB); the GPU holds ~9% of them
[!] most of the experts are read from the SSD while it answers: expect it to be much slower than with enough RAM (a faster SSD and a smaller size help)
[ok] size: Q2_0
Context length = how much text the model can see at once (your chat, files, tool output).
Longer needs more VRAM for it, so fewer experts fit on the GPU:
- 8K tokens
- 32K tokens (recommended for your GPU)
- 64K tokens
- 128K tokens
- 256K tokens
Context? [2]: 3
[ok] context: 65536 tokens
KV cache precision (the model's memory of the conversation):
- 8-bit (recommended: what every published number was measured with)
- 4-bit half the memory (about 4% faster at 128K), but measurably less precise on long
documents; long-context lookups (needle tests) still pass
KV cache? [1]: 1
[ok] KV cache: 8-bit
Images: the model can also read pictures (screenshots, photos, scanned pages). This adds a 0.9 GB
download and keeps ~1.4 GB of VRAM free for the image encoder, so text is a few % slower.
Do you want images? [n]: n
[ok] images: off
EXPERIMENTAL - speed projection: a small control vector applied while the model runs (layers 4-44).
It changes how the model answers: its package describes it as a refusal-direction projection (the
model declines far fewer requests). Off unless you choose it; when on, the web app can switch it off
per chat. Details: data/experimental-speed-projection/README.md
Turn on the experimental speed projection? [n]: n
[ok] experimental speed projection: off
=== Step 3: Python packages ===
Installing numpy, jinja2, regex, pyyaml, tqdm, requests, cmake, ninja, pillow, psutil ...
C:\Users\user\Downloads\Strata-main\Strata-main.venv\Scripts\python.exe -m pip install --quiet --disable-pip-version-check numpy jinja2 regex pyyaml tqdm requests cmake ninja pillow psutil
[ok] numpy, jinja2, regex, pyyaml, tqdm, requests, cmake, ninja, pillow, psutil installed
=== Step 4: the Strata engine ===
llama.cpp source: 33.6 MB
[ok] llama.cpp source downloaded
Traceback (most recent call last):
File "C:\Users\user\Downloads\Strata-main\Strata-main\setup.py", line 2057, in
sys.exit(main())
~~~~^^
File "C:\Users\user\Downloads\Strata-main\Strata-main\setup.py", line 1877, in main
llama = get_llama_cpp()
File "C:\Users\user\Downloads\Strata-main\Strata-main\setup.py", line 641, in get_llama_cpp
f.extractall(tmp)
~~~~~~~~~~~~^^^^^
File "C:\Users\user\AppData\Local\Python\pythoncore-3.14-64\Lib\zipfile_init_.py", line 1876, in extractall
self.extract_member(zipinfo, path, pwd)
~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
File "C:\Users\user\AppData\Local\Python\pythoncore-3.14-64\Lib\zipfile_init.py", line 1947, in _extract_member
open(targetpath, "wb") as target:
~~~~^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'C:\Users\user\Downloads\Strata-main\Strata-main\third_party\_unpack\llama.cpp-3cf03257f219afbe7334045ff7c6a06ac68c627d\tools\ui\src\lib\components\app\chat\ChatAttachments\ChatAttachmentsList\ChatAttachmentsListItem\ChatAttachmentsListItemThumbnailImage.svelte'
Press any key to continue . . .
Activity
gsiisg commentedon Sep 30, 2026
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem
• If it doesn't exist: Right-click an empty space in the right pane, select New > DWORD (32-bit) Value, and name it LongPathsEnabled.
Niko1221 commentedon Sep 30, 2026
Thanks. That's Windows' 260-character path limit: the zip holds llama.cpp's web UI with very deep paths. 0.1.28 (in testing) doesn't unpack that part, which Strata doesn't use. Until then, move the folder somewhere short like
C:\Strataand run it again.setup: --host/--api-key/--draft-vocab on a start are kept for the mod…
Niko1221 commentedon Sep 30, 2026
Fixed in 0.1.28: setup no longer unpacks llama.cpp's web UI (the deep paths). Please reopen if setup still stops in a deep folder.
setup: --host/--api-key on a start are kept for the model (Niko1221#179…
setup: --host/--api-key/--draft-vocab on a start are kept for the mod…
setup: --host/--api-key/--draft-vocab on a start are kept for the mod…
ElGuyARc commentedon Nov 3, 2026
New evidence for the deep-folder setup problem in this thread: the
tools/uiexclusion fixes that member family, but a different member in the same pinned llama.cpp archive still exceeds Windows' ordinary path limit.Area / entry point
[Other] — source setup,
get_llama_cpp().What happened
A desktop integration originally reported
FileNotFoundError: [Errno 2]while extracting this archive member at an absolute destination length of exactly 260 characters:That original October report came from an older setup copy; its upstream version was not recorded, so I am not labelling it a complete 0.2.10 setup run. I have now reproduced the extraction failure with the actual 0.2.10
get_llama_cpp()function, isolated offline as described below.Current reproduction
ffa80d73133ea5c548aefe57889a16e8085a938f.setup.pySHA-256:fc761f5cf6fcc2b70118ae6c484e72c7dd554262ebff59e12a380206a1563c55.10.0.26220; Python 3.13.15. GPU and RAM are not involved in this fixture.The self-contained fixture below reads only that function's AST from an unmodified
setup.py. It supplies a tiny local ZIP with the exact member name above and synthetic{}contents, plus the two directory markers the function expects. Its download hook only checks that the local fixture ZIP exists. All extraction, filtering, moves and cleanup otherwise use the real function. No Windows errors are injected.Run from a source checkout:
If the process's temporary directory is itself deeply nested, pass
--scratch-rootnaming an existing short writable temporary directory.Observed on the same Windows process:
FileNotFoundError, errno 2FileNotFoundError, errno 2 / WinError 3The variant changes only the temporary extraction destination to Windows' extended-path form. It still includes the benchmark member and still excludes
tools/ui. This is evidence for the path handling direction, not a proposed complete patch or an archive-security review. Machines where ordinary long paths already work may not reproduce the failure.Expected / actual
Expected: setup can unpack its pinned source dependency in a deep writable cache without requiring a machine-wide policy change, or gives an actionable path-length error.
Actual:
setup.pylines 3615–3653 still extracts to an ordinarytmppath and filters onlytools/ui; the benchmark member remains eligible and fails before engine preparation finishes. The existingstrata_control.longpathhelper appears relevant, but I have not changed upstream code.Scope and limits
Reproduced locally: the isolated current source function's extraction failure and the extended-path control. Not run: a full 0.2.10 model download/setup, engine launch, inference or GPU test. The original user's private paths and logs are omitted.
Would you prefer to reopen this thread for the remaining non-UI archive paths? I searched open and closed issues for the function name, MAX_PATH, the benchmark filename and
benches/dgx-spark; I did not find a more specific existing report.Offline reproduction: reproduce_zip_maxpath.py
Sanitized result
{ "schema": 1, "sourceSha256": "fc761f5cf6fcc2b70118ae6c484e72c7dd554262ebff59e12a380206a1563c55", "functionSha256": "80fcfb9263ed1580f36ac42c00afc8912d80255263cdeb7e2eed79e64672d2ad", "sourceLines": [ 3615, 3653 ], "llamaCommit": "3cf03257f219afbe7334045ff7c6a06ac68c627d", "archiveMember": "llama.cpp-3cf03257f219afbe7334045ff7c6a06ac68c627d/benches/dgx-spark/aime25_openai__gpt-oss-120b-high_temp1.0_20251109_094547_allresults.json", "syntheticMemberContents": true, "python": "3.13.15", "os": "Windows", "windowsVersion": "10.0.26220", "cases": [ { "pathCharacters": 260, "extendedTmpCandidate": false, "outcome": "failed", "exceptionType": "FileNotFoundError", "errno": 2, "winerror": null, "calls": { "cachedZipHook": 1 } }, { "pathCharacters": 365, "extendedTmpCandidate": false, "outcome": "failed", "exceptionType": "FileNotFoundError", "errno": 2, "winerror": 3, "calls": { "cachedZipHook": 1 } }, { "pathCharacters": 365, "extendedTmpCandidate": true, "outcome": "passed", "calls": { "cachedZipHook": 1 } } ], "publicPathsRedacted": true, "networkOperations": 0, "modelsUsed": 0, "registryChanges": 0, "sourceFilesChanged": 0, "limits": [ "The 260/365-character ordinary cases record this actual Python/Windows environment, not a simulated MAX_PATH refusal. Other environments may already support ordinary long paths.", "The extended-destination AST variant proves direction only, not a complete upstream patch or archive-security review." ] }