How do I make it use the models I already have?

I already have an SSD drive with multiple versions of Qwen 3.8 Flash Next, including Unsloth's Qwen3.8-Flash-Next-UD-IQ3_XXS and Qwen3.8-Flash-Next-UD-Q2_K_XL. I intend to download coder and put that there too. The folder is
D:\LLM_Small

How do I make strata use those models instead of trying to download them?

Also, if you're using the HF huggingface downloader, it's bugged. It doesn't resume partial downloads, instead if it gets cut off it restarts from the beginning of the latest incomplete file. So if you've got 29GB downloaded of a 30GB file and it stops, you have to restart that 30GB file from the very beginning.

huggingface/huggingface_hub#4196

So for people with slower connections like me, it's necessary to download models with an ordinary download manager that can properly resume partial downloads. I don't know if you use the HF downloader library, I mention it just in case because it's another reason people may need to manually set an existing model folder when setting up Strata.

Here is the correct command for the standard model folder: #132
If you want a different folder theres also a command but I dont remember it just type "start-here.bat --help" and check the commands.

Thanks. I tried that but unfortunately strata is insiting there's no Unsloth I3Q_XXS model even though there is.

G:\Strata-main>start-here.bat --gguf-dir D:\LLM_Small --family unsloth --model IQ3_XXS
Strata - Qwen3.8-Flash-Next on a normal PC (a GPU + system RAM + CPU)

=== Step 1: checking your PC ===
  [ok] GPU: NVIDIA GeForce RTX 5060 Ti, 15.9 GB VRAM, compute capability 12.0, driver 616.56
  [ok] RAM: 64 GB
  [ok] CPU: AMD Ryzen 5 3600X 6-Core Processor (AVX2)

=== Step 2: your choices ===
  [ok] model: Qwen3.8-Flash-Next (Unsloth)


  [X]  Qwen3.8-Flash-Next (Unsloth) has no IQ3_XXS model file
       choose one of: UD-Q4_K_XL

Setup stopped. Fix the item above and run it again - everything already done is kept and skipped.
Press any key to continue . . .

I wondered if it was a folder name issue so I tried --gguf-dir D:/LLM_Small, --gguf-dir "D:\LLM_Small" and --gguf-dir "D:/LLM_Small" as well.

I guess it only works with very specific models. I'll download coder and see how things go.

You can point setup at files you already have with --gguf-dir, for example START-HERE.bat --setup --family coder --model IQ1_M --gguf-dir D:\LLM_Small. It only takes the files Strata supports: ISTA-DASLab's GSQ-RCO quants (Q2_0, IQ2_XS, IQ3_XXS, IQ3_S), the Coder's IQ1_M, Swift 1.5's, and Unsloth's UD-Q4_K_XL. Unsloth's other quants (UD-IQ3_XXS, Q2_K_XL, ...) are different files and won't work. The next release's message says this when it rejects a file. Downloads also resume by themselves, so an interrupted download is not lost. Closing as answered; reopen if --gguf-dir doesn't take a supported file.

setup (--gguf-dir): GGUFs Strata cannot run are named as such, with t…

Engine 0.1.35: Windows AMD uses its bundled HIP runtime (#468 #461), …

Merge upstream/main (v0.1.38) into the fork

docs/FORK.md: record the v0.1.38 merge in the divergence map

setup: Unsloth's UD-IQ4_XS as an experimental choice (#621)

setup (--gguf-dir): GGUFs Strata cannot run are named as such, with t…

Engine 0.1.35: Windows AMD uses its bundled HIP runtime (#468 #461), …

setup: Unsloth's UD-IQ4_XS as an experimental choice (#621)

setup (--gguf-dir): GGUFs Strata cannot run are named as such, with t…

Engine 0.1.35: Windows AMD uses its bundled HIP runtime (Niko1221#468 N…

setup: Unsloth's UD-IQ4_XS as an experimental choice (Niko1221#621)

setup (--gguf-dir): GGUFs Strata cannot run are named as such, with t…

Engine 0.1.35: Windows AMD uses its bundled HIP runtime (Niko1221#468 N…

setup: Unsloth's UD-IQ4_XS as an experimental choice (Niko1221#621)

docker: GGUF_DIR, RESIDENT_BUDGET_GIB and KV_STREAMING env vars; docu…

setup --gguf-dir: 動かせない GGUF はそう言い、使える --family と --model を示す（upstream …

This doesn't appear to work with gguf files that have been un-split. e.g. unsloths Qwen3.8-Flash-Next-UD-Q4_K_XL.gguf is not recognized at all by Strata @ engine 0.1.43, despite being called out explicitly as a supported model type.

Also, why the convoluted multi flag launch for this? Why not support --model-file /path/to/your/model/model_name.gguf

Navigation Menu

Repository navigation

How do I make it use the models I already have? #444

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

How do I make it use the models I already have? #444

Description

Activity

technerd-official commented on Oct 2, 2026

mkultra333 commented on Oct 2, 2026

Niko1221 commented on Oct 2, 2026

CharlesScoville commented on Oct 11, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions