Skip to content

Add docker build - #96

Closed
djmaze wants to merge 4 commits into
Niko1221:mainfrom
djmaze:docker
Closed

Add docker build#96
djmaze wants to merge 4 commits into
Niko1221:mainfrom
djmaze:docker

Conversation

@djmaze

@djmaze djmaze commented Sep 28, 2026

Copy link
Copy Markdown

The engine is compiled during docker build, so the first start only downloads the model.

docker build has no GPU, so the CUDA architectures are fixed by an ARG (80;86;89;120) instead of read from nvidia-smi; the engine is a fat binary with a cubin per arch. BUILD.json is written with source=local and the source hash, so setup reuses it instead of recompiling.

The entry point keeps the install config on the volume and selects it by family and model, so switching between models already set up needs no setup pass. REINSTALL stays for settings recorded inside a config.

@Niko1221

Niko1221 commented Sep 29, 2026

Copy link
Copy Markdown
Owner

Thanks @djmaze - I'd like to take the Docker build. One change first: os.execv in setup.py changes every Linux start, not only the container's; could you put it behind an environment variable the image sets? Then it can merge.

@Niko1221

Niko1221 commented Sep 29, 2026

Copy link
Copy Markdown
Owner

Thanks @djmaze, still keen on this. The one change left is putting os.execv in setup.start() behind an environment variable that the Dockerfile sets, so normal Linux starts stay as they are. Also note #153 changes Ctrl+C handling in serve/server.py, so we'll check both together. Then it can merge.

@djmaze

djmaze commented Sep 29, 2026

Copy link
Copy Markdown
Author

Env flag is now there.

djmaze and others added 3 commits September 29, 2026 23:47
@djmaze DeepSeek Harness (Qwen3.8-Flash-Next) Strata
Docker: the engine is compiled during docker build, so the first star…
e907dac 
…t only downloads the model

docker build has no GPU, so the CUDA architectures are fixed by an ARG (80;86;89;120) instead of read from nvidia-smi; the engine is a fat binary with a cubin per arch. BUILD.json is written with source=local and the source hash, so setup reuses it instead of recompiling.

The entry point keeps the install config on the volume and selects it by family and model, so switching between models already set up needs no setup pass. REINSTALL stays for settings recorded inside a config. HEALTHCHECK hits /health, which is answered before the API key gate, with a start period covering the model load.

Co-Authored-By: DeepSeek Harness (Qwen3.8-Flash-Next) <noreply@deepseek.com>
@djmaze DeepSeek Harness (Qwen3.8-Flash-Next) Strata
Docker: handle SIGTERM so docker stop ends the container cleanly
37580f9 
PID 1 in a PID namespace ignores a signal whose disposition is the default, and Python installs a handler only for SIGINT, so docker stop's SIGTERM was dropped and the container was killed after the grace period (exit 137). The cleanup path ran only on Ctrl-C, and setup.py spawned the server as a child, so the signal never reached the process that owns the engine's stdin.

server.py now handles SIGTERM the way it handles Ctrl-C, and setup.py replaces itself with the server on POSIX, so the server is PID 1 in the container. The engine still ends by its QUIT line, which only the server can write, so stop with a longer grace: engine.close() waits 10s for the reply, hub.close() 2s more, and the engine reads QUIT between requests.

Co-Authored-By: DeepSeek Harness (Qwen3.8-Flash-Next) <noreply@deepseek.com>
@djmaze DeepSeek Harness (Qwen3.8-Flash-Next) Strata
Docker: gate the execv in setup.start behind STRATA_EXECV
c72b121 
os.execv changed every Linux start, not only the container's. setup.start now replaces itself with the server only when STRATA_EXECV is set, and the Dockerfile sets it: normal Linux starts keep spawning the server as a child, the container still gets the server at PID 1 so docker stop's SIGTERM reaches the process that can answer the engine with QUIT.

Co-Authored-By: DeepSeek Harness (Qwen3.8-Flash-Next) <noreply@deepseek.com>
@djmaze

djmaze commented Sep 29, 2026 •
edited
Loading

Copy link
Copy Markdown
Author

I realize that some features have been added since, which are now missing from the docker setup. Those would necessitate new GPUS, LAYER_SPLIT, KV and LOW_RAM env variables. (Leaving the experimental AMD HIP backend support out for now.)

Would you like me to update the branch and add these? With a rebase or merge? Or should we get this in first so I can create a follow-up PR?

Strata
Docker: KV, GPUS/GPU/LAYER_SPLIT and LOW_RAM as env vars
ed70a77 
The setup options added since 0.1.20 had no way through the container:
- KV: --kv int8|q4_0|k8v4 (k8v4 came with 0.1.25).
- GPUS / GPU / LAYER_SPLIT: the layer split from 0.1.21. Passed at the start
  too, where they pin the cards for this model; without them a config that
  names one card is offered once to a pair when the host has two that can
  share it (setup.offer_together), and that answer is written to the volume.
- LOW_RAM: --low-ram on|off|auto. setup.py reads the RAM from /proc/meminfo,
  which in a container is the host's total, so a memory-capped container has
  to ask for the low-RAM mode itself.
- VISION now also takes cpu, which setup.py already accepted.
@djmaze

djmaze commented Sep 29, 2026

Copy link
Copy Markdown
Author

I just rebased and added the missing features.

@Niko1221

Niko1221 commented Sep 30, 2026

Copy link
Copy Markdown
Owner

Thanks. #153 (merged for 0.1.28) changes the same Ctrl+C block in server.py main(). Please rebase on it and keep its timed sleep loop (Windows) beside your SIGTERM handler. Also, the Dockerfile says CMake refuses below 80; since 0.1.27 it allows 75.

@Niko1221

Niko1221 commented Sep 30, 2026

Copy link
Copy Markdown
Owner

Released in 0.1.30: hand-merged onto the current tree: the engine compiled in docker build, the server as PID 1 with a clean SIGTERM stop, and the image's CUDA archs 75;80;86;89;120. Merged under your name (GitHub doesn't show it as merged because it went in by hand). Thank you!

@Niko1221 Niko1221 closed this Sep 30, 2026
shiftySenpai pushed a commit to shiftySenpai/Strata that referenced this pull request Sep 30, 2026
Strata @claude @Niko1221
Docker build (Niko1221#96): the engine compiled in docker build, STRA…
a94fe10 
…TA_EXECV so the server is PID 1, SIGTERM stops it cleanly, KV/GPUS/LOW_RAM as env vars

Merged by hand onto 0.1.29: SIGTERM takes Niko1221#153's Ctrl+C path (the timed sleep loop and the second-Ctrl+C kill stay);
SIGINT keeps Python's handler; the image builds RTX 20 (75) too.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0154bwiad8gY7wRehCXy27pN
@djmaze
djmaze deleted the docker branch September 30, 2026 21:14
@djmaze

djmaze commented Sep 30, 2026

Copy link
Copy Markdown
Author

Btw, thanks for this amazing piece of software!

Hugua700 pushed a commit to Hugua700/Strata that referenced this pull request Oct 6, 2026
Strata @Niko1221
Docker build (Niko1221#96): the engine compiled in docker build, STRA…
b782066 
…TA_EXECV so the server is PID 1, SIGTERM stops it cleanly, KV/GPUS/LOW_RAM as env vars

Merged by hand onto 0.1.29: SIGTERM takes Niko1221#153's Ctrl+C path (the timed sleep loop and the second-Ctrl+C kill stay);
SIGINT keeps Python's handler; the image builds RTX 20 (75) too.
anon761 pushed a commit to anon761/Strata-upstream that referenced this pull request Oct 6, 2026
Strata @Niko1221
Docker build (Niko1221#96): the engine compiled in docker build, STRA…
621f449 
…TA_EXECV so the server is PID 1, SIGTERM stops it cleanly, KV/GPUS/LOW_RAM as env vars

Merged by hand onto 0.1.29: SIGTERM takes Niko1221#153's Ctrl+C path (the timed sleep loop and the second-Ctrl+C kill stay);
SIGINT keeps Python's handler; the image builds RTX 20 (75) too.
anon761 pushed a commit to anon761/Strata-upstream that referenced this pull request Oct 6, 2026
Strata @claude @Niko1221
Docker build (Niko1221#96): the engine compiled in docker build, STRA…
a5d098b 
…TA_EXECV so the server is PID 1, SIGTERM stops it cleanly, KV/GPUS/LOW_RAM as env vars

Merged by hand onto 0.1.29: SIGTERM takes Niko1221#153's Ctrl+C path (the timed sleep loop and the second-Ctrl+C kill stay);
SIGINT keeps Python's handler; the image builds RTX 20 (75) too.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0154bwiad8gY7wRehCXy27pN
Rarebigshot17 pushed a commit to Rarebigshot17/Strata that referenced this pull request Oct 19, 2026
@claude
The server handles SIGTERM even when its launcher ignored it
93054d5 
main() installed its stop handler only for a SIGTERM that was not ignored from the start, as for nohup's SIGHUP.
A launcher that ignores SIGTERM (a supervisor, a script with trap '' TERM) passes that on, and the server then did
nothing on kill, docker stop or systemctl stop until the final SIGKILL, which loses the engine's expert profile -
the server before these changes handled SIGTERM always (Niko1221#96). SIGTERM now always takes the stop path; an ignored
SIGINT or SIGHUP stays ignored.

The new test starts the server through a launcher that ignores SIGTERM and sends it one: the engine gets QUIT (it
timed out before, and passed on the code before these changes).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019zfufgbZ8KfWVmJivGBFLe
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@djmaze @Niko1221