Skip to content

[Other] 0.2.1-cuda12 image has no sm_60 engine (P100 restart loop); source Dockerfile's terminal stage lacks git #3503

Closed
Closed
[Other] 0.2.1-cuda12 image has no sm_60 engine (P100 restart loop); source Dockerfile's terminal stage lacks git#3503

Description

@maxlawrie
maxlawrie
opened on Oct 20, 2026

Area

Docker

Desktop app, Terminal or source?

Docker image (and the source Dockerfile as a workaround)

What happened

docs/DOCKER.md lists GTX 10 (Pascal) / Tesla P100 / V100 for ghcr.io/niko1221/strata:0.2.1-cuda12, but the engine in that image (0.1.46-linux-x86_64-cuda-cuda12) has code for sm_75, sm_86, sm_89 and compute_89 only. On 3x Tesla P100 the container stops at start and restarts in a loop:

No engine for this PC was found in /opt/strata/components: 0.1.46 linux-x86_64-cuda has code for sm_75, sm_86, sm_89, compute_89, this PC's GPU is sm_60, sm_60, sm_60.
[X] The engine in this image does not match this computer's graphics card, or the card is not visible to the container.

strata-feed.json for v0.2.1 agrees: only the Windows cuda12 package (strata-engine-0.1.46-windows-x86_64-cuda-cuda12.tar.gz) lists sm_60 / sm_61 / sm_70; the Linux cuda12 package does not.

Building from source as a workaround hits a second problem: the root Dockerfile's terminal stage installs only python3 ca-certificates, but packaging/package_terminal.py runs git rev-parse HEAD (line 348), so the build fails with FileNotFoundError: [Errno 2] No such file or directory: 'git'. (.git is in .dockerignore, so with git installed the call just returns nothing and the COMMIT file is skipped.)

Workaround that works: build the root Dockerfile with --build-arg CUDA_IMAGE=nvidia/cuda:12.9.1-devel-ubuntu24.04 --build-arg CUDA_ARCHITECTURES=60 --build-arg BUILD_VISION=0, and patch it to

  • add -DSTRATA_EXPERIMENTAL_SM60=ON to the engine's cmake line,
  • change --min-driver nvidia=580.65 --variant cuda13 to --min-driver nvidia=575.51 --variant cuda12 (and the build id / "cuda": "13.0" in BUILD.json to 12.9),
  • add git to the terminal stage's apt-get install.

That image runs Flash-Next IQ3_S at 256K on 3x P100 fine (0.1.46 engine, ~+12% decode on code vs our 0.1.42 build, same answers).

Smallest steps or command that reproduce it

  1. On a Linux machine with a Pascal card: docker run --rm --gpus all -e MODEL=IQ3_S ghcr.io/niko1221/strata:0.2.1-cuda12
  2. The container reports the engine/GPU mismatch above and exits.
  3. For the second issue: docker build --build-arg CUDA_IMAGE=nvidia/cuda:12.9.1-devel-ubuntu24.04 --build-arg CUDA_ARCHITECTURES=60 . at v0.2.1 fails in [terminal 4/5].

Reproduction status

Reproduces every time

Strata version

0.2.1 (engine 0.1.46)

Graphics card, memory and system

3x Tesla P100 16 GB (PCIe), Ryzen 5 2600, 64 GB DDR4, Ubuntu 24.04, NVIDIA driver 580.178.04, Docker with the NVIDIA Container Toolkit

Model, size and settings

Qwen3.8-Flash-Next IQ3_S, 256K context, pictures off, STRATA_SET="resident-experts kv-resident=32768", GPUS=0,1,2

Possible fix

Build the Linux cuda12 engine package with -DSTRATA_EXPERIMENTAL_SM60=ON and sm_60/61/70 like the Windows one (or narrow the table in docs/DOCKER.md), and add git to the terminal stage of the root Dockerfile.

A small related note: with GPUS=all in the container, the low-RAM resident mode was placed on one card only (GPU 0); naming the cards (GPUS=0,1,2) gave the layer split across all three. That may be intended (#364), but all reading as "the engine's choice" was a surprise.

Activity

  1. github-actions
    added
    needs-formatTitle or template does not follow CONTRIBUTING.md yet
    on Oct 20, 2026

  2. github-actions commented on Oct 20, 2026

    @github-actions
    github-actionsBot
    on Oct 20, 2026 – with GitHub Actions

    Thanks for the issue. To keep the tracker readable, every item needs an area prefix in the title and the parts of the issue form filled in. This one is missing:

    • Start the title with its area: [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other], then a short description. Example: "[Engine] decode stops after 2 minutes".

    Edit the issue (title and description) and this note goes away on its own. If it is still missing parts in 5 days it will be closed, which is not a rejection: fix it and comment /reopen.

    Details: CONTRIBUTING.md

  3. maxlawrie
    changed the title [-][Docker] 0.2.1-cuda12 image has no sm_60 engine (P100 restart loop); source Dockerfile's terminal stage lacks git[/-] [+][Other] 0.2.1-cuda12 image has no sm_60 engine (P100 restart loop); source Dockerfile's terminal stage lacks git[/+] on Oct 20, 2026
  4. github-actions
    removed
    needs-formatTitle or template does not follow CONTRIBUTING.md yet
    on Oct 20, 2026

  5. github-actions commented on Oct 20, 2026

    @github-actions
    github-actionsBot
    on Oct 20, 2026 – with GitHub Actions

    Thanks, this follows the format now.

  6. Niko1221 commented on Oct 21, 2026

    @Niko1221
    Niko1221
    on Oct 21, 2026
    Owner

    The Linux cuda12 engine package of 0.2.1 was built without sm_60/61/70 (it had no build recipe in the repo). For 0.2.2, the next release (not out yet): packaging/build_cuda_linux.sh (VARIANT=cuda12) is added, the packager refuses a cuda12 package without Pascal/Volta code, and the source Dockerfile takes CUDA_IMAGE/CUDA_ARCHITECTURES for CUDA 12 and has git in the terminal stage. The published cuda12 image gets fixed when the 0.2.2 release is built with the recipe.

  7. Niko1221
    closed this as completedin 366c8dbon Oct 21, 2026
  8. Niko1221
    added a commit that references this issue on Oct 21, 2026

    Merge f-022-engfix: #3465 NaN retry reads the prompt batched, #3441/#…

    e0cfb80
  9. CC-David-CC
    added a commit that references this issue on Oct 21, 2026

    Fix Niko1221#3503: the Linux cuda12 engine package carries sm_60/61/7…

    366c8db

  10. hsntrk commented on Oct 21, 2026

    @hsntrk
    hsntrk
    on Oct 21, 2026

    New evidence after the 0.2.2 fix: the original report noted that .git is in .dockerignore. With git now in the terminal stage, the v0.2.2 source build stops at the commit check instead.

    Problem: Building the root Dockerfile (the "From source" build in docs/DOCKER.md) fails at v0.2.2 in the terminal stage.

    Environment: Strata v0.2.2 (011c9e6), Ubuntu 24.04.4 LTS, Docker 29.6.2, RTX 3090 24 GB, driver 595.99.02, CUDA_ARCHITECTURES=86

    Reproduce: on a clean checkout of v0.2.2 (git status --short shows no changes):

    docker build --build-arg CUDA_ARCHITECTURES=86 -t strata:test .

    I first hit it with docker compose build using the same context and build arg; both fail the same way.

    Expected: the image builds. v0.2.1.4 built and runs on the same host with the same setup.

    Actual:

    => CACHED [terminal 2/5] RUN apt-get update && apt-get install -y --no-install-recommends python3 ca-certificates git && rm -rf /var/lib/apt/lists/*
=> ERROR [terminal 4/5] RUN python3 /src/packaging/package_terminal.py --out /out --cache /cache ...
0.433 the commit of these sources is unknown (no .git here, no --commit, no STRATA_COMMIT): the archive would carry no COMMIT file, and every `strata bench run` on it would be nonconforming
ERROR: failed to build: failed to solve: ... exit code: 1

    Cause (git in the terminal stage is visible in the log above; the rest is from reading the source and is consistent with the workaround below): .dockerignore excludes .git, so the packager finds no repository. Since 436fbd0, docker/Dockerfile and docker/build.sh pass the commit (ARG STRATA_COMMIT → ENV STRATA_BUILD_COMMIT), but the root Dockerfile has no such argument.

    Workaround: add ARG STRATA_BUILD_COMMIT before the package_terminal.py RUN line and build with --build-arg STRATA_BUILD_COMMIT=$(git rev-parse HEAD). Result: passed (build finished, container starts and serves with engine 0.1.47).

    Reproduction status: reproduced locally (the build failure on a clean checkout, and the workaround).

  11. TheHubUser4860938
    added a commit that references this issue on Oct 21, 2026

    Merge upstream v0.2.2 into memory-core: the fork rides 0.2.2 (engine …

    94e905f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions