Skip to content

[Other] docker image: stage the nested tools/ scripts the Terminal bundle reads (build stops at "installer tool missing: tools/dsv41/tune.py") - #3974

Closed
hurui200320 wants to merge 1 commit into
Niko1221:mainfrom
hurui200320:docker-ctx-nested-tools
Closed

hurui200320 wants to merge 1 commit into
Niko1221:mainfrom
hurui200320:docker-ctx-nested-tools

Conversation

@hurui200320

@hurui200320 hurui200320 commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Problem and result

Since v0.2.4.1 (b7958c04), packaging/package_terminal.py bundles the DeepSeek installer scripts: INSTALLER_SCRIPTS (line 128) names the nested modules dsv41/pack_gguf, dsv41/smoke, dsv41/tune, which setup.py runs by path. docker/build.sh fills the Docker build context's terminal-src/ with git ls-files ... ':(glob)tools/*.py' — top-level tools/ modules only. Inside the image, package_terminal.py runs on that context (docker/Dockerfile line 37) and add_installer() → local_imports() stops with:

SystemExit: installer tool missing: tools/dsv41/tune.py

Every image built from the v0.2.4.1 or v0.2.4.2 tree fails in the terminal stage this way, for every flavor (the context is flavor-independent). After this change the nested modules are in the context and the terminal stage completes; the image contents are unchanged otherwise — add_installer() still decides what enters the Terminal bundle, the context only gains source files.

Reproduced on main 7dd5cf18 (identical to v0.2.4.2 in every file this PR touches): sh docker/build.sh --flavor cuda --engine <stub package> --ctx ... --ctx-only, then package_terminal.local_imports() over the staged terminal-src → SystemExit: installer tool missing: tools/dsv41/tune.py. With this change the same call passes and finds dsv41/pack_gguf, dsv41/smoke, dsv41/tune.

Related discussion

None. Focused fix; no existing issue or PR covers it (searched open and closed). Not related to #3644 (root Dockerfile source build at v0.2.2): the root Dockerfile copies the whole tree (COPY . .), so only docker/build.sh's hand-written context list has this gap.

Approach

  • docker/build.sh: widen the pathspec to ':(glob)tools/**/*.py'. Git's :(glob) magic makes ** match zero or more directories, so the top-level modules are still staged exactly as before (all 158 still match) and tools/dsv41/*.py now reaches the context.
  • The no-.git fallback (source ZIP) used the shell glob tools/*.py, top-level only; it now stages find tools -name '*.py' so both branches carry the same set.
  • tests/terminal/test_docker_files.py: test_build_script_exits_zero_after_assembling_a_context already runs build.sh --ctx-only for real; it now reads INSTALLER_TOOLS and INSTALLER_SCRIPTS from package_terminal.py (ast, the way the sibling test reads PAYLOAD) and asserts every tools/<name>.py is in the context. Against the pre-fix script it fails with build.sh did not copy tools/dsv41/pack_gguf.py into the context; with the fix it passes.

Validation

Passed (branch 04b73643, Linux x86_64):

  • sh -n docker/build.sh → ok; shellcheck 0.11.0 reports nothing on the changed lines (the file's pre-existing CDPATH= warnings remain).
  • python -m pytest tests/terminal/test_docker_files.py -q → 17 passed (host, Python 3.13).
  • Regression check against the pre-fix script (same test file, docker/build.sh stashed to HEAD): test_build_script_exits_zero_after_assembling_a_context fails with build.sh did not copy tools/dsv41/pack_gguf.py into the context; restored → 17 passed.
  • Real context assembly: docker/build.sh --flavor cuda --engine <stub> --ctx ... --ctx-only; tools/dsv41/{pack_gguf,smoke,tune}.py present; package_terminal.local_imports() over the staged terminal-src passes.
  • No-.git tree: same sources staged without .git, its docker/build.sh --ctx-only stages tools/dsv41/tune.py; local_imports() passes.

Failed: none from this change.

Not run:

  • A full docker build on this machine. The failing stage is the terminal stage, and the checks above are the exact inputs it reads. External evidence, not validation of this commit: hurui200320/llm-ops CI run 40100350366 (2026-10-24, success, commit 2e44931) applied the same one-line pathspec change (via sed) to a v0.2.4.2 checkout — docker/, Dockerfile and package_terminal.py are identical to this branch — and built and smoke-tested ghcr.io/hurui200320/llm-ops-strata:0.2.4.2-rocm-gfx110x (Terminal version, component inventory, strata-dsv41).
  • GPU execution: the failing path is build-time and hardware-independent; no inference was run.

Limits

  • The already-cut tags v0.2.4.1 and v0.2.4.2 still carry the bug. Images for those versions need docker-images.yml's source_ref input pointed at the fixed commit (or a backport to the tag); later tags get the fix.
  • The find fallback assumes no whitespace in tools/ filenames, like the script's hand-written path list; git ls-files -- tools | grep -c ' ' → 0.

…nal bundle reads (INSTALLER_SCRIPTS names dsv41/*, so the terminal stage stopped at "installer tool missing: tools/dsv41/tune.py" since v0.2.4.1); the context test checks every installer tool is staged
Niko1221 pushed a commit that referenced this pull request Oct 27, 2026
…nal bundle reads (#3974)

docker/build.sh staged only top-level tools/*.py, so the Terminal stage stopped at 'installer tool missing: tools/dsv41/tune.py'.
Ported from PR #3974 by hurui200320: the pathspec is tools/**/*.py, the no-.git fallback finds nested files, and the context test reads
INSTALLER_TOOLS and INSTALLER_SCRIPTS from packaging/package_terminal.py. Fixes the failure that images built from v0.2.4.1 and v0.2.4.2 had.
@Niko1221

Copy link
Copy Markdown
Owner

Thanks. Confirmed on main: the build context only carries top-level tools/*.py, so the Terminal stage stops at tools/dsv41/tune.py. Ported with your authorship and your test to 0.2.7. Fixed in 0.2.7.

Release: https://github.com/Niko1221/Strata/releases/tag/v0.2.7

@Niko1221 Niko1221 closed this Oct 27, 2026
@hurui200320
hurui200320 deleted the docker-ctx-nested-tools branch October 27, 2026 19:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants