Skip to content

[Other] tests: three suite gaps a live PC showed (no engine, setup, serve, or docs change) - #3236

Closed
Buddychek wants to merge 1 commit into
Niko1221:mainfrom
Buddychek:tests/live-pc-suite-gaps
Closed

Buddychek wants to merge 1 commit into
Niko1221:mainfrom
Buddychek:tests/live-pc-suite-gaps

Conversation

@Buddychek

Copy link
Copy Markdown
Contributor

Problem and result

On a live PC (Windows 11, RTX 4090, a Strata server serving on the default port 8080) three suite gaps made the tools tests fail for reasons unrelated to the code under test:

  • tools/test_iq_pack.py died at import on a checkout without llama.cpp's gguf-py (_paths.gguf_py() exits hard there). After the rebase onto current main, upstream's new HadamardTests/NativeExpertsV5Tests classes hit NameError: name 'iq_pack' is not defined (18 errors).
  • tools/test_strata_mcp.py: the discovery scans were answered by the live server on 8080, not by the tests' own servers.
  • six tools/test_conversation_cache_*.py files fail when run from the repo root (python -m unittest), as the suite is run, because tools/ is not on sys.path.

After: the iq_pack file imports without gguf-py (iq_pack needs numpy + gguf_reader only, so it is imported unguarded), the GGUF-fixture classes (Compatibility, Conversion, SplitArtifact, NativeExpertsV5) skip with a named reason, the MCP tests see only their own servers, and the cache tests run from the repo root. No engine, setup, serve, or docs change.

Related discussion

Closed in the 0.2.0-beta tracker reset; rebased on main and re-verified per the reset note.

Approach

Guarded import + setUpClass skip in test_iq_pack.py (split_model's gate1 default moved to runtime so the module imports without gguf-py); DEFAULT_PORT patch in test_strata_mcp.py; sys.path insert in the six cache tests.

Validation

Passed, rebased on 0.2.0-beta main (e88ee39d), Windows 11, Python 3.15:

  • python -m unittest tools.test_iq_pack and python tools/test_iq_pack.py: 3 ran / 4 skipped, exit 0 (both start forms; the 4 skips are the GGUF-fixture classes on a checkout without llama.cpp).
  • tools.test_strata_mcp: OK, exit 0.
  • tools.test_conversation_cache_disabled / _growth / _isolation: 14 tests OK, exit 0.

Not run: the full tools suite in CI; no engine, setup, serve, or docs code touched.

Limits

The skips are conditional on gguf-py being absent; with llama.cpp cloned the classes run as before. No behavior change for any shipped code.

… docs change)

- tools/test_iq_pack.py: gguf-py is a setup.py clone, absent on a plain checkout; _paths.gguf_py() exits hard there, which killed the whole file. Import iq_pack unguarded (it needs numpy + gguf_reader only), guard the GGUF-fixture classes (Compatibility, Conversion, SplitArtifact, NativeExpertsV5) with a setUpClass skip, and make split_model's gate1 a runtime default so the module imports without gguf-py. Verified on 0.2.0-beta main (e88ee39): 3 ran / 4 skipped in both start forms (python tools/test_iq_pack.py and python -m unittest tools.test_iq_pack), exit 0.
- tools/test_strata_mcp.py: a live server on the default port 8080 answered the discovery scans; patch DEFAULT_PORT so the tests see only their own servers.
- tools/test_conversation_cache_*.py (6 files): sys.path insert so they run from the repo root with python -m unittest, as the suite is run.
Niko1221 pushed a commit that referenced this pull request Oct 27, 2026
test_iq_pack imports without llama.cpp's gguf-py and skips the classes that build GGUF fixtures with a named reason; the MCP discovery tests use
a free port, not whatever Strata serves on 8080; the six conversation-cache tests add the repo root to sys.path so they run from the repo root
as the suite does. Ported from PR #3236 by Buddychek (tests only, no shipped code).
@Niko1221

Copy link
Copy Markdown
Owner

Thanks. The gaps are still on main (test_iq_pack import without gguf-py, the cache tests from the repo root, the MCP tests answered by a live server). Ported to 0.2.7. Fixed in 0.2.7.

Release: https://github.com/Niko1221/Strata/releases/tag/v0.2.7

@Niko1221 Niko1221 closed this Oct 27, 2026
ruc1j added a commit to ruc1j/XeStrata-890M that referenced this pull request Oct 31, 2026
…の 1)

serve の変更
- 宣言していない名前の tool call を文のまま返すとき、その名前を strata_rejected_tool_calls で答えに添え、/metrics/prometheus で数える(upstream 805ebe1、Niko1221#3796)
- 名前空間つきの tool は namespace=member、namespace_member、member だけの綴りでも受け、`__` を `_` に潰した綴りも宣言した tool として読む(upstream fd8d37e、805ebe17b、Niko1221#3972)
- Chat Completions の tool の結果を、答えた call の順に並べ直す(upstream 11237f8、Niko1221#3395)
- message のない chat の要求は 400 にする(upstream 8a3ca07)
- --host :: で IPv6 で待ち受ける(upstream eba3106、Niko1221#3212)
- /v1/responses の max_output_tokens は 0 と -1 を上限なしとして受け、答えに timings を付ける(upstream 9c2d35d、a0fc88abd、Niko1221#4036、Niko1221#3291)
- response_format の repair_latex_escapes で、LaTeX のエスケープで壊れた JSON を直してから検査する(upstream 2cc6ea1、Niko1221#2893)
- 1 トークンの答えには decode の速さを出さない(upstream 5d5c26d)
- elastic な画像エンコーダーが起動できなければ 503 を返す(upstream 092c7d1、Niko1221#2435)
- lifeline の識別は子プロセスの開始時刻にし、sampler は壊れた source に耐える(upstream b3db5a1、Niko1221#3590、Niko1221#3407)
- web ページ: 答えを追うのは読み手が離れていないあいだだけ(upstream 89a68d8)、広い窓を使い狭い画面に収める(upstream ad1a310、Niko1221#3176)

setup の変更
- cmake と ninja は venv のものを先に取り、CMakeLists.txt が求める 3.24 より古い cmake は使わない(upstream 5bfce4b、Niko1221#3822)
- 別の generator で configure された build フォルダーは作り直す(upstream bf994ca、Niko1221#4038)
- ダウンロードの長さが取れなければ公開されている大きさを使う(upstream a1d2fd7、Niko1221#2287)
- 2 分データが来ないダウンロードは止まっていると言う(upstream ed0c893、Niko1221#3567)
- --rope-scale は有限で 1 以上の値だけ受ける(upstream a332cc9、Niko1221#3393)
- pip の記録はどの OS が書いたかを覚え、別の OS の記録は信じない(upstream fb3bae2、Niko1221#3634)

tools の変更
- gguf_reader.py はヘッダーの読みに上限を置き、壊れた metadata を ValueError にする(upstream 409f246、Niko1221#2288、Niko1221#1919)
- strata_bench.py は amd-smi しかない ROCm 7 以降でも AMD のカードの名前を書く(upstream c471588、Niko1221#3630)
- calibrate.py は計測の途中でエンジンが落ちても、その候補だけを落として続ける(upstream 41188ec、Niko1221#3317)
- chat.py は端末で fenced code block に色を付ける(upstream 3e81173、PR Niko1221#1895)
- 試験: llama.cpp の gguf-py がなければ test_iq_pack は skip、ほか試験の穴 3 つ(upstream 5b14fe2、Niko1221#3236)

fork に合わせて変えたこと: upstream の docs/DETAILS.md の差分は取らず、文書は別に書く。logprobs(upstream 9ea0449)は fork にすでにあり、allowed_token_ids は見送り。Control(strata_control)に依る setup の変更と、fork にない機能(presets の警告、Delete a model、queue timeout の badge)は取らない。
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants