[alextra-lab]

Summary

Adds a managed MTPLX backend (backend: "mtplx"): the launcher starts mtplx serve, the router routes to it by model id, and the watchdog restarts it. llama.cpp stays the default backend.

Spec: docs/superpowers/specs/2026-09-14-mtplx-backend-design.md
Plan: docs/superpowers/plans/2026-09-14-mtplx-backend.md

Draft: do not merge yet. Merge only when all three conditions hold:

1. PR feat: pin the llama.cpp build, add sidecar draft models, fix backend reliability #16 (llamacpp-pinned-build) has merged into main. This branch contains that PR's commits. After feat: pin the llama.cpp build, add sidecar draft models, fix backend reliability #16 merges, merge or rebase main here, so that this diff shows only the MTPLX work.
2. The FRE-1517 study window has ended.
3. Live verification after the study passes (spec §8, step 3).

Changes

Config (config.py)

• backend: "mtplx" and the fields mtp_depth, reasoning_effort, preserve_thinking (default off), mtplx_profile, mtplx_batching_preset (unset = serial), mtplx_fan_mode, and peak_memory_gib (all backends).
• Validation: the pack must be a local directory with config.json and mtplx_runtime.json. mtp_depth must be at most the pack's mtp_depth_max. Effort keys are rejected inside an MTPLX entry's config chat_template_kwargs. A warning is given for llama.cpp-only fields, a serial max_concurrency above 1, an unusual host, and a fan mode without the thermalforge socket.
• check_memory_budget(): SLM_MEMORY_BUDGET_GIB (default 100). An error for two or more heavy entries where any lacks peak_memory_gib, or a declared total over the budget.

Launcher (start_backends.py, start.sh)

• find_mtplx_binary() (SLM_MTPLX_BIN, then PATH, then ~/.mtplx/bin/mtplx) and build_mtplx_command(). Always --host 127.0.0.1 --no-auth.
• main() exits before starting any backend when the memory budget reports an error.
• start.sh: SLM_BACKEND_READY_TIMEOUT (default 180 s) replaces the fixed readiness loop.

Watchdog (watchdog.py)

• HTTP 507 (MTPLX memory refusal) gets the ignore verdict: no restart, and no reset of the failure streak.
• Backend log trimming also matches mtplx-<id>-<port>.log.

Router and telemetry (router.py, telemetry.py)

• _prepare_chat_template_kwargs() at chat completions and both /v1/responses fallbacks. For MTPLX it merges config and request kwargs (request wins) and lifts a request's reasoning_effort to the top level. llama.cpp and MLX behaviour is unchanged.
• cache_reuse falls back to usage.prompt_tokens_details.cached_tokens when llama.cpp timings are absent. New reasoning_tokens doc field and slm.reasoning_tokens span attribute.

Docs

• README: MTPLX backend section, memory budget, field reference, and watchdog table. config/models.yaml.example: a disabled MTPLX entry. CLAUDE.md: module descriptions.

Behaviour changes for existing clients

• A disabled model id returns 404 instead of 503. A request for the engine that is not loaded is normal during swaps. litellm does not retry a 404, so such a request now fails fast instead of after retries. Seshat has no code that treats 503 and 404 differently (checked in llm_client, sub_agent and executor).
• Every request telemetry doc gains a reasoning_tokens key.
• Telemetry cache_reuse now also fills in for backends without llama.cpp timings, such as MLX.
• A llama.cpp or MLX entry with a top-level reasoning_effort or preserve_thinking value outside the new allowed values now fails config validation. The owner's live config still loads.
• start.sh waits up to SLM_BACKEND_READY_TIMEOUT (default 180 s) for backend ports. A backend that never comes up now takes up to 3 minutes to report, not about 40 s.
• Deploy risk: when the memory budget check reports an error, the backend launcher exits before starting anything, start.sh exits 1, and the whole server stays down, the router included. Two or more enabled lm or multimodal entries where any lacks peak_memory_gib is an error. Before the first restart on this code, run check_memory_budget(load_model_config()) against the live config.
  • Result at 38fea8f against the owner's live config: errors []. Warning: the one enabled heavy entry has no peak_memory_gib. A single heavy entry only warns, so today's config starts.
  • When live verification enables a second heavy entry (an MTPLX entry), set peak_memory_gib on both entries. Otherwise the launcher refuses to start.

Tests

• Mac at 38fea8f: uv run pytest 285 passed, uv run ruff check src/ tests/ clean, uv run mypy src/ clean.
• uv run ruff format --check src/ tests/ flags only the 5 rerank files that already fail on main.
• New test files: test_config_mtplx.py, test_start_backends_mtplx.py, test_memory_budget.py, test_router_mtplx.py. Extended: test_watchdog.py, test_router_watchdog.py, test_router_model_selection.py, test_router_rerank.py, test_backend_log_trim.py, test_start_backends_main.py.
• Each of the 9 tasks had a spec and quality review, and a final whole-branch review followed. Its findings were fixed in 5d2a807 and 38fea8f and re-reviewed clean.

Test plan

• Unit tests, lint and type check on the Mac (results above)
• Independent review on the VPS at 38fea8f (cc-master): pytest 284 passed, 1 skipped (mlx smoke test). ruff check clean. ruff format and mypy show only the existing rerank and mlx findings. Code read: no blocking findings. Approved for merge once the conditions above hold.
• Memory budget check against the live config at 38fea8f: no errors, one warning (see Behaviour changes)
• Before the first restart on this code, run the memory budget check again if the live config changed
• Live verification: a streamed MTPLX request's span carries non-null cache_reuse and reasoning_tokens, which requires the final usage chunk in the stream
• PR feat: pin the llama.cpp build, add sidecar draft models, fix backend reliability #16 merged (deddce8), and main merged into this branch (10972fe). Mac at 10972fe: pytest 285 passed, ruff check clean, mypy clean.
• Live verification after the FRE-1517 study (spec §8, step 3), including a Seshat end-to-end turn and a litellm 507 retry count

Known limits (not in this PR)

• stop.sh has no sweep for an orphaned mtplx serve process. Processes are stopped by port.
• A deterministic MTPLX config error is retried up to the supervisor limit, as for other backends.
• The router's first-byte stall rule cannot detect a wedged backend that still sends keep-alives (MTPLX) or pings (llama.cpp). That needs generation-aware health (FRE-1519).

🤖 Generated with Claude Code

https://claude.ai/code/session_01HKZSVJuS3ehHkGcCt6msQL

[alextra-lab]

Merged main into this branch (7908a19). The merge was clean, with no conflicts.

Merge condition 1 is met. PR #16 merged into main on 2026-09-15. The diff now contains only the MTPLX work.

The merge brings in from main:

• the llama.cpp pin at the rebased PR #28243 head (6fcaa16f4), with the simplified scripts/build_llama.sh
• dependency floors for the Dependabot alerts (anyio, aiohttp, pillow, transformers, torch, setuptools, cryptography, json-repair, urllib3)
• a 400 answer for llama-server's request-level 500s (One malformed tool-call request (llama.cpp 500) makes the origin answer 503 to every request for about a minute #15)
• a bounded router-port poll in start.sh

start.sh merged with both readiness changes intact: this branch's SLM_BACKEND_READY_TIMEOUT loop for the backends, and main's 60-second poll for the router.

Verified in an isolated env with the dev, mlx and llamacpp extras, importing from this branch: 287 tests pass (main's 230 plus 57 from this PR), and ruff and mypy are clean.

Still a draft. Conditions 2 (the FRE-1517 study window has ended) and 3 (live verification, spec §8 step 3) are open and need the owner's call.