feat(server): implement b10621 prompt-cache and slot-scheduling semantics #1453

Parent

• epic(server): close llama-server b10621 compatibility gaps #1431
• Immediate collision fix: fix(server): align llama-server b10621 aliases and defaults #1430

Problem

#1430 stops treating integer LLAMA_ARG_CACHE_REUSE as a boolean prompt-cache switch, but it does not implement b10621 KV-shift chunk reuse. Server-wide prompt caching, per-request cache_prompt, slot prompt similarity, explicit slot selection, continuous batching, and parallel-slot scheduling also have distinct externally visible behavior.

Scope

• Implement --cache-prompt and --cache-reuse N with the b10621 dependency, default, and minimum-chunk semantics.
• Align per-request cache_prompt, id_slot, prompt similarity, slot reuse, and cache invalidation behavior.
• Classify static/dynamic lookup caches, context/SWA checkpoints, checkpoint spacing, cache RAM limits, unified KV topology, idle-slot caching, and context shifting as implemented, not applicable, or deferred.
• Align --parallel, continuous-batching toggles, queue selection, cancellation, and fairness semantics where MLX execution permits it.
• Define cache identity across model, tokenizer, template, reasoning kwargs, adapters, modalities, and speculative configuration.
• Expose cache hit/reuse counters without logging prompt content or creating unbounded labels.

Acceptance criteria

• cache_reuse=0 means no KV-shift chunk reuse while prompt caching remains independently enabled. (--cache-reuse is now a real option with the LLAMA_ARG_CACHE_REUSE binding, so a command-line value beats the environment; 0 is accepted and inert and says nothing about whether the cache is on.)
• Positive reuse thresholds produce measurable compatible prefix reuse or fail explicitly until implemented. (Fails explicitly: no KV operation in this tree re-bases the rotary positions of a cached span, so chunk reuse cannot be expressed. The diagnostic names the missing operation.)
• Per-request cache disablement forces reprocessing and does not mutate reusable state. (cache_prompt: false on the chat-shaped routes withholds the request's cache context, which is the single handle the scheduler reaches the store through for both lookup and donate-back. Proven end to end: the opt-out reports cached_tokens == 0 and a following identical request still hits.)
• Explicit slot, similarity, continuous-batching, fairness, cancellation, and saturation cases have deterministic integration tests. (Continuous batching is done: --cont-batching / --no-cont-batching are real toggles with tests. Explicit slot and similarity are not, and cannot be without slot objects that hold prompt state; mlxcel synthesizes /slots from the active count and queue depth and reuses through a process-wide prefix trie. Recorded as divergences on --slot-prompt-similarity; id_slot belongs to feat(server): align props, slots, metrics, health, and slot persistence #1440's shard.)
• Lookup-cache files, context checkpoints, cache limits, KV topology, idle-slot retention, and context-shift options cannot be accepted without changing the corresponding runtime state. (--cache-ram is accepted because it does change the budget; --ctx-checkpoints, --checkpoint-min-step, --kv-unified, --cache-idle-slots and --slot-prompt-similarity are not accepted at all rather than accepted inert, each with the divergence recorded. Context shift is fix(inference): align context, batching, RoPE, and runtime override semantics #1450's entry and stayed deferred there.)
• Cache entries cannot cross model/template/adapter/media or authenticated tenant boundaries. (Audited: PromptCacheKey already carries model id, chat template, template kwargs including reasoning effort, tools and tool_choice, the multimodal digest, and the session key. Two gaps remain and are not closed here: lora_id is a field the chat route always fills with None, and neither the KV cache mode nor the speculative drafter configuration is in the key. All three are process-constant today because the store is in-process and never persisted, so nothing crosses in practice; making them explicit is follow-up work rather than a live defect.)

Numerical validation

Compare teacher-forced logit traces for cold prompt evaluation, prefix cache hit, KV-shift reuse, and mixed batch widths, reporting disagreement at decided positions.