[inureyes]

Summary

Every option in b10621's GGML runtime, placement, and memory group describes the GGML backend: which CPU cores its thread pool runs on, how many layers to copy into VRAM, how to split a model across GPUs, whether to mmap or mlock the GGUF file, which RPC servers to farm work out to, and which GGML quantizer stores the KV cache. mlxcel runs every tensor operation through MLX on one Metal or CUDA device and has no GGML backend, so almost none of it has a counterpart.

Before this change six of those options parsed and did nothing while the other thirty-seven were rejected by clap as unknown arguments. Neither group told the operator anything true: a deployment passing --n-gpu-layers 20 believed twelve layers were on the CPU, and -ngl auto (b10621's own default spelling) did not start at all.

What changed

• src/cli/ggml_compat_args.rs (new): one flattened clap group carrying all forty-three spellings, hidden from --help, that classifies every value at startup before a weight is read.
  • Inert, accepted silently: --split-mode none, --threads -1, --cpu-strict 0, --poll 50, --prio 0, --flash-attn on|auto, --gpu-layers all, --main-gpu 0, --load-mode auto|mmap, --mmap, --no-direct-io, both halves of --repack, --kv-offload, --op-offload, --no-perf, --n-cpu-moe 0, --fit off, and --backend-sampling, which asks for the on-accelerator sampler mlxcel always uses.
  • Rejected with the option, the value, the limitation, and the alternative: --split-mode row and --tensor-split point at docs/distributed.md, --rpc at --node-role / --peers, --mlock and --load-mode mlock at MLXCEL_WIRED_LIMIT, --device at MLXCEL_DEVICE, --perf at --metrics, --fit on at --estimate-memory, --no-kv-offload at --cache-type-k / --cache-type-v, --check-tensors and --list-devices at mlxcel inspect.
• src/cli/turbo_args.rs: --cache-type-k / --cache-type-v gain a value-specific diagnostic. A GGML quantizer name now explains why the vocabularies do not carry over and lists the mlxcel modes that do exist; anything else gets the plain accepted list.
• src/main.rs, src/bin/mlx_server.rs, src/commands/serve.rs: the group is flattened into both binaries, replacing the four ad-hoc _n_gpu_layers / _flash_attn / _mlock / _no_mmap fields that parsed and did nothing. Classification runs in two stages, before and after model resolution.
• compat/llama-server/b10621/ggml-runtime.json: all forty-two entries move off deferred.
• tests/llama_compat_manifest.rs: the two census floors become one partition invariant.
• docs/llama-server-compat.md, docs/environment-variables.md: the operator-facing boundary and the environment vocabulary.

KV cache formats: exactly one alias, and no invented equivalence

f16 names the same thing on both sides, unquantized half-precision KV storage, so it maps onto KVCacheMode::Fp16. That is the only value the two vocabularies share.

Every other GGML name is a different quantizer, not a different name for the same one. q8_0 is block-wise with a per-block scale; mlxcel's int8 is per-token absmax; the Turbo modes are PolarQuant with a Walsh-Hadamard rotation. None of q8_0, q4_0, q4_1, iq4_nl, q5_0, q5_1, f32 or bf16 is mapped onto a TurboQuant mode, because no numerical or storage equivalence has been demonstrated and naming similarity is not evidence. Each is rejected at startup with a diagnostic naming the value, why the vocabularies do not carry over, and the mlxcel modes that do exist. mlxcel's own int8, fp16+turbo4, fp16+turbo3, turbo4 and turbo4-delegated stay on the same flag and stay visible in --help.

No new KV representation or mapping is introduced, so the issue's teacher-forced logit-trace requirement has nothing to measure here. If anyone later wants one of those GGML types aliased, that procedure is what would have to run first; the doc says so.

Two design points worth review

• --gpu-layers is classified after the model resolves. Only the checkpoint's num_hidden_layers separates a full offload (inert: mlxcel always runs every layer on the accelerator) from a partial one. Everything else is classified before, so --numa distribute is reported immediately instead of after a multi-gigabyte download. When the layer count cannot be read, a non-negative budget is unsupported rather than guessed at.
• Nine options bind their environment variable at runtime, not through clap. b10621 fires a value-less option from the environment only for on/enabled/true/1 (common_arg_utils::is_truthy, case-sensitive) and reads a --x / --no-x pair through parse_bool_value plus a LLAMA_ARG_NO_* alias meaning false. clap's boolish parser accepts a wider vocabulary and errors outside it, so binding through clap would both enable options on spellings b10621 ignores and refuse to start on values it silently drops. LLAMA_ARG_CPU_MOE=0 therefore does not enable --cpu-moe, and LLAMA_ARG_PERF=sometimes stops startup exactly as upstream throws.

Behavior change for existing deployments

Five options that mlxcel previously accepted and ignored can now stop startup: --n-gpu-layers <partial>, --flash-attn off, --mlock, --no-mmap, and the GGML KV cache types. That is the point of the issue ("No option with an observable value is accepted and ignored"), but it is a deployment-visible change for anyone copying a llama-server command line, so every rejection names what to pass instead and docs/environment-variables.md says so.

Manifest census

The two assertions in tests/llama_compat_manifest.rs floored the claimed and unclaimed halves of the option inventory at hand-picked numbers. This PR converts forty-two entries from unclaimed to claimed, which is exactly what epic #1431 exists to do, so the unclaimed floor tripped. Rather than lowering a magic number that will trip again on the next chain, the two halves now have to add up to b10621's frozen 249 help entries. That invariant cannot go stale and still catches a shard that loses entries.

Test plan

• cargo test --profile test-fast --features metal,accelerate --test llama_ggml_runtime_cli (9 passed) — every upstream spelling reaches mlxcel's own classification rather than clap; inert values fail only for the missing model; unsupported values name option, value, limitation and alternative on both binaries; a GGML KV quantizer is refused with the mlxcel vocabulary; the environment vocabulary matches b10621's; nothing appears in --help.
• cargo test --profile test-fast --features metal,accelerate --lib cli::ggml_compat_args (27 passed) and --lib cli::turbo_args (19 passed)
• cargo test --profile test-fast --features metal,accelerate --test llama_compat_manifest (4), --test llama_model_source_cli (16), --test cli_help_consistency (25)
• cargo test --profile test-fast --features metal,accelerate --lib cli:: (107), --lib server::cli_input (120), --lib server::startup (61), --lib downloader:: (162)
• cargo test --profile test-fast --features metal,accelerate --bin mlxcel (214) and --bin mlxcel-server (28)
• make verify-llama-compat
• cargo clippy --profile test-fast --features metal,accelerate --lib --tests --bins -- -D warnings
• cargo fmt --all -- --check

Review round two

A review found two false rejections that would refuse a command line llama-server starts, a diagnostic class naming flags mlxcel does not have, and the short spellings still going nowhere. All fixed in the second commit.

• --defrag-thold is a no-op upstream, so it is inert here. b10621's handler is literally GGML_UNUSED(params); GGML_UNUSED(value); plus a deprecation warning. Rejecting it refused a command line that starts upstream, and the manifest note claimed a defragmentation pass upstream no longer has.
• read_model_layer_count reproduced the fix: KV estimation misses n_layer / n_embd for GPT-2 and GPT-BigCode #927 gap. It read num_hidden_layers by hand; execution::config_fields::LAYER_COUNT_KEYS exists because that alias chain was already copied five times. gpt2 and gpt_bigcode use n_layer, dbrx / falcon_ocr / jina_vlm / molmo use n_layers, exaone uses num_layers; on all of them every non-negative --gpu-layers was rejected, including a full offload, and this PR newly binds LLAMA_ARG_N_GPU_LAYERS, so a container carrying LLAMA_ARG_N_GPU_LAYERS=99 would have stopped starting.
• --load-mode is now the authority over the deprecated loading flags. All four write one field upstream in command-line order, and b10621 deprecates the three in favour of the first while warning that combining them is last-wins, so --no-mmap --load-mode mmap asks for mmap. The residue (a deprecated flag written after --load-mode) is recorded as a divergence on all four entries rather than reconstructed from argv order.
• The diagnostics named flags that do not exist. --tensor-parallel, --pipeline-parallel and --peers are now --tp-size, --pp-size and --cluster-peers. An alternative that produces "unexpected argument" misses the issue's own acceptance criterion.
• All twenty-eight upstream short spellings now reach their option. -ngl is the most common llama-server flag and clap read every one of these as a cluster of one-letter shorts. SHORT_ALIASES in the argv pre-pass gains them, whether the rewritten option consumes the next entry is now read from the command rather than assumed, and a test asserts no table token shadows an mlxcel short.
• f32 and bf16 are float types, not quantizers, and get their own sentence naming the real reason (mlxcel's KV cache has no f32 or bf16 storage).
• Smaller: the KV cache type is validated before the model resolves, so --cache-type-k q8_0 --hf-repo … reports the value instead of downloading first; --threads accepts any value <= 0 (upstream's own "use hardware concurrency"); inert numeric values compare as numbers so 00 and +0 work; --numa "" is b10621's distribute; and --load-mode none|dio no longer offers MLXCEL_WIRED_LIMIT, which is a residency control.

The manifest validator now checks the function half of a file::name test pointer. Several entries here pointed at an integration test that did not cover them and nothing noticed, because only the file half was validated. scripts/ci/check_llama_compat_manifest_test.sh gains the negative case and a positive control, and every entry's pointer now names the test that actually exercises it.

Closes #1445