[anirbanbasu]

Fixes #17 and #16.

Summary

• default_device() (src/clm/heads.py) now tries MPS after CUDA before falling back to CPU, so clm-serve runs the projection heads on Apple Silicon's GPU without any flags. --device/CLM_DEVICE still override it.
• vllm is now gated to sys_platform == 'linux' in pyproject.toml, since vLLM has never published a macOS wheel — this was silently breaking pip install -e . / pip install contrastive-lm on macOS (see installation on mac #16), with pip backtracking through vllm sdists that need a Linux/CUDA build toolchain.
• README documents running the embedder with llama.cpp's llama-server --embedding --pooling last as a drop-in replacement for vllm serve on Apple Silicon — Embedder (src/clm/embedder.py) already speaks plain OpenAI-compatible /v1/embeddings, so no code changes were needed there.
• train/finetune.py and evaluation/bon_eval.py hardcoded cuda-or-cpu device selection, so fine-tuning and best-of-N evaluation silently ran on CPU on Apple Silicon even though torch sees MPS. Both now resolve their --gpu flag through a small _device(gpu) helper that falls back to default_device() (so MPS is picked automatically) instead of cpu directly.
• serve_qwen3_8b.sh was vLLM-only. It now takes an ENGINE variable (vllm or llama.cpp), auto-detected from the OS (Darwin → llama.cpp) and overridable, so the same script launches llama-server with the flags above on Apple Silicon instead of vllm serve.
• VectorArena (src/clm/cache.py), which backs clm-serve's --action-cache, only queried real device memory for its budget on CUDA (torch.cuda.mem_get_info); on mps it silently assumed a fixed 8 GiB total/free, making the 0.02-style fraction budget meaningless on Apple Silicon machines with much less (8 GiB) or much more (36-128 GiB) unified memory. It now reads torch.mps.recommended_max_memory() / driver_allocated_memory() (torch 2.3+) when on mps, falling back to the old fixed guess only if that API isn't available.

Test plan

Verified end-to-end on Apple Silicon (M-series, 64GB unified memory):

• llama-server -hf unsloth/Qwen3-8B-GGUF:BF16 --embedding --pooling last -ngl 1024 -fa on -c 2048 --port 8090 serves the embedder correctly (checked usage.prompt_tokens for single and batched inputs).
• pip install -e . succeeds in a clean venv on macOS with no vllm installed.
• clm-serve logs models [...] on mps and reserves the vector cache on mps, with no --device flag passed.
• POST /v1/systemone and Engine.rank(...) return correct, sensible answers (e.g. ranks "The Moon's gravitational pull." highest for "What causes tides on Earth?").
• GET /health reports embedder: true and device mps.
• serve_qwen3_8b.sh resolves ENGINE=llama.cpp automatically on Darwin and ENGINE=vllm when explicitly overridden.
• VectorArena('mps', budget=0.02) reserves memory scaled to this machine's actual unified memory via torch.mps.recommended_max_memory() (not the old fixed 8 GiB guess), and arena.get(...) correctly fills, hits and returns mps-device tensors.

Related PRs

Three other Apple Silicon efforts are already open/closed against this repo; this PR takes a smaller, dependency-light approach:

• Add an optional Apple Silicon encoder path for CLM #5 (closed) adds a bespoke MLX embedding server plus an FP8-to-MXFP8 converter script for Qwen3-8B — new Python modules, a new conversion step, and a custom quantized checkpoint. This PR adds no embedding-server code at all: it points Embedder (already generic OpenAI-compatible HTTP) at llama-server, an existing external binary, and uses an off-the-shelf GGUF instead of converting anything.
• Add PyTorch MPS serving with the original Qwen3-8B weights #7 (open) adds a new clm-mps-embed command that loads Qwen3-8B safetensors in-process via PyTorch/Transformers on MPS (src/clm/mps_embed.py, a new macos extra, plus changes to cache.py, engine.py, server.py, evaluation/bon_eval.py, and the playground JS). This PR only touches default_device() in heads.py (and now the same pattern in finetune.py/bon_eval.py), plus a small MPS memory-query fix in cache.py's existing VectorArena — the encoder stays a separate llama-server process, so there's no new in-repo embedding implementation to maintain.
• Add optional Apple Silicon MLX encoder and Metal heads #9 (open) adds a full mlx device backend: MLX-based heads (mlx_heads.py), an MLX embedding server (mlx_embed.py), and unified-memory preflight logic (memory.py) — a new backend alongside cpu/cuda. This PR doesn't introduce a new device backend; it only extends PyTorch's existing mps support in default_device(), so the heads stay on stock PyTorch.
• Like Add PyTorch MPS serving with the original Qwen3-8B weights #7 and Add optional Apple Silicon MLX encoder and Metal heads #9, this PR also gates vllm behind a sys_platform marker in pyproject.toml so it doesn't block pip install on macOS — same idea, arrived at independently.

Net effect: no new source files, no new dependencies, and no new device string — just teaching default_device() (and the two training/eval scripts that duplicated its logic) about MPS, and treating llama-server as an already-installed external tool that speaks the API CLM already expects. Trade-off: unlike #7/#9, there's no bundled one-command setup script for fine-tuning from scratch on Apple Silicon; serve_qwen3_8b.sh covers the encoder, but train/finetune.py's from-scratch embedding path still needs a reachable --embed-url (llama-server) since there is no in-process llama.cpp Python backend equivalent to vLLM's offline LLM(...).

[anirbanbasu]

Note that llama.cpp must be installed on macOS. The Homebrew managed installation is simple enough to get started, see: https://formulae.brew.sh/formula/llama.cpp.