feat(server): add Vertex AI (GCP) predict route adapter for llama-server b10621 compatibility #1456

Parent

• feat(server): classify and implement optional UI, tools, MCP, and GCP surfaces #1435 (classification sub-issue of the llama-server b10621 epic epic(server): close llama-server b10621 compatibility gaps #1431)

Problem

llama-server b10621 registers Google Cloud Vertex AI custom-container compatibility purely from environment variables (tools/server/server-http.cpp, gcp_params and register_gcp_compat, reference commit c1d0e7a004015f23bc0233470b747b596f29b264). mlxcel-server has none of it, although every route the adapter fans out to already exists in mlxcel. #1435 classified this surface as Supported because it is a pure HTTP adapter over existing handlers and matters for the Linux/CUDA deployment target where Vertex custom containers run.

Upstream semantics to match

• Activation: only when AIP_MODE=PREDICTION. Otherwise nothing is registered and the variables are ignored.
• AIP_HTTP_PORT (default 8080) overrides --port; upstream logs a warning when the two differ.
• AIP_HEALTH_ROUTE (default empty, meaning the existing /health only): when set, a leading slash is ensured and the value is registered as a GET alias of the health handler.
• AIP_PREDICT_ROUTE (default /predict): a leading slash is ensured; startup fails if the path collides with an already registered route.
• Request: JSON object with an instances array (max 128 entries). Each instance is an object that must carry a string @requestFormat; the field is stripped and the remainder is forwarded as the request body. A client-provided stream field is ignored with a warning; streaming responses are rejected because the predict route is non-streaming.
• Alias table: every registered route path is converted with path_to_gcp_format: strip a leading /v1, strip the leading /, stop at the first : path parameter, and convert /, -, _ boundaries to camelCase (/v1/chat/completions becomes chatCompletions, /v1/embeddings becomes embeddings, /v1/messages becomes messages, /tokenize becomes tokenize). First registration wins on collision.
• Instances are executed concurrently and results are returned in order as {"predictions":[...]}; per-instance failures become an error object in that slot rather than failing the whole batch. Malformed envelopes return 400 with the llama-server error envelope.

Scope

• Add an env-gated adapter in the axum app builder that registers the health alias and the predict route and applies the AIP_HTTP_PORT override with the same precedence as upstream, on both mlxcel serve and mlxcel-server.
• Build the alias table from mlxcel's actual registered routes rather than a hand-maintained list, so new routes are picked up automatically.
• Dispatch each instance through the existing handlers in-process (for example tower::ServiceExt::oneshot on the router) so request validation, API-key auth, and rate limits apply unchanged.
• Enforce the 128-instance cap, reject streaming, and bound per-instance concurrency so one predict call cannot starve the decode queue.
• Document the variables in docs/environment-variables.md and add the surface to the b10621 compatibility manifest from test(server): add a pinned b10621 compatibility manifest and conformance harness #1443.

Acceptance criteria

• With AIP_MODE unset, /predict is not mounted and the AIP_* variables have no effect.
• With AIP_MODE=PREDICTION, POST /predict accepts chatCompletions, completions, embeddings, rerank, messages, and tokenize instances and returns {"predictions":[...]} in request order.
• Malformed envelopes (non-object body, missing or non-array instances, more than 128 instances, instance without string @requestFormat, unknown format, streaming request) return the documented 400 or per-instance error shape matching b10621.
• AIP_HTTP_PORT overrides --port, AIP_HEALTH_ROUTE mounts a public health alias, and a colliding AIP_PREDICT_ROUTE fails startup with a diagnostic.
• API-key authentication and request-size limits apply to the predict route exactly as to the aliased routes.
• Integration tests cover the above, and a real-checkpoint smoke test exercises one chatCompletions batch against a dense text model.

References

• https://github.com/ggml-org/llama.cpp/blob/c1d0e7a004015f23bc0233470b747b596f29b264/tools/server/server-http.cpp
• https://docs.cloud.google.com/vertex-ai/docs/predictions/custom-container-requirements#aip-variables