[autrk]

Overview

When running llama-server in router mode with --timeout 0, every proxied request fails with:

proxy error: Failed to read connection

Root Cause

The router proxy reuses timeout_read/timeout_write for its internal HTTP client (httplib::ClientImpl), which uses poll() - where {0, 0} means "return immediately", not "no timeout". This differs from the non-router server, which uses setsockopt(SO_RCVTIMEO) where {0, 0} correctly disables the timeout (blocking mode).

Fix

Skip setting httplib timeouts when the value is 0, falling back to httplib's built-in defaults (connection_timeout: 300s, write_timeout: 5s, read_timeout: 300s). Note that --timeout 0 in router mode therefore does not mean "no timeout" - users who need longer timeouts should set an explicit value like --timeout 86400.

Additional information

Commit ff6b106 (PR #22003) replaced the hardcoded 5-second connection timeout in server_http_proxy with the user-configurable timeout_read value:

cli->set_connection_timeout(timeout_read, 0); // was: (5, 0)

The same issue affects set_write_timeout(timeout_read, 0) and set_read_timeout(timeout_write, 0).

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: AI was used in an assistive capacity during debugging and implementation. I do fully understand the code and can explain it.

[ggml-gh-bot]

Hi @autrk, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

• AI-generated content: This project does not accept PRs, descriptions or commit messages that are fully or predominantly AI-generated. If you have used AI to assist you in writing code, please make sure to disclose that explicitly.

---

Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.