Skip to content

LLM clients have no timeouts + blocking call in async handler — prompt can hang forever #1

Description

@grave0x

Problem

All LLM HTTP clients are created with no timeout: reqwest::blocking::Client::new() in crates/llm/src/lib.rs:182 (Anthropic), :227 (OpenAI), :251 (OpenAI-compatible). A blackholed connection (established but never delivering data — e.g. the IPv6-vs-IPv4 path upstream root-caused in anomalyco/opencode#39859) hangs forever.

Worse: provider.complete() is a blocking call invoked directly inside an async tokio handler at crates/server/src/handler/session.rs:270. A hung provider stalls the entire server handler (no spawn_blocking, no timeout), not just one request.

This mirrors upstream anomalyco/opencode#40069 ("apply default header timeout to all providers", closes #39859) — upstream applied a 300s header timeout at the fetch layer to every provider because only OpenAI had one.

Impact

  • POST /api/session/:id/prompt can hang indefinitely when a provider connection blackholes.
  • No error is ever surfaced; operator must kill the process.

Spec / Acceptance criteria

  1. All provider clients (AnthropicProvider, OpenAIDirectProvider, OpenAICompatibleProvider) get explicit timeouts:
    • connect timeout ~30s, header timeout ~300s (config-overridable, false disables — keep ??-style semantics for an optional timeout_ms).
  2. Run complete() off the async executor: tokio::task::spawn_blocking with a timeout wrapper in the session handler (crates/server/src/handler/session.rs:262-286).
  3. On timeout/error: return a structured error response (status: "error" + message) instead of hanging or silently falling back to admitted.
  4. Test: mock a server that accepts TCP but never responds → prompt must fail within bounded time.

Reference: upstream anomalyco/opencode#40069 (closes #39859); our code: crates/llm/src/lib.rs, crates/server/src/handler/session.rs.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions