[Bug] Codex WS upstream: any request >=16 MiB dies as 502 upstream_server_error, with no SSE fallback #2471

Summary

Since 2.31.0 — the first release bundling Bun 1.4.0, which satisfies MIN_BOUNDED_CODEX_WS_BUN_VERSION — every streaming ChatGPT-Codex turn rides the responses_websockets transport. The backend closes any WS message larger than 16 MiB, and codexWsUpstreamFetch only falls back to SSE when the upgrade fails. A close after open produces a hard 502 upstream_server_error with no fallback and no actionable message.

The practical effect: once a thread's serialized request crosses 16 MiB (trivially reached by pasting ~11–12 screenshots under full replay), that thread is permanently dead — every retry re-sends the same oversized frame. Other threads on the same account keep working, so it looks like a per-conversation upstream fluke rather than a size limit.

Measured threshold (byte-exact)

Replaying a real thread's payload through the local proxy:

16,777,000 B = 16.000 MiB -> OK   (6.6s)
16,777,300 B = 16.000 MiB -> FAIL (1.0s)

The boundary is exactly 16 MiB = 16,777,216 B. 100% reproducible; failures return in ~1–2s with streamAborted: true.

This is not a Bun client-side cap. A local Bun WS server with maxPayloadLength: 512 MiB happily accepts a 20 MiB frame from the same Bun client. The limit is enforced by the receiving side (chatgpt.com), consistent with the documented upstream close code 1009 Message Too Big.

Real-world trigger

A thread with 15 pasted clipboard screenshots — ~1.4 MiB base64 each, 20.9 MiB of images, 22.6 MiB of serialized items. Under full replay every turn resends all of it, so the thread failed 100% of the time from the moment it crossed the line, while concurrent threads (~800 KB payloads) on the same account and model were unaffected.

Diagnosis is currently very hard

usage.jsonl and /api/logs only expose status: 502, errorCode: upstream_server_error, streamAborted: true. The real reason appears nowhere in opencodex's own telemetry — it is only visible in the client's rollout:

~/.codex/sessions/<date>/rollout-*.jsonl  →  task_complete.error
"stream disconnected before completion: Upstream stream terminated unexpectedly:
 codex websocket closed before a Responses terminal event"

The close handler in src/server/responses/ws-upstream.ts discards the close code and reason, so a 1009 is indistinguishable from an ordinary network drop.

Why it started now

• WS transport landed 2026-08-12 (6f2be3514), gated on MIN_BOUNDED_CODEX_WS_BUN_VERSION = "1.4.0".
• 2.28.0 / 2.29.0 shipped Bun 1.3.14 → gate closed → everything went over SSE POST, which has no 16 MiB ceiling.
• 2.31.0 (published 2026-08-22; 2.30.0 was never published to npm) is the first release with Bun 1.4.0 → gate opens → the ceiling becomes reachable for every user on that build.

How upstream Codex handles the same wall

Official Codex hits it too — openai/codex#24550 reports a hard fallback once replacement_history reached ~17.38 MB with 42 inline images — but it has three mitigations we have none of:

1. HTTP/SSE fallback. core/src/client.rs logs falling back to HTTP (codex.transport.fallback_to_http) and sets disable_websockets once stream retries are exhausted.
2. Within-turn incremental append. previous_response_id + get_incremental_items() over the reused socket; only deltas after the first create. Note this is strictly within one turn (fresh ModelClientSession per turn, store: false), so it does not avoid the ceiling on a turn's first frame.
3. Client-side image budget / resize. resize_all_images, unified_image_budget, compaction_image_budget, and "default high resized behavior".

The same class of proxy bug, with the same recommended fix, is documented in router-for-me/CLIProxyAPI#4409: estimate the serialized size before opening the WS, bypass to SSE when it exceeds the limit, and do not rotate credentials on a 1009 — otherwise one oversized payload gets retried against every account in the pool.

Proposed fix, in three cycles

Cycle 1 — stop the bleeding (small, self-contained)

• Pre-flight size check in shouldUseCodexWsUpstream / codexWsUpstreamFetch: if the serialized frame is at or above a threshold (16 MiB minus headroom), skip WS and take the SSE path directly.
• Treat "socket opened, closed before any event arrived, nothing streamed downstream" as retryable over SSE rather than a terminal 502.
• Capture the WS close code and reason. Map an oversized close (1009) to a distinct errorCode with an actionable message instead of a generic upstream_server_error, and suppress account rotation for that code.

Cycle 2 — bound what goes on the wire (medium)

• Mirror the official image budget: downscale inline input_image payloads above a size/dimension threshold and cap total image bytes per request, with a resize notice. This is what actually keeps the ceiling out of reach — under full replay each pasted screenshot is permanent rent, and ~11–12 of them is enough to cross 16 MiB.

Cycle 3 — within-turn incremental append (large; performance, not correctness)

• Pool the upstream WS per conversation across the inbound HTTP round-trips of a single turn. Send previous_response_id + delta items when the request is a strict superset of (previous input + returned items) and the non-input properties match: model, instructions, tools, tool_choice, reasoning, store, stream, include, service_tier, prompt_cache_key, text.
• src/responses/state.ts already retains per-response items plus providerOutputStart, which is exactly the anchor this diff needs.
• Must not reuse a socket across turns (sticky-routing token contract, per the upstream comment). Main open risk: a pooled socket pins the account, which conflicts with rotation / pacing / failover.
• Benefit: today every tool round-trip re-uploads the entire context (~21 MiB per call in the thread above); this removes that amplification. It does not raise the 16 MiB ceiling.

Repro

import json, urllib.request, time

def probe(target_bytes):
    pad = "x" * target_bytes
    body = json.dumps({
        "model": "gpt-5.6-sol",
        "input": [{"role": "user", "content": [{"type": "input_text", "text": pad}]}],
        "stream": True, "max_output_tokens": 1500,
    }).encode()
    req = urllib.request.Request(
        "http://127.0.0.1:10100/v1/responses", data=body,
        headers={"Content-Type": "application/json", "Authorization": "Bearer local"})
    t0 = time.time(); done = False
    r = urllib.request.urlopen(req, timeout=300)
    for raw in r:
        line = raw.decode(errors="replace").strip()
        if line.startswith("data:") and line[5:].strip() not in ("", "[DONE]"):
            try: ev = json.loads(line[5:].strip())
            except Exception: continue
            if ev.get("type") == "response.completed": done = True
    print(f"{len(body)} B = {len(body)/1048576:.3f} MiB -> {'OK' if done else 'FAIL'} {time.time()-t0:.1f}s")

probe(16_700_000)   # OK
probe(16_800_000)   # FAIL — socket closed before any terminal event

(The real thread reproduced identically with 15 genuine input_image items; text padding is only for a byte-exact boundary.)

Environment

• opencodex 2.31.0 (npm global, bundled Bun 1.4.0)
• codex-cli 0.149.1, macOS (darwin 25.5.0)
• model gpt-5.6-sol, service tier priority, ChatGPT auth