Skip to content

chat: an error sent inside the stream is dropped and the answer is shown as complete #516

Open

Description

@rominf

Current behavior

The OpenAI-compatible stream parser used by rocm chat ignores an error object delivered inside the stream. Given

data: {"choices":[{"delta":{"content":"partial"}}]}
data: {"error":{"message":"...","type":"..."}}
data: [DONE]

the result is the text partial followed by a clean end-of-stream event. No error is surfaced, so a truncated answer is presented as though it were complete.

Expected behavior

An in-band error ends the stream as an error, and the user is told the answer was cut short.

Inconsistency worth noting

The Anthropic stream parser in the same module does stop on its error event type. The two parsers disagree about whether a mid-stream error is fatal.

Steps to reproduce

Feed the parser the three lines above. A reproducer is written as a property test (in_band_stream_errors_are_not_reported_as_a_clean_finish).

Caveat

The error shapes in the reproducer were written from vLLM's streaming handler, not captured from a live failing response. Both a nested {"error":{...}} object and a bare error object were tested. If an engine is known to emit a different shape, the fix should cover it.

Why this is filed rather than fixed

Whether a mid-stream error should discard the partial text, keep it with a warning, or keep it silently is a product decision about what rocm chat shows. The defect is only that it is currently invisible.

Environment

  • Built from main. Platform-independent.

How it was found

Property-testing the stream readers. Parsing itself is chunking-invariant — every one- and two-point split of the repo's fixture bodies, including splits mid-UTF-8 and between \r and \n, produced identical events — so this is a semantic gap, not a framing bug.

Activity

  1. added a commit that references this issue on Nov 17, 2026
    088e75f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions