[BUG] AiProxy switches providers after partial SSE output and mixes stream content

[Aias00]

Description

AiProxyExecutorService.executeDirectStream applies retry to the main provider Flux and then unconditionally switches to the fallback provider through onErrorResume.

If the main provider emits one or more chunks before failing, those chunks have already been written to the client SSE response. The fallback provider then starts a new completion from the beginning and appends its chunks to the already committed response.

This issue is distinct from #6647:

• [BUG] AiProxy stream retry replays the whole upstream, duplicating SSE content #6647 tracks replaying the main upstream after a mid-stream retry.
• This issue tracks switching to a different fallback provider after partial output, producing one SSE response containing content from multiple completions/providers.

Related PR: #6341, reviewed at head 8fa93d7446e052f149d3740bbe7ff2052c375e23.

Affected code

• shenyu-plugin-ai-proxy/.../AiProxyExecutorService.java: retryWhen(...).onErrorResume(...fallback...) wraps the whole streaming Flux.
• shenyu-plugin-ai-proxy/.../AiProxyPlugin.java: the Flux is written directly to the committed SSE response.

Reproduction

An equivalent Reactor sequence:

main provider: emit "main-partial" -> error
retry:         emit "main-partial" -> error
fallback:      emit "fallback-completion"

produces:

main-partial, main-partial, fallback-completion

Even after #6647 removes the retry replay, an unconditional fallback after the first emitted chunk would still produce:

main-partial, fallback-completion

Impact

• Clients receive a syntactically valid SSE stream with semantically corrupted, mixed completion content.
• Tool calls or structured output can be duplicated or combined across providers.
• Multiple providers may bill for the same client request.
• The response is already committed, so the gateway cannot safely replace earlier chunks.

Expected behavior

Provider retry and fallback are allowed only before the first response chunk is emitted. After any chunk is sent, an upstream failure must terminate the current stream without starting another provider completion.

Acceptance criteria

• Track whether the main stream has emitted a chunk.
• Allow retry/fallback only before the first emission.
• After partial emission, propagate/terminate the stream without invoking fallback.
• Add a regression test where the main provider emits a chunk and then errors while fallback is configured.
• Assert no duplicated prefix and no fallback chunk is appended after partial output.
• Retain the existing error-before-first-chunk fallback behavior.
• Coordinate the retry portion with [BUG] AiProxy stream retry replays the whole upstream, duplicating SSE content #6647 rather than implementing two competing fixes.

[github-actions]

👋 Hello @yuluo-yx

A new issue has been opened by @Aias00:

• Title: [BUG] AiProxy switches providers after partial SSE output and mixes stream content
• Link: [BUG] AiProxy switches providers after partial SSE output and mixes stream content #7021

Please review when you have time. Thank you! 🙏

[zhang-arvin]

I would like to work on this issue.