Skip to content

[codex] Skip parsing incomplete response text - #3427

Open
fengjikui wants to merge 1 commit into
openai:mainfrom
fengjikui:codex/skip-incomplete-response-parse
Open

fengjikui wants to merge 1 commit into
openai:mainfrom
fengjikui:codex/skip-incomplete-response-parse

Conversation

@fengjikui

@fengjikui fengjikui commented Jun 23, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • avoid structured-output parsing for response message text unless the message is completed
  • preserve incomplete output text while leaving parsed as None
  • add a regression test for incomplete JSON output in parse_response

Fixes #2486.

Root cause

parse_response attempted to parse every message.output_text item with the requested structured output type. For incomplete responses, that text can contain partial JSON, so Pydantic raises a validation error instead of returning the incomplete response payload.

Validation

  • PYTHONPATH=src uv run --frozen --python 3.9 --with pytest --with pytest-asyncio --with respx --with inline-snapshot pytest tests/lib/responses/test_responses.py::test_parse_incomplete_output_text -q -o addopts="" (1 passed)
  • env -u HTTP_PROXY -u HTTPS_PROXY -u ALL_PROXY -u http_proxy -u https_proxy -u all_proxy PYTHONPATH=src uv run --frozen --python 3.9 --with pytest --with pytest-asyncio --with respx --with inline-snapshot pytest tests/lib/responses/test_responses.py -q -o addopts="" (8 passed)
  • uv run --frozen --python 3.9 --with ruff ruff format --check src/openai/lib/_parsing/_responses.py tests/lib/responses/test_responses.py
  • uv run --frozen --python 3.9 --with ruff ruff check src/openai/lib/_parsing/_responses.py tests/lib/responses/test_responses.py
  • uv run --frozen --python 3.9 --with pyright==1.1.399 --with pytest --with pytest-asyncio --with respx --with inline-snapshot pyright src/openai/lib/_parsing/_responses.py tests/lib/responses/test_responses.py (0 errors)

@fengjikui
fengjikui marked this pull request as ready for review June 23, 2026 16:00
@fengjikui
fengjikui requested a review from a team as a code owner June 23, 2026 16:00
@fengjikui
fengjikui force-pushed the codex/skip-incomplete-response-parse branch from 21b3fca to 387108b Compare July 10, 2026 19:16
@fengjikui

Copy link
Copy Markdown
Contributor Author

Rebased this fix onto current main (f16fbbd) and resolved the test insertion conflict while preserving the upstream program-item coverage. The refreshed head is 387108b.

Validation on Python 3.9: focused regression 1 passed, full tests/lib/responses/test_responses.py 8 passed, Ruff format/check clean, Pyright 0 errors, and the incomplete-JSON reproduction now returns the response with parsed=None.

@jnohclee-rgb

Copy link
Copy Markdown

Independent offline verification (AI-assisted): I extended the status check to 27 constructed cases: three outer response statuses × three message statuses × valid JSON/truncated JSON/refusal. Main passed 15/27; this head passed 27/27. The useful boundaries are that a completed message is still parsed even when the outer response differs; incomplete/in-progress messages retain their text and leave parsed=None; refusals remain unchanged; truncated JSON inside a completed message still raises ValidationError. These cross-status combinations deliberately isolate helper policy, not a claim that the provider emits every combination.

Compared main 919b6236382f3f9f76d8453efb9f9ad03cff1126 with PR head 387108b10f842a02a91d6462caff6aee4c1827b9, Python 3.14.7, isolated source trees, shared existing dependencies and isolated legacy HTTPX compatibility dependencies. Runs denied network access. Not a historical dependency matrix, full SDK or merge approval.

Runnable separately against each source tree:

import importlib,itertools,json
from pydantic import BaseModel
from openai._types import omit
from openai._models import construct_type_unchecked
from openai.types.responses import Response
source=importlib.import_module('openai.lib._parsing._responses')
class Result(BaseModel):name:str
rows=[]
for overall,message_status,content_kind in itertools.product(['completed','incomplete','failed'],['completed','in_progress','incomplete'],['valid','truncated','refusal']):
 text='{"name":"fictional"}' if content_kind=='valid' else '{"name":'
 item={'type':'refusal','refusal':'fictional refusal'} if content_kind=='refusal' else {'type':'output_text','text':text,'annotations':[]}
 raw=construct_type_unchecked(type_=Response,value={'id':'fictional','object':'response','created_at':0,'model':'fictional','status':overall,'output':[{'id':'fictional-message','type':'message','role':'assistant','status':message_status,'content':[item]}]})
 expected_error=content_kind=='truncated' and message_status=='completed';expected_parsed={'name':'fictional'} if content_kind=='valid' and message_status=='completed' else None
 try:
  result=source.parse_response(text_format=Result,input_tools=omit,response=raw);parsed=result.output_parsed;actual=parsed.model_dump() if parsed else None;error=None
  preserved=result.status==overall and result.output[0].status==message_status and (result.output[0].content[0].refusal=='fictional refusal' if content_kind=='refusal' else result.output[0].content[0].text==text)
 except Exception as exc:error=type(exc).__name__;actual=None;preserved=None
 ok=error=='ValidationError' if expected_error else error is None and actual==expected_parsed and preserved is True
 rows.append({'overall':overall,'message_status':message_status,'content_kind':content_kind,'error':error,'actual_parsed':actual,'expected_parsed':expected_parsed,'preserved':preserved,'pass':ok})
print(json.dumps({'source_module':source.__file__,'cases':rows,'passed':sum(r['pass'] for r in rows),'failed':sum(not r['pass'] for r in rows),'scope':'Constructed Responses objects; message completion controls parsing even when outer response differs. Invalid JSON in completed messages still fails; no provider emission claim.'}))

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Incomplete responses are not handled by parse_response when using Structured Output

2 participants