Repository navigation
openai.Completion.acreate(stream=True) raises an Exception #171
Description
See stack trace below
Traceback (most recent call last):
File "/Users/nuno/dev/concat/server-py/concat/conversation.py", line 75, in handle_conversation
await run
File "/Users/nuno/dev/concat/server-py/concat/agent/openai/openai.py", line 122, in __call__
msg = await ctx.stream_message_async(
File "/Users/nuno/dev/concat/server-py/concat/context.py", line 287, in stream_message_async
async for text in iterator:
File "/Users/nuno/dev/concat/server-py/concat/agent/openai/openai.py", line 60, in openai_completion_stream
iterator = await openai.Completion.acreate(*args, **kwargs, stream=True)
File "/Users/nuno/Library/Caches/pypoetry/virtualenvs/platform-api-30l3Kv3P-py3.10/lib/python3.10/site-packages/openai/api_resources/completion.py", line 45, in acreate
return await super().acreate(*args, **kwargs)
File "/Users/nuno/Library/Caches/pypoetry/virtualenvs/platform-api-30l3Kv3P-py3.10/lib/python3.10/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 230, in acreate
return (
TypeError: 'async_generator' object is not iterable
I believe this should be an async for in line 239 of engine_api_resource.py. Once I make that change, then I get this error
Traceback (most recent call last):
File "/Users/nuno/Library/Caches/pypoetry/virtualenvs/platform-api-30l3Kv3P-py3.10/lib/python3.10/site-packages/openai/api_requestor.py", line 645, in _interpret_response_line
data = json.loads(rbody)
File "/opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "/opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 2 column 1 (char 1)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/nuno/dev/concat/server-py/concat/conversation.py", line 75, in handle_conversation
await run
File "/Users/nuno/dev/concat/server-py/concat/agent/openai/openai.py", line 122, in __call__
msg = await ctx.stream_message_async(
File "/Users/nuno/dev/concat/server-py/concat/context.py", line 287, in stream_message_async
async for text in iterator:
File "/Users/nuno/dev/concat/server-py/concat/agent/openai/openai.py", line 61, in openai_completion_stream
async for chunk in iterator:
File "/Users/nuno/Library/Caches/pypoetry/virtualenvs/platform-api-30l3Kv3P-py3.10/lib/python3.10/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 230, in <genexpr>
return (
File "/Users/nuno/Library/Caches/pypoetry/virtualenvs/platform-api-30l3Kv3P-py3.10/lib/python3.10/site-packages/openai/api_requestor.py", line 611, in <genexpr>
self._interpret_response_line(
File "/Users/nuno/Library/Caches/pypoetry/virtualenvs/platform-api-30l3Kv3P-py3.10/lib/python3.10/site-packages/openai/api_requestor.py", line 647, in _interpret_response_line
raise error.APIError(
openai.error.APIError: HTTP code 200 from API (
)
which appears to be caused by trying to json-decode the following string \n. Once I fix that (by doing eg. data = json.loads(rbody) if rbody.strip() else None, it appears the stream never finishes, ie. the next part never arrives
Activity
nfcampos commentedon Jan 7, 2023
Obviously this is using the newly released version 0.26.0
nfcampos commentedon Jan 7, 2023
The final issue is in this line
openai-python/openai/api_requestor.py
Line 297 in 4fee0da
We exit the
withstatement before the response stream is consumed by the caller, therefore, unless we're using a global ClientSession, the session is closed (and thus the request) before it should be.ddeville commentedon Jan 12, 2023
Thanks for reporting and debugging this @nfcampos, we'll take a look and hopefully we'll get a fix out shortly.
ddeville commentedon Jan 12, 2023
Oh I just saw your PR, thank you so much, I'll review it.
[doc] add mlflow tracing integration (openai#171)