Repository navigation
Stream error after 5 minutes #399
Description
Activity
I have the exact same issue. I can generate a little over 2'000 tokens via GPT-4 stream and I stop receiving stream updates after exactly 300 seconds.
Running on something similar as this code: https://github.com/Nutlope/twitterbio/blob/e799871764e559cd96608bfefcc3976e7daf346a/utils/OpenAIStream.ts#L26
The default timeout of the connection with aiohttp library(that openai librayr utilizes) is 300 seconds.
Have you changed the timeout period when you send your request?
It's an optional parameter timeout_period and you set it's value in seconds.
@EthanBlackburn if you provide me a generator prompt/model-name for a long text I can test in my custom API client if the connection is closed from server side or it's an issue with the openAPI python library.
@oomti This prompt should exceed 5min with GPT-4.
Please write a JSON that counts from 1 to 500 like this:
{
"One": 1,
"Two": 2,
}Thanks @karim-attia
Tested with a 600seconds timeout on my part.
After 5 minutes the stream has ended with the following error message:
openai-chatgpt\openai\Lib\site-packages\aiohttp\streams.py", line 304, in _wait await waiter aiohttp.client_exceptions.ClientPayloadError: Response payload is not completedThis indicates that the socket from the serverside is disconnected after 5 minutes, which is probably a default setting.
Reacted by Karim Attia, Xiaoyong Zhu and Shantanu NairDoublechecked and tested with various settings -
timeout=600, stream=true - connection closed in 5 minutes
timeout=600, stream=false - connection closed in 5 minutes(so won't receive a response)Also tested with the official web UI - completes only 87 lines and closes response after(there might be other limitations there)
I think the large token count was designed with a larger input in mind and a large sized output was not considered.
Reacted by Karim Attia, Ethan Blackburn, yannoth, Jack Webb-Heller and Andrey SaksonovSame bug here
Reacted by Karim Attiasame here
requests.exceptions.ChunkedEncodingError: ("Connection broken: InvalidChunkLength(got length b'', 0 bytes read)", InvalidChunkLength(got length b'', 0 bytes read)) ________________________________________________________ Executed in 301.97 secs fish external usr time 2.23 secs 0.09 millis 2.23 secs sys time 1.41 secs 1.76 millis 1.40 secsI met the same issue too with 4000 toknes prompt and asking near 4000 toknes.
timeout=1000, stream=true - connection closed in 5 minutes (call: invalid chunk Connection broken: InvalidChunkLength(got length b'', 0 bytes read)
timeout=1000, stream=false - connection closed in 5 minutes(so won't receive a response)We find out this is not a bug of client, but a bug of the openai server
Reacted by Xiaoyong ZhuI have the exact same error, and interact with ChatGPT and documentation, it seems to be a server side error.
"
From the information you've provided, it does indeed seem that there's a timeout issue with the OpenAI GPT-4 streaming API that's closing the connection after exactly 5 minutes (300 seconds). This could be due to a server-side setting.As for the error you're encountering, InvalidChunkLength(got length b'', 0 bytes read), it's likely that when the server-side socket disconnects after 5 minutes, the client is still trying to read from the connection and encountering an empty string, which leads to this error. This seems to be supported by the fact that the error is thrown when trying to parse an empty string as an integer, which is invalid.
However, without having access to the underlying implementation of the OpenAI API, it's hard to know for sure. It's also worth noting that as an AI developed by OpenAI, I don't have the ability to access or manipulate the OpenAI servers in real-time.
As I recommended earlier, the best course of action would be to reach out to OpenAI's support or raise an issue on the OpenAI GitHub repository. They would be best equipped to address this issue and potentially provide a fix or workaround.
""
anyone has possible solutions?
Reacted by DUT-LiuYangI have the same issue when I generate long text using GPT-4.
Reacted by DUT-LiuYang and Jack Webb-HellerSame issue with long GPT-4 outputs, although I'm using the PHP client. After extensive debugging I'm pretty certain the issue is server-side with OpenAI, but besides this thread, this is the only place I've found it being discussed.
The issue seems to affect both standard and streamed responses. With standard responses on nginx, the OpenAI server closes the connection after 5 minutes, and nginx reports this as a
Gateway Timeout.With streamed responses, the final delta doesn't specify a
finish_reason, suggesting the model is still running. But the stream is terminated anyway.I can reproduce this on both macOS and Ubuntu. Changing nginx directives makes no difference, nor does overriding HTTP client timeouts.
This appears to be a recent change, it worked as expected for me up until last week. Hopefully this is fixed soon — I've sent a link to this comment to OpenAI's support, hopefully someone sees it 🤞
6 remaining items
Does anyone have any update regarding this issue? I met this error when using gpt-4-0613 and gpt-3.5-turbo-16k-0613 with stream=True :(
I read this post too https://community.openai.com/t/how-to-set-a-timeout-on-an-api-function-call-using-using-the-python-library/4213/4, but I'm confused at the parameter's name, should it be
timeoutorrequest_timeout? Or they both exist and function differently?By the way, timeout-relevant parameters were not mention in openai's api documentation https://platform.openai.com/docs/api-reference/chat/create, so I didn't realize these parameters were available in the first place.... I have to say I am so overwhelmed with all the scattered documentations... what api documentations do you guys refer to mostly ...
- added a commit that references this issue
on Jul 8, 2023 By the way, timeout-relevant parameters were not mention in openai's api documentation https://platform.openai.com/docs/api-reference/chat/create
The timeout parameters you mentioned aren’t to do with the API, they’re specific to the Python client (hence not mentioned in the API docs). Those timeout params just provide a way to close the connection after a fixed time.
The 5-minute timeout issue discussed here is unrelated to any API parameters, and seemingly beyond our control, apparently due to an OpenAI server misconfiguration.
mark
Meet the same issue, any help/workaround will be appreciated!
Facing this issue as well.
@oomti , I see the same error you do. Can you confirm the OP issue is related? Did you find a fix?
This is fixed in v1; please upgrade to the latest version of the library.
This is fixed in v1; please upgrade to the latest version of the library.
That's great news! @rattrayalex: Can you share why upgrading to the newest version of the library fixes the issue? The problem is (was?) with the underlying REST API, so either the servers' configuration has changed, or REST users need to update their code. (I suspect the former.)
I believe there was an issue in the client (no longer there) and a limitation on the server side which has also been improved since then (other users have reported they hit a timeout at 10min recently).
Reacted by Christian Cantrell- added a commit that references this issue
on Feb 27, 2024 - added a commit that references this issue
on Mar 27, 2025
Describe the bug
Streaming output from gpt4 consistently fails with
Connection broken: InvalidChunkLength(got length b'', 0 bytes read)for prompts that trigger long-form responses. I've been able to reproduce this by prompting gpt to rewrite a long document.A few observations
Code snippet from
urllib3/response.py:It seems like a problem on the server side but Im not confident
To Reproduce
Code snippets
No response
OS
macOs
Python version
Python 3.10.9
Library version
v.0.26.4