Skip to content

Error: Response payload is not completed #574

Closed
Closed
Error: Response payload is not completed#574
Labels
Azurefor issues relating to the Azure OpenAI servicebugSomething isn't workingfixed in v1Issues addressed by the v1 beta

Description

@zzc0430
zzc0430
opened on Aug 17, 2023

Describe the bug

When using the acreate interface and attempting to utilize streaming output, I encountered "aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed"

  • Time: (UTC) 2023-08-17 06:08:24,030 ~ 2023-08-17 06:13:24,180
  • Request ID: 9c46544d6517b7b8e9df1e9e61fd7f53

To Reproduce

  1. using acreate interface
  2. stream = True

Code snippets

async for chunk in completion_stream:
File "/usr/local/lib/python3.9/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 230, in <genexpr>
return (
File "/usr/local/lib/python3.9/site-packages/openai/api_requestor.py", line 319, in wrap_resp
async for r in resp:
File "/usr/local/lib/python3.9/site-packages/openai/api_requestor.py", line 633, in <genexpr>
return (
File "/usr/local/lib/python3.9/site-packages/openai/api_requestor.py", line 114, in parse_stream_async
async for line in rbody:
File "/usr/local/lib/python3.9/site-packages/aiohttp/streams.py", line 35, in __anext__
rv = await self.read_func()
File "/usr/local/lib/python3.9/site-packages/aiohttp/streams.py", line 311, in readline
return await self.readuntil()
File "/usr/local/lib/python3.9/site-packages/aiohttp/streams.py", line 343, in readuntil
await self._wait("readuntil")
File "/usr/local/lib/python3.9/site-packages/aiohttp/streams.py", line 304, in _wait
await waiter
aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed

OS

Linux

Python version

Python v3.9

Library version

openai-python v0.27.2

Activity

  1. zzc0430
    added
    bugSomething isn't working
    on Aug 17, 2023
  2. logankilpatrick
    added
    Azurefor issues relating to the Azure OpenAI service
    on Sep 6, 2023

  3. logankilpatrick commented on Sep 6, 2023

    @logankilpatrick
    logankilpatrick
    on Sep 6, 2023
    Contributor

    cc @dereklegenzoff

  4. ShantanuNair commented on Sep 7, 2023

    @ShantanuNair
    ShantanuNair
    on Sep 7, 2023

    Looks like this is a regression of a bug on openai servers: openai-php/laravel#36 and #399

  5. ShantanuNair commented on Sep 11, 2023

    @ShantanuNair
    ShantanuNair
    on Sep 11, 2023

    @zzc0430 Did you fix this? I am running into the same issue on langchain using 3.5-turbo-16k where I have long input prompts of about 10-12k tokens, and see this error. I am not using streaming.

  6. zzc0430 commented on Sep 11, 2023

    @zzc0430
    zzc0430
    on Sep 11, 2023
    Author

    @zzc0430 Did you fix this? I am running into the same issue on langchain using 3.5-turbo-16k where I have long input prompts of about 10-12k tokens, and see this error. I am not using streaming.

    @ShantanuNair Not yet, it looks like a server-side issue.

  7. ShantanuNair commented on Sep 11, 2023

    @ShantanuNair
    ShantanuNair
    on Sep 11, 2023

    @zzc0430 What about this - aio-libs/aiohttp#4581 (comment) ?
    I have a feeling this is potentially not the issue, but the issue is actually from session setting in the openai client.

    I may have narrowed it down, but could use your help debugging - What version of openai do you have installed with langchain when you run into this issue? Try upgrading and see if it's fixed?

  8. zzc0430 commented on Sep 12, 2023

    @zzc0430
    zzc0430
    on Sep 12, 2023
    Author

    @zzc0430 What about this - aio-libs/aiohttp#4581 (comment) ? I have a feeling this is potentially not the issue, but the issue is actually from session setting in the openai client.

    I may have narrowed it down, but could use your help debugging - What version of openai do you have installed with langchain when you run into this issue? Try upgrading and see if it's fixed?

    @ShantanuNair The version I'm using is 0.27.2, and I will try to update to 0.28.0, but it seems that this bug cannot be consistently reproduced.

  9. ShantanuNair commented on Sep 12, 2023

    @ShantanuNair
    ShantanuNair
    on Sep 12, 2023

    @zzc0430 Let's do our best to find a root cause, if you do figure out a way to reproduce it let's work on it. I am trying to make sure I never see this again. Do you also see 502 errors Cloudflare bad_gateway much? My environment is modal.com Please let me know if this is at all similar for you, we may be able to narrow it down.

  10. zzc0430 commented on Sep 12, 2023

    @zzc0430
    zzc0430
    on Sep 12, 2023
    Author

    @zzc0430 Let's do our best to find a root cause, if you do figure out a way to reproduce it let's work on it. I am trying to make sure I never see this again. Do you also see 502 errors Cloudflare bad_gateway much? My environment is modal.com Please let me know if this is at all similar for you, we may be able to narrow it down.

    Similarly, I've also encountered 502 errors.

  11. ShantanuNair commented on Sep 13, 2023

    @ShantanuNair
    ShantanuNair
    on Sep 13, 2023

    @zzc0430 Appreciate it!! I might be able to narrow this down. Can you provide a few details so I can check what we have in common?

    • What environment are you running chains on? Local like macOS? Some compute provider like AWS Lambda/EC2 or Modal like in my case?
    • Do you have the default retries (max_retries default is 6) or retries set for your chain? In that case, does your retry complete after a 502? In my case all further retries for that chain fails until my compute hits its timeout.
    • Does the above mentioned retry run after the time it's supposed to? Mine says retry in 4/8s etc but ends up hanging for 6-7 minutes before running the retry! This could be related to your OP issue. I'm investigating potential causes in langchain lib and the openai-python client.

    Really appreciate if you could help me out with these details.

  12. ShantanuNair commented on Sep 25, 2023

    @ShantanuNair
    ShantanuNair
    on Sep 25, 2023

    Getting this on the latest openai-python version

  13. zzc0430 commented on Sep 25, 2023

    @zzc0430
    zzc0430
    on Sep 25, 2023
    Author

    Getting this on the latest openai-python version

    @ShantanuNair Thanks, which commit fixed this issue?

  14. ShantanuNair commented on Sep 25, 2023

    @ShantanuNair
    ShantanuNair
    on Sep 25, 2023

    @zzc0430 I meant to say it is still occurring on the latest version.

  15. rattrayalex commented on Oct 4, 2023

    @rattrayalex
    rattrayalex
    on Oct 4, 2023
    Contributor

    Can you try the beta of our upcoming v1.0.0 and let us know if the issue persists?

  16. rattrayalex commented on Nov 10, 2023

    @rattrayalex
    rattrayalex
    on Nov 10, 2023
    Contributor

    This should be fixed in v1.

  17. rattrayalex
    closed this as completedon Nov 10, 2023
  18. rattrayalex
    added
    fixed in v1Issues addressed by the v1 beta
    on Nov 10, 2023
  19. stainless-app
    added a commit that references this issue on Mar 27, 2025

    chore(ci): run release workflow once per day (#574)

    465236d
  20. safa0
    added a commit that references this issue on Apr 27, 2025

    Allow cancel out of the streaming result (openai#579)

    a113fea
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Azurefor issues relating to the Azure OpenAI servicebugSomething isn't workingfixed in v1Issues addressed by the v1 beta

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions