Client Throws 'Error while copying content to a stream' on 401 API Response

When making a call to SendGridClient.SendEmailAsync using a deliberately invalid API Key, rather than receiving a Response with a StatusCode of 401 Unauthorized, an exception is thrown:

System.Net.Http.HttpRequestException occurred
  HResult=0x80131620
  Message=Error while copying content to a stream.
  Source=<Cannot evaluate the exception source>
  StackTrace:
   at System.Net.Http.HttpContent.<LoadIntoBufferAsyncCore>d__49.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Net.Http.HttpClient.<FinishSendAsyncBuffered>d__58.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SendGrid.SendGridClient.<MakeRequest>d__23.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SendGrid.SendGridClient.<RequestAsync>d__24.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SendGrid.SendGridClient.<SendEmailAsync>d__25.MoveNext()

Technical details:

sendgrid-csharp Version: 9.9.0
Azure Function running Net462

Looking at the data received from SendGrid, this may or may not be due to the fact that the Content-Length header is > 0 but there is no response content:

Upon further investigation it appears that this is somehow linked to the original content-length of the POST request.

If the original content is small (i.e. less in the email body) then I get the correct response content:

{
    "errors": [
        {
            "message": "The provided authorization grant is invalid, expired, or revoked",
            "field": null,
            "help": null
        }
    ]
}

@oatsoda

Hi @oatsoda,

That's an interesting bug, thanks for sharing it with us!

I've added this to our backlog for further investigation.

With Best Regards,

Elmer

@thinkingserious

@thinkingserious Thanks. I have also opened a support ticket with SendGrid directly regarding the lack of content returned as it may be an API issue rather than a client issue. I will update here once I receive a response.

Source.zip
SendGridTest.zip

Attached are Source and Binaries of a test app demonstrating the failure. I really suspect this is an API issue rather than a SendGridClient issue, but updating it here anyway.

Run the app with a value to repeat the lines in the email body. Once you exceed 56 it gets the exception. Tested on multiple machines and also from an Azure VM.

@thinkingserious

@thinkingserious SendGrid Support have sent me your way :)

It looks like you've been working with one of our Engineers via the github repository. He is definitely your best resource for this particular issue. This bug falls a bit outside of the scope of Support, but regardless of whether it's client related or API related, the engineer you were in contact with via GitHub should be able to assist.

Any ideas on what the issue could be? Can you reproduce it using my test app?

@oatsoda

Hello @oatsoda,

Thank you for the updated information. This issue is still working its way up our backlog.

In the mean time, you may want to try the API call directly to officially rule out the client as the culprit.

@thinkingserious

Hi @thinkingserious

Yes, I can reproduce it directly to the API in Postman too. However, I sent SendGrid Support the sample Postman call and it didn't reproduce for them! I suspect this is API related rather than client related. The API just doesn't return any content if the original request is above a certain size. I'd be really interested to know if the sample app reproduces it for you - I have tried on multiple machines in multiple geographic locations and it reproduces it each time. Quite why it didn't work via Postman for SendGrid Support I don' know - perhaps a network issue which isn't occurring inside the SendGrid network? (I vaguely remember coming across something similar years ago regarding MTU sizes on network devices and responses being split over multiple packets being lost - but I think that is clutching at straws!)

@oatsoda

I can confirm some of the observations that @oatsoda made regarding this issue. Please note that I used StrongGrid instead of SendGrid's C# client in order help determine if this issue is specific to the client or not:

Observation 1:
Using the same email content included in @oatsoda's sample code (including the large text and HTML content), I observed that the response from SendGrid's API is HTTP/1.1 401 Unauthorized (this is expected since the sample code is intentionally using a bogus api key). I also observe that the content of the response is empty. Finally, I observe that the response contains the following header: Content-Length: 116 which is contradictory with the fact that the response content is empty. This points to a server-side issue.

Observation 2:
Using the email content included in @oatsoda's sample code with much smaller text and HTML content, I observed that the response from SendGrid's API is HTTP/1.1 401 Unauthorized and the content of the response is {"errors":[{"message":"The provided authorization grant is invalid, expired, or revoked","field":null,"help":null}]} which is 116 characters long and is consistent with the Content-Length: 116 header that I also observe in this scenario. There is no issue here, everything is working as expected.

Observation 3:
I am unable to reproduce the Error while copying content to a stream exception that @oatsoda experienced when the response is empty. Since the only difference between my setup and his is that I am using an alternate C# client, this leads me to conclude that this issue is specific to SendGrid's C# client.

Conclusion:
There seems to be two distinct issues:

The server is not returning the expected content with the response in the scenario described in observation 1 (combined with the fact that it returns incorrect Content-Length)
The client is not properly handling situations where the body of the response is empty and/or relying on the incorrect Content-Length to determine if the body of the response contains any content.

@Jericho

Thanks @Jericho for confirming!

Strange that the error doesn't occur in other clients. The stack trace indicates that it is an error in HttpClient.FinishSendAsyncBuffered - but I suppose it could be that other clients may not use this...

@thinkingserious

@thinkingserious Any confirmation of an API problem regarding the empty response? Causing me quite a lot of grief as authentication failures not being detected.

I suspect the secondary issue regarding the client side exception is actually just net core http client expecting responses to be well formed - hence relying on content-length.

@oatsoda

@oatsoda,

I can check on that for you. What's the support ticket number?

I also experienced the "Error while copying content to a stream." exception on incorrect key. Not a big issue to handle it but the exception is not really helpful. Still the same bug after what looks like about 32 months - not good.

Similar to sendgrid/sendgrid-java#259 and sendgrid/sendgrid-java#641

Closing this as a non-library issue since it cannot/should not be remedied in client-side code. I've opened an internal ticket for tracking (reference ID: CL-2667). Please reference that ticket when contacting support regarding this issue.

Note that support and backend product teams do not use GitHub to prioritize work. Contacting support is the best way to push this issue along.

@childish-sambino

@childish-sambino

There was already an support request for this.
If you see this post, there is technically a weakness in the library too.

@oatsoda

@oatsoda This is something the needs to be handled via support so I recommend opening a new ticket there. The engineers that manage these libraries do not handle backend issues and support is the path forward.

For the item related to this library handling invalid API responses, I'm not convinced this library should be doing something different. It would basically be just failing in a different way. If the backend issue is resolved and a proper response is returned then there would be no special error handling here.

Oh, I see but disagree. Getting a response of 'Error while copying content to a stream' when the failure is an invalid API – well this response does not help any developer. From: childish-sambino [mailto:notifications@github.com] Sent: 02 September 2020 23:19 To: sendgrid/sendgrid-csharp <sendgrid-csharp@noreply.github.com> Cc: AndreduToit <dutoita1011@gmail.com>; Comment <comment@noreply.github.com> Subject: Re: [sendgrid/sendgrid-csharp] Client Throws 'Error while copying content to a stream' on 401 API Response (#648) @oatsoda <https://github.com/oatsoda> This is something the needs to be handled via support so I recommend opening a new ticket there. The engineers that manage these libraries do not handle backend issues and support is the path forward. For the item related to this library handling invalid API responses, I'm not convinced this library should be doing something different. It would basically be just failing in a different way. If the backend issue is resolved and a proper response is returned then there would be no special error handling here. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#648 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHOBDK2OV3IM34ZCGYSMOFDSDZO5RANCNFSM4EJCAKXQ> .

Oh, I see but respectfully disagree. Getting a response of 'Error while copying content to a stream' when the failure is an invalid API – well this response does not help any developer. The one thing that the library may do differently is to provide a response which would aid sendgrid developer customers to track down their mistakes more efficiently. Maybe something along the lines of "Error - API key ......"?

We use the standard HttpClient which is where the exception emanates from. Adding logic/handling for issues where the response is not valid (e.g., content length header does not match actual content length) is better suited there. I suspect the reason this was not seen in the other repo is because of the
Pathoschild/FluentHttpClient client wrapper.

Still, given that this is an API-level bug that affects multiple libraries/languages, I don't see creating workarounds in multiple places a viable solution for handling API bugs regarding large payloads and bad API keys.

I can't believe this is still an issue after 3 years, and there's no clear documentation on how to fix this one.
We are a paid customer, and the PO has escalated to suggest switching the integration.
Also if you can't support/maintain your API, please do not open one. just waste people's time and money.

@oatsoda

HI we have submitted a support ticket and they asked us to open a github issue here.
Would you please unify your response?

@oatsoda This is something the needs to be handled via support so I recommend opening a new ticket there. The engineers that manage these libraries do not handle backend issues and support is the path forward.

For the item related to this library handling invalid API responses, I'm not convinced this library should be doing something different. It would basically be just failing in a different way. If the backend issue is resolved and a proper response is returned then there would be no special error handling here.

Still getting this problem in 2021. I don't understand why the client cannot correctly process the result and expose the real error rather than just throwing an obscure exception. I can see in Postman that a perfectly valid 401 Unauthorised response comes back (due to non whitelisted IP), except instead of just setting IsSuccessStatusCode to false it throws an unhelpful exception.

Navigation Menu

Repository navigation

Client Throws 'Error while copying content to a stream' on 401 API Response #648

Technical details:

53 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Client Throws 'Error while copying content to a stream' on 401 API Response #648

Description

Technical details:

Activity

oatsoda commented on Dec 20, 2017

thinkingserious commented on Dec 20, 2017

oatsoda commented on Dec 21, 2017

oatsoda commented on Dec 22, 2017

oatsoda commented on Jan 2, 2018

thinkingserious commented on Jan 2, 2018

oatsoda commented on Jan 3, 2018

Jericho commented on Jan 3, 2018

oatsoda commented on Jan 4, 2018

oatsoda commented on Feb 11, 2018

thinkingserious commented on Feb 14, 2018

53 remaining items

AndreduToit commented on Aug 28, 2020

childish-sambino commented on Aug 28, 2020

oatsoda commented on Sep 2, 2020

childish-sambino commented on Sep 2, 2020

AndreduToit commented on Sep 2, 2020

AndreduToit commented on Sep 3, 2020

childish-sambino commented on Sep 4, 2020

joanna1010 commented on Nov 18, 2020

joanna1010 commented on Nov 26, 2020

nmg196 commented on Jan 20, 2021

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions