Skip to content

Support SETTINGS_MAX_HEADER_LIST_SIZE on HTTP/2 and HTTP/3 #78193

Closed

Description

@MihaZupan

HTTP/2 and HTTP/3 specs define an advisory SETTINGS_MAX_HEADER_LIST_SIZE setting:

This advisory setting informs a peer of the maximum field section size that the sender is prepared to accept, in units of octets. The value is based on the uncompressed size of field lines, including the length of the name and value in units of octets plus an overhead of 32 octets for each field line.

An endpoint can use the SETTINGS_MAX_HEADER_LIST_SIZE to advise peers of limits that might apply on the size of uncompressed field blocks. This setting is only advisory, so endpoints MAY choose to send field blocks that exceed this limit and risk the request or response being treated as malformed

So while HttpClient isn't violating the spec by not enforcing this limit, we could play nicer here.

The fact that we're not enforcing the limit leads to situations that can be very difficult to debug. The server is free to close the connection in situations where it receives too many header bytes. On HttpClient's side, we're then forced to fail all the requests that are being multiplexed over that connection and report a non-helpful exception, while logging on the server may not even indicate that any such offending request was received.

System.Net.Http.HttpRequestException: An error occurred while sending the request.
System.IO.IOException: The request was aborted.
System.IO.IOException: The response ended prematurely while waiting for the next frame from the server.
    at System.Net.Http.Http2Connection.<ReadFrameAsync>g__ThrowMissingFrame|57_1()   
    at System.Net.Http.Http2Connection.ReadFrameAsync(Boolean initialFrame)  
    at System.Net.Http.Http2Connection.ProcessIncomingFramesAsync()   
    --- End of inner exception stack trace ---
    at System.Net.Http.Http2Connection.ThrowRequestAborted(Exception innerException)
    at System.Net.Http.Http2Connection.Http2Stream.CheckResponseBodyState() 
    at System.Net.Http.Http2Connection.Http2Stream.TryEnsureHeaders()
    at System.Net.Http.Http2Connection.Http2Stream.ReadResponseHeadersAsync(CancellationToken cancellationToken)

I believe that HttpClient should enforce this setting if the server announced it. We should do so by failing the request before we send any data from that request on the shared connection.

Activity

  1. added this to the 8.0.0 milestone on Nov 10, 2022
  2. MihaZupan commented on Nov 10, 2022

    @MihaZupan
    MemberAuthor

    Triage: This is a problem that is difficult to diagnose and trace back to the headers, and we've already seen users hitting it (e.g. dotnet/yarp#1875). Moving to 8.0.

  3. self-assigned this
    on Nov 29, 2022
  4. ghost added
    in-prThere is an active PR which will close this issue when it is merged
    on Dec 6, 2022
  5. ghost removed
    in-prThere is an active PR which will close this issue when it is merged
    on Dec 27, 2022
  6. MihaZupan commented on Dec 27, 2022

    @MihaZupan
    MemberAuthor

    Reopening for 6.0 and 7.0 servicing

  7. added
    tenet-reliabilityReliability/stability related issue (stress, load problems, etc.)
    on Jan 3, 2023
  8. ghost added
    in-prThere is an active PR which will close this issue when it is merged
    on Jan 4, 2023
  9. modified the milestones: 8.0.0, 6.0.x on Jan 4, 2023
  10. ghost removed
    in-prThere is an active PR which will close this issue when it is merged
    on Jan 5, 2023
  11. karelz commented on Jan 6, 2023

    @karelz
    Member

    Fixed in main (8.0) in PR #79281 and in 7.0.3 in PR #79994 and in 6.0.14 in PR #79997 (for HTTP/2 only).

  12. ghost locked as resolved and limited conversation to collaborators on Feb 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions