Authentication headers for HTTP Proxy are not sent proactively

[ptarjan]

Description

Context

In the https://claude.ai/code environment, the proxy doesn't work for dotnet restore. This is because dotnet restore expect the proxy to respond with an HTTP 407 before it sends the authentication headers, but this one does not.

Problem

When using an authenticated HTTP proxy configured via environment variables (HTTP_PROXY/HTTPS_PROXY with credentials in the URL), dotnet restore and other HttpClient-based operations fail with 401 Unauthorized.

Root cause: The current implementation waits for a 407 Proxy Authentication Required response before sending credentials. However, many proxies:

1. Return 401 Unauthorized instead of 407
2. Simply drop/timeout unauthenticated connections without any response
3. Close the connection after 407, making retry impossible for CONNECT tunnels

Reproduction:

export HTTPS_PROXY="http://user:password@proxy.example.com:8080"
dotnet restore  # Fails with 401 or timeout

Direct test showing the issue:

# Without Proxy-Authorization header:
CONNECT example.com:443 HTTP/1.1 → HTTP/1.1 401 Unauthorized
 
# With Proxy-Authorization header:
CONNECT example.com:443 HTTP/1.1
Proxy-Authorization: Basic dXNlcjpwYXNzd29yZA==
→ HTTP/1.1 200 OK

Solution

Modify AuthenticationHelper.SendWithProxyAuthAsync to proactively send Proxy-Authorization: Basic ... header on the first request when NetworkCredential is available, rather than waiting for a challenge.

This matches the behavior of curl, wget, and other HTTP clients that send proxy auth proactively.

Affected scenarios

• Corporate proxies requiring authentication
• Container environments with authenticated egress proxies
• CI/CD pipelines behind authenticated proxies

Reproduction Steps

1. Fork https://github.com/dotnet/runtime/
2. Open https://claude.ai/code
3. Choose "runtime" and type "run the tests" and choose "Full Internet" as the environment then press enter

Expected behavior

The tests run

Actual behavior

NuGet won't download

Regression?

No response

Known Workarounds

No response

Configuration

No response

Other information

No response

[dotnet-policy-service]

Tagging subscribers to this area: @karelz, @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

[wfurt]

For reference, there was some debate here: #100515
One can add the header easily for Basic auth, but I can see that is more difficult when using 3rd part tools where there is no control for the end user.

[dmitry-azaraev]

However, many proxies:

Is completely not true, most proxies doesnt do that, because it is just incorrect and must be considered as custom authentication scheme, which mimics to "Basic", but effectively is not.

401+Authentication header is dedicated to origin authentication, while 407+Proxy-Authentication dedicated to proxy authentication. They are must be separated because http proxies designed to work without tunnel in first place. Also http authentication by design is challenge based authentication, and user agent technically doesnt know how form auth header until challenged by server, while user agent doesnt know who asking credentials in proxy chain.

Popular browsers, for example, never send auth data proactively, more over in most cases is simply impossible, because they must ask credentials in dialog.

I'm complain only about formulation about most proxies.

In practice is not what bad to have "option", but you can implement this yourself: just connect to proxy as she want via sockets, establish tunnel, and integrate this to SocketsHttpClientHandler in ConnectCallback.

And also blame claude for using broken proxies.

[ManickaP]

For context: #123328

[ManickaP]

Should this rather be reported to claude.ai / anthropic? Seems like their proxy is misbehaving.
Do we know of any other instance of a proxy behaving this way?

[dmitry-azaraev]

Should this rather be reported to claude.ai / anthropic? Seems like their proxy is misbehaving. Do we know of any other instance of a proxy behaving this way?

They might have reasons to do this in this way too, and same situation eventually will happen with some other provider anyway.

In past (very long time ago) I'm used not small proxy provider, nowadays it changed his name, this doesn't matter. But this provider sometimes simply responds with invalid responses, valid headers, but short random binary in body which match to length, but doesn't represent requested resource. So practically I'm get 200 OK and random binary response. :) With our bills I'm expected what they will try to fix that, but it was in another reality, and more practical was just adapt and handle this by self (and eventually I'm learned how to identify this broken responses to filter/retry them).

I'm just want restate or clarify mine previous message, what, there is nothing wrong with implementing proactive authentication, as option. I'm not against it, it has own uses, for example, it shorts number of request/responses, which alone may significantly shorts time to initiate connection with distant proxy.

But this option completely eliminates challenge from equation, and by so, client might send authentication data to the host which which should not see this data (for example by DNS hijacking or so). [But Challenge+Basic offer no protection here.]

But at same time, this is also has no sense, because target servers usually doesn't require any challenge, as well there is no universal value of realm in challenge, no way to protect it, it can be simply some software default and even doesn't configured.

So, whenever client care only about (IsProxy, Challenger(Protocol, Host, Port), Scheme(Basic)) and ignores Realm (which almost always a case) - this is nearly semantic equivalent of short-circuiting and sending authentication data without being challenged.

There is also exist one interesting related but I guess less realistic edge case, is when proxy allows both: anonymous and authenticated access. and without way to force authentication it is not possible to reach protected area, because client never will be challenged. This is just as example, when fine tuning client's behavior is required in same way, as in whole discussion.

[ManickaP]

We shouldn't add functionality just because of a faulty proxy encountered out in the wild.

This should be reported to Claude AI.

We can continue the discussion here in the meantime.

[dotnet-policy-service]

This issue has been marked needs-author-action and may be missing some important information.