common: honor HTTP_PROXY/HTTPS_PROXY env vars in http client by texasich · Pull Request #21752 · ggml-org/llama.cpp

texasich · 2026-04-11T01:06:06Z

What

common_http_client() builds the httplib client but never checks standard proxy environment variables, so any download that goes through this path (e.g. -hf flag, model presets) silently fails in environments that require a proxy.

How

After constructing the client, read HTTPS_PROXY/https_proxy for https:// URLs and HTTP_PROXY/http_proxy for http:// ones (scheme-specific takes precedence). If set and parseable, pass it to cli.set_proxy() and optionally cli.set_proxy_basic_auth(). Malformed proxy URLs are caught and silently ignored so nothing breaks for callers without a proxy configured.

Reuses the existing common_http_parse_url() to handle auth credentials embedded in the proxy URL (e.g. http://user:pass@proxy:3128).

Notes

NO_PROXY/no_proxy bypass list not implemented yet; can be added as follow-up
ALL_PROXY not checked; HTTPS_PROXY and HTTP_PROXY cover the common case
std::getenv is available via <cstdlib> already pulled in by httplib.h

ggml-gh-bot · 2026-04-11T01:10:07Z

Hi @texasich, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

Multiple open PRs from a new contributor: We limit new contributors (those without a previously merged PR) to 1 open PR at a time. You currently have 2 open PRs.

Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.

texasich · 2026-04-13T18:33:50Z

closing this to get back under the 1-PR limit for new contributors — #21630 is closer to merging so keeping that one active. will reopen this once that lands. branch is preserved on my fork.

texasich · 2026-04-14T18:42:53Z

reopening — #21630 just merged so the new-contributor limit no longer applies. ready for review whenever the maintainers have a minute.

MoRanYue · 2026-05-18T09:16:20Z

Why is this still unmerged?

texasich · 2026-05-22T01:22:18Z

Gentle nudge on this one — been open a while. Happy to rebase if there are conflicts, or make any changes needed. Let me know if anything's blocking review. 🙏

BoneGoat · 2026-05-26T11:12:34Z

Please consider this PR.

nicriverhoo · 2026-05-27T12:33:19Z

+1. Please consider this PR.

MoRanYue · 2026-05-27T14:49:17Z

For a modern software, not respecting to proxy is pretty bad.

ngxson · 2026-05-28T11:06:26Z

I'd assume most maintainers don't use proxy so no one can confidently review / test this PR (me including)

Asking @ServeurpersoCom for your thoughts about this feature / this implementation

ServeurpersoCom · 2026-05-28T11:32:41Z

Need to methodically test the HTTP client on Linux and on Windows with a small Squid as a classic enterprise forward proxy... I'm setting up the platform and will keep you updated.

ServeurpersoCom · 2026-05-28T13:29:29Z

Tested this and the proxy path works fine for -hf, both direct and through the proxy, Linux only. Windows check in progress.

A few non-blocking notes:

NO_PROXY/no_proxy isn't honored. The client also fetches arbitrary --model-url, docker manifests and runtime image_url, but those are all external targets so it's not urgent, possible follow-up.
A malformed proxy url is caught silently, so it fails open and falls back to a direct connection with no log. A LOG_WRN in the catch would make that debuggable.
nit: for http you read uppercase HTTP_PROXY first. Out of habit a lot of code only trusts lowercase http_proxy for the http scheme and ignores the uppercase form, since httpoxy (see: https://httpoxy.org). It doesn't apply here since this isn't CGI, so it's purely cosmetic.
nit: the comment mentions checking the scheme var then a generic fallback, but both vars are scheme specific and there's no generic one, so it doesn't match the code.

ServeurpersoCom · 2026-05-28T13:56:57Z

Windows OK, quick test with llama-installer.sh Wine + BorringSSL (default) with Zig. If OK, LGTM.

ServeurpersoCom

Clarify or drop that comment, otherwise LGTM

texasich · 2026-05-28T14:46:15Z

Dropped the misleading fallback comment, the code only has the two scheme-specific vars, no generic one. Good catch.

ServeurpersoCom · 2026-05-28T15:10:37Z

It's a minimalist proxy support that will unlock GGUF downloads for a lot of users, and we can easily do a follow-up PR later if we need to expand it. You could add the env var to the docs, but it's a super standard one for every web app on Linux.

namniav · 2026-06-11T06:18:58Z

Thank you for the patch and it works for me.

May I ask if there is any issue that caused this PR to be closed without being merged?

ServeurpersoCom · 2026-06-11T06:28:24Z

There's quite a bit of work pending on the proxy; I'll try to make progress this weekend.

bsproger · 2026-06-18T15:41:13Z

+1. Please consider this PR.

+1 from me too

marecabo · 2026-09-05T11:34:47Z

@texasich Thanks for this feature. I addressed @ServeurpersoCom's review comments (no_proxy & logging) in texasich#1.
Note, that since 2026-07-08 #24948 no_proxy handling is required to ensure CLI loopback is not sent to proxy.

On top of that, I also merged master back in proxy-env-vars-merged (cc576d1) (just one small readme conflict), tested it on RHEL 9 with ZScaler and can confirm it works, see detailed logs in the PR.

ServeurpersoCom · 2026-09-05T11:44:38Z

Needs a rebase, there's a README conflict with master. I already tested the proxy and network side, Linux and Windows, and it behaves as expected, so once that's sorted the only thing missing is a second approval.

ngxson · 2026-09-05T12:08:35Z

+                cli.set_proxy_basic_auth(proxy.user, proxy.password);
+            }
+        } catch (...) {
+            // ignore malformed proxy URL — don't break existing callers


per agents.md: no em-dash

@texasich is fixed in https://github.com/texasich/llama.cpp/pull/1/changes#diff-30865ebe44d6724ca6dc550d4eefe93d3ca32abf04dbd8e3869fdfd472c4b632L115

marecabo · 2026-10-14T11:49:41Z

@ServeurpersoCom
@texasich rebased the branch and it does not conflict now anymore. If there is anything left to do, please let me know.

ServeurpersoCom · 2026-10-14T15:09:27Z

@ServeurpersoCom @texasich rebased the branch and it does not conflict now anymore. If there is anything left to do, please let me know.

Thanks @marecabo, the NO_PROXY handling and the warning look good.

Tested on Linux behind Squid: with HTTP_PROXY set and no NO_PROXY, llama cli now hangs forever on "Loading model...". It talks to its own server on 127.0.0.1 through common_http_client, so the /health polling goes to the proxy and gets a 403 in a loop. Same with lowercase http_proxy, and setting NO_PROXY=localhost,127.0.0.1 makes it work. Master works fine in the same setup, so this is a regression for exactly the users this PR targets.

We're not sure yet which way to fix it is best, there are a few options:

always bypass the proxy for loopback hosts (localhost, 127.0.0.0/8, ::1) in common_http_client, like Go's net/http does, though curl and Python don't, so it's not a universal convention
apply the proxy only on the download paths (download.cpp, hf-cache.cpp), leaving every other common_http_client caller direct as on master, but then --server-base to a remote server never goes through the proxy
keep the env handling as is and have llama cli reach its spawned server with a direct client, since it knows that address is local, which keeps --server-base proxy aware

texasich · 2026-10-14T18:08:00Z

Good catch, thanks for the detailed report. Pushed a fix that makes common_http_client bypass the proxy unconditionally for loopback destinations (localhost, *.localhost, 127.0.0.0/8, ::1, 0.0.0.0): a remote proxy can never reach the machine's own loopback, so the 403 loop was unavoidable there. Same approach Go net/http uses. NO_PROXY handling and proxy support for downloads and remote --server-base endpoints stay unchanged. @marecabo GitHub shows the branch merging clean again, so nothing else needed on the rebase.

ngxson · 2026-10-14T22:42:15Z

+> [!TIP]
+> If you're behind a corporate proxy, set the `HTTP_PROXY` and `HTTPS_PROXY` environment variables before running commands that download models (e.g. `export HTTPS_PROXY=http://proxy.example.com:8080`). Hosts listed in `NO_PROXY` and loopback addresses are contacted directly.
+


this is not a good place to put this, move it to server docs

ngxson

can merge once you resolve my remarks about docs

common_http_client() was ignoring standard proxy env vars, causing downloads via -hf and similar flags to fail silently in environments behind an HTTP proxy (corporate networks, air-gapped infra, etc.). Checks HTTPS_PROXY/https_proxy for https:// URLs and HTTP_PROXY/http_proxy for http:// URLs, in that order. If a proxy URL is set and parseable, passes it to httplib's set_proxy() and set_proxy_basic_auth() as appropriate. Malformed proxy URLs are silently ignored so existing callers aren't affected. Fixes ggml-org#21694

@ServeurpersoCom

The comment referenced a 'generic one' that doesn't exist in the code — both HTTP_PROXY and HTTPS_PROXY are scheme-specific. Reviewer noted the mismatch. Addresses review feedback from @ServeurpersoCom.

llama cli talks to its spawned server on 127.0.0.1 through common_http_client, so with HTTP_PROXY set and no NO_PROXY the health polling went to the corporate proxy, which rejects loopback targets, and the wait loop never exits ("Loading model..." hangs forever). A remote proxy can never reach the client machine's own loopback, so these destinations are now always contacted directly, before NO_PROXY is consulted: localhost, *.localhost, 127.0.0.0/8, ::1 and 0.0.0.0. Similar to Go's net/http. Proxy handling for downloads and remote --server-base endpoints is unchanged. Signed-off-by: texasich <101962694+texasich@users.noreply.github.com>

per review: the top level README stays lean; the proxy howto now lives with the server docs under Usage Signed-off-by: texasich <101962694+texasich@users.noreply.github.com>

texasich · 2026-10-16T05:12:00Z

Moved. The tip is out of the README and now lives in tools/server/README.md under Usage (Running behind a proxy). Branch rebased onto current master. @ngxson

@ServeurpersoCom

…g#21752) * common: honor HTTP_PROXY/HTTPS_PROXY env vars in http client common_http_client() was ignoring standard proxy env vars, causing downloads via -hf and similar flags to fail silently in environments behind an HTTP proxy (corporate networks, air-gapped infra, etc.). Checks HTTPS_PROXY/https_proxy for https:// URLs and HTTP_PROXY/http_proxy for http:// URLs, in that order. If a proxy URL is set and parseable, passes it to httplib's set_proxy() and set_proxy_basic_auth() as appropriate. Malformed proxy URLs are silently ignored so existing callers aren't affected. Fixes ggml-org#21694 * common: drop misleading comment about generic proxy fallback The comment referenced a 'generic one' that doesn't exist in the code — both HTTP_PROXY and HTTPS_PROXY are scheme-specific. Reviewer noted the mismatch. Addresses review feedback from @ServeurpersoCom. * common : warn when the proxy URL from the environment is malformed * common : honor NO_PROXY when selecting the http proxy * ci : retrigger checks (test-thread-safety flake) * common : bypass the proxy for loopback destinations llama cli talks to its spawned server on 127.0.0.1 through common_http_client, so with HTTP_PROXY set and no NO_PROXY the health polling went to the corporate proxy, which rejects loopback targets, and the wait loop never exits ("Loading model..." hangs forever). A remote proxy can never reach the client machine's own loopback, so these destinations are now always contacted directly, before NO_PROXY is consulted: localhost, *.localhost, 127.0.0.0/8, ::1 and 0.0.0.0. Similar to Go's net/http. Proxy handling for downloads and remote --server-base endpoints is unchanged. Signed-off-by: texasich <101962694+texasich@users.noreply.github.com> * docs: move proxy tip from README to server docs per review: the top level README stays lean; the proxy howto now lives with the server docs under Usage Signed-off-by: texasich <101962694+texasich@users.noreply.github.com> --------- Signed-off-by: texasich <101962694+texasich@users.noreply.github.com> Co-authored-by: texasich <texasich@users.noreply.github.com> Co-authored-by: texasich <texasich@gmail.com> Co-authored-by: marecabo <23156476+marecabo@users.noreply.github.com>

@ServeurpersoCom

…g#21752) * common: honor HTTP_PROXY/HTTPS_PROXY env vars in http client common_http_client() was ignoring standard proxy env vars, causing downloads via -hf and similar flags to fail silently in environments behind an HTTP proxy (corporate networks, air-gapped infra, etc.). Checks HTTPS_PROXY/https_proxy for https:// URLs and HTTP_PROXY/http_proxy for http:// URLs, in that order. If a proxy URL is set and parseable, passes it to httplib's set_proxy() and set_proxy_basic_auth() as appropriate. Malformed proxy URLs are silently ignored so existing callers aren't affected. Fixes ggml-org#21694 * common: drop misleading comment about generic proxy fallback The comment referenced a 'generic one' that doesn't exist in the code — both HTTP_PROXY and HTTPS_PROXY are scheme-specific. Reviewer noted the mismatch. Addresses review feedback from @ServeurpersoCom. * common : warn when the proxy URL from the environment is malformed * common : honor NO_PROXY when selecting the http proxy * ci : retrigger checks (test-thread-safety flake) * common : bypass the proxy for loopback destinations llama cli talks to its spawned server on 127.0.0.1 through common_http_client, so with HTTP_PROXY set and no NO_PROXY the health polling went to the corporate proxy, which rejects loopback targets, and the wait loop never exits ("Loading model..." hangs forever). A remote proxy can never reach the client machine's own loopback, so these destinations are now always contacted directly, before NO_PROXY is consulted: localhost, *.localhost, 127.0.0.0/8, ::1 and 0.0.0.0. Similar to Go's net/http. Proxy handling for downloads and remote --server-base endpoints is unchanged. Signed-off-by: texasich <101962694+texasich@users.noreply.github.com> * docs: move proxy tip from README to server docs per review: the top level README stays lean; the proxy howto now lives with the server docs under Usage Signed-off-by: texasich <101962694+texasich@users.noreply.github.com> --------- Signed-off-by: texasich <101962694+texasich@users.noreply.github.com> Co-authored-by: texasich <texasich@users.noreply.github.com> Co-authored-by: texasich <texasich@gmail.com> Co-authored-by: marecabo <23156476+marecabo@users.noreply.github.com>

texasich requested a review from a team as a code owner April 11, 2026 01:06

texasich closed this Apr 13, 2026

texasich reopened this Apr 14, 2026

ServeurpersoCom approved these changes May 28, 2026

View reviewed changes

Comment thread common/http.h Outdated

texasich requested a review from ggerganov as a code owner May 28, 2026 17:08

texasich closed this May 30, 2026

ServeurpersoCom requested a review from ngxson June 11, 2026 06:26

ServeurpersoCom reopened this Jun 11, 2026

marecabo mentioned this pull request Sep 5, 2026

common : honor no_proxy and warn on malformed URLs texasich/llama.cpp#1

Merged

ServeurpersoCom approved these changes Sep 5, 2026

View reviewed changes

ngxson reviewed Sep 5, 2026

View reviewed changes

github-actions Bot added the documentation Improvements or additions to documentation label Sep 5, 2026

ngxson reviewed Oct 14, 2026

View reviewed changes

ngxson approved these changes Oct 14, 2026

View reviewed changes

texasich and others added 7 commits October 15, 2026 23:50

common: drop misleading comment about generic proxy fallback

2fb10a7

The comment referenced a 'generic one' that doesn't exist in the code — both HTTP_PROXY and HTTPS_PROXY are scheme-specific. Reviewer noted the mismatch. Addresses review feedback from @ServeurpersoCom.

common : warn when the proxy URL from the environment is malformed

cec2b03

common : honor NO_PROXY when selecting the http proxy

8062be6

ci : retrigger checks (test-thread-safety flake)

2cf9816

docs: move proxy tip from README to server docs

3ba20f8

per review: the top level README stays lean; the proxy howto now lives with the server docs under Usage Signed-off-by: texasich <101962694+texasich@users.noreply.github.com>

texasich force-pushed the fix/proxy-env-vars-http-client branch from 55a122e to 3ba20f8 Compare October 16, 2026 05:11

texasich requested a review from a team as a code owner October 16, 2026 05:11

github-actions Bot added the server label Oct 16, 2026

ngxson approved these changes Oct 16, 2026

View reviewed changes

ngxson merged commit 7182ef7 into ggml-org:master Oct 16, 2026
9 of 12 checks passed

ggerganov mentioned this pull request Oct 28, 2026

llama.cpp : bump version to 0.8.0 #31309

Merged

		> [!TIP]
		> If you're behind a corporate proxy, set the `HTTP_PROXY` and `HTTPS_PROXY` environment variables before running commands that download models (e.g. `export HTTPS_PROXY=http://proxy.example.com:8080`). Hosts listed in `NO_PROXY` and loopback addresses are contacted directly.

Conversation

texasich commented Apr 11, 2026

What

How

Notes

Uh oh!

ggml-gh-bot Bot commented Apr 11, 2026

Uh oh!

texasich commented Apr 13, 2026

Uh oh!

texasich commented Apr 14, 2026

Uh oh!

MoRanYue commented May 18, 2026

Uh oh!

texasich commented May 22, 2026

Uh oh!

BoneGoat commented May 26, 2026

Uh oh!

nicriverhoo commented May 27, 2026

Uh oh!

MoRanYue commented May 27, 2026

Uh oh!

ngxson commented May 28, 2026

Uh oh!

ServeurpersoCom commented May 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ServeurpersoCom commented May 28, 2026

Uh oh!

ServeurpersoCom commented May 28, 2026

Uh oh!

ServeurpersoCom left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

texasich commented May 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ServeurpersoCom commented May 28, 2026

Uh oh!

namniav commented Jun 11, 2026

Uh oh!

ServeurpersoCom commented Jun 11, 2026

Uh oh!

bsproger commented Jun 18, 2026

Uh oh!

marecabo commented Sep 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ServeurpersoCom commented Sep 5, 2026

Uh oh!

ngxson Sep 5, 2026

Choose a reason for hiding this comment

Uh oh!

marecabo Sep 5, 2026

Choose a reason for hiding this comment

Uh oh!

marecabo commented Oct 14, 2026

Uh oh!

ServeurpersoCom commented Oct 14, 2026

Uh oh!

texasich commented Oct 14, 2026

Uh oh!

ngxson Oct 14, 2026

Choose a reason for hiding this comment

Uh oh!

ngxson left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

texasich commented Oct 16, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

ServeurpersoCom commented May 28, 2026 •

edited

Loading

texasich commented May 28, 2026 •

edited

Loading

marecabo commented Sep 5, 2026 •

edited

Loading

ngxson left a comment •

edited

Loading