Gateway egress allowlist: hostname entries block all TLS egress, including the allowlisted hosts #345

Summary

A Gateway whose egress.allowlist.hosts entries are hostnames blocks all TLS egress from the task, including to the hosts it explicitly allows. An allowlisted host and a non-allowlisted host fail identically, so from inside the sandbox there is no way to tell a working allowlist from a broken one.

CIDR entries work correctly. "*" works correctly. Only the hostname form — which is what docs/concepts.md describes ("an egress allowlist of hosts and ports") — is affected.

Tested on AX v0.3.0 (d8ed0fe), Agent Substrate on a 2-node kind cluster, atelet v0.2.0-beta4.

Reproduction

Three gateways, same task image, same cluster, same minute. The probe is curl https://<host>/ from inside the task via ax ssh.

1. Wildcard — works

egress:
  allowlist:
    hosts:
      - host: "*"

litellm  HTTP=200
litellm  HTTP=200
litellm  HTTP=200

2. Hostnames — blocks everything, including the allowlisted hosts

egress:
  allowlist:
    hosts:
      - host: "litellm-agentic.apps.example.com"
        port: 443
      - host: "git.example.com"
        port: 443

litellm-agentic.apps.example.com   curl: (35) TLS connect error: error:0A000126:SSL routines::unexpected eof while reading
git.example.com                    curl: (35) TLS connect error: ... unexpected eof while reading   <- ALSO allowlisted
example.com                        curl: (35) TLS connect error: ... unexpected eof while reading   <- correctly denied

All three identical, consistently ~1.03s.

3. CIDRs — correct

egress:
  allowlist:
    hosts:
      - host: "172.16.1.153/32"   # litellm
      - host: "46.17.95.101/32"   # git

litellm  HTTP=200
git      HTTP=200
example  curl: (35) Recv failure: Connection reset by peer    <- denied, and distinguishable

The remote is healthy throughout: from the cluster host, outside the sandbox, the same URL returns HTTP=200 in ~22ms, 3/3.

Where it breaks

Not DNS and not TCP. Inside the task, under the hostname gateway:

getent hosts litellm-agentic.apps.example.com   ->  172.16.1.153      (resolves)
</dev/tcp/172.16.1.153/443                      ->  TCP-OK            (connects)
openssl s_client -connect ...:443               ->  "SSL handshake has read 0 bytes and written 1588 bytes"

The ClientHello goes out and nothing comes back.

Logs

atenet-egress, envoy container — note the SNI is parsed correctly, but there is no upstream and the flag is UH (no healthy upstream host):

{"actor":".../actor/killtest2","flags":"UH","sni":"litellm-agentic.apps.example.com",
 "destination":null,"upstream":null,"bytes_rcvd":0,"bytes_sent":0,"duration_ms":0}

Against the wildcard gateway, same host, same minute:

{"actor":".../actor/openprobe","flags":"-","sni":"litellm-agentic.apps.example.com",
 "destination":"172.16.1.153:443","upstream":"172.16.1.153:443","bytes_rcvd":1829,"bytes_sent":5611,"duration_ms":36}

atenet-egress, ext-proc container:

{"level":"INFO","msg":"egress tunnel opened: no address rule allows the destination,
  requests inside it are decided one by one","destination":"172.16.1.153:443","rule":-1}

Reading of the cause

AX's own translation looks correct. internal/substrate/client.go, ApplyEgressPolicy routes any host containing / into a CIDRRule and everything else into HostnameRule.Patterns:

if strings.Contains(h.Host, "/") {
    cidrs = append(cidrs, h.Host)
} else if h.Host != "" {
    patterns = append(patterns, h.Host)
}

So a hostname produces a HostnameRule and no address rule. Substrate then opens a tunnel intending to adjudicate requests inside it one at a time — which works for plaintext HTTP, where there are requests to inspect — but a TLS passthrough connection has none, so envoy never gets an upstream cluster and closes with UH. The SNI is matched and then not used to admit the connection.

If that reading is right, the fix is either for the hostname rule to resolve to an address rule for TLS, or for the documentation to say that hostname allowlisting does not cover TLS and CIDRs are required.

Second, smaller issue: port is silently ignored

HostRule.Port is accepted by the API and never read by ApplyEgressPolicy — neither HostnameRule nor CIDRRule carries it. A Gateway declaring port: 443 constrains nothing, and reads as though it does.

Impact

The failure is silent and closed. Everything reports healthy — the Task is Running, all three conditions are True, GatewayReady says "Network policies active" — while no egress works. The first guess from inside is that the remote is down, because that is exactly what it looks like. This cost us about an hour before the envoy UH flag gave it away.

Happy to test a patch on this cluster.