Feature request: opt-out for the mandatory token when binding non-loopback (Kubernetes / trusted-network deployments) #2023

First off, thanks for the security work in 0.9.0. The secure-by-default direction
is the right call for anyone exposing the server straight to the internet. But the
guard leaves Kubernetes deployments in a genuinely awkward spot, and I wanted to
lay out the situation and ask for a small escape hatch.

After upgrading, the container refuses to boot:

CRITICAL:server:Refusing to start: binding 0.0.0.0 with no CRAWL4AI_API_TOKEN
and jwt_enabled=false would expose an unauthenticated API. Set
CRAWL4AI_API_TOKEN=$(openssl rand -hex 32), enable jwt, or bind loopback.

The suggested "bind to loopback" fallback is a non-starter on Kubernetes. A process
listening on 127.0.0.1 is only reachable inside its own pod's network namespace, so
no other pod and no Service can route to it. Binding to 0.0.0.0 / the pod IP is the
only way to be reachable inside a cluster at all. And the loopback path still mints
an ephemeral token and keeps the auth gate in front of every route, so it doesn't
even yield an unauthenticated service.

That leaves setting a token. I can do that, and I can thread it through my callers.
But in this kind of deployment it buys nothing: the server already lives in a
namespace with default-deny network policies, only a handful of explicitly allowed
workloads can reach it, and nothing outside the cluster can route to it at all. The
trust boundary is enforced at the network layer, which is exactly the model the
platform is built around. Forcing an in-app bearer token on top of that is pure
friction for zero additional security in this topology, and it has to be wired
through every caller and proxy by hand.

What I'd love is a simple, explicit opt-out, something like
security.allow_insecure_bind: true in config.yml or a CRAWL4AI_ALLOW_INSECURE=1
env var, that says "yes, I know what I'm doing, I'm running behind my own trust
boundary" and lets the server bind non-loopback without a token. Keep it off by
default so the secure-by-default promise still holds; it would just be opt-in for
those of us running inside a controlled network.

For now I've had to pin back to the last pre-0.9.0 image to keep things working,
which isn't where I want to stay. A flag like this would let me move onto current
releases cleanly.

Is there an officially supported way to do this that I've missed? If not, would you
be open to adding an opt-out? Happy to send a PR; looks like the check sits right
in _resolve_auth.