Tailscale sidecar: userspace SOCKS5 and localhost HA proxy (#74)

CHANGELOG.md

@@ -12,6 +12,11 @@ Until 1.0.0, releases stay on 0.x: `Added` bumps minor, `Fixed`/`Changed` (and t
 ### Added
 
 - Keep a Changelog, automatic `package.json` version bumps from Unreleased notes, and a CI gate that requires a changelog entry unless the pull request is labeled `skip-changelog`.
+- Optional Tailscale userspace sidecar and localhost HA proxy so Azure Pixel can reach Home Assistant on the mesh without a public HA URL (#74, #76).
+
+### Changed
+
+- When Tailscale is on, Azure Pixel uses `HOME_ASSISTANT_URL=http://127.0.0.1:8123` (no Nabu Casa fallback). Discord probes stay Discord-only. Enable only in gitignored tfvars after Key Vault `tailscale-auth-key`.
 
 ## [0.1.0] - 2026-10-09
 

docs/adr/0009-container-apps-dev.md

@@ -18,7 +18,7 @@ Ground truth is the architecture Deployment section and the comments on #9, not
 - **`admins.yaml`** is a read-only Key Vault secret, mounted as a file. Admin membership is never changed by a command.
 - **Secret values never go in Terraform state.** They may be passed as **ephemeral write-only apply inputs** (`write_secrets = true`, `value_wo` on `azurerm_key_vault_secret`) from gitignored `terraform.tfvars` / `TF_VAR_*`, or set with `az` / the portal. CI plans with `terraform.example.tfvars`, which leaves `write_secrets` off. The Container App references secrets by URI only.
 - **Azure Files SMB still needs the storage account key** to register the share on the Container Apps environment. Identity-based mounts are not supported there yet. That key is a platform credential and will appear in state. Pixel tokens and YAML do not. NFS was rejected because it needs a VNet and a premium 100 GiB share.
-- **Home Assistant:** a Tailscale sidecar on the replica is the preferred path (`tailscale_enabled`). It runs userspace (no TUN). `HOME_ASSISTANT_URL` + `HOME_ASSISTANT_TOKEN` stay as today, so a Nabu Casa URL works if Tailscale is not ready. Both URL and token or neither. Dev must not control the real doors.
+- **Home Assistant:** a Tailscale sidecar on the replica is the preferred path (`tailscale_enabled`). How that sidecar actually reaches HA (userspace SOCKS5 + `ha-proxy`, not MagicDNS in Pixel) is [ADR 0011](0011-tailscale-sidecar.md) / #74. A Nabu Casa URL still works when Tailscale is off. Both URL and token or neither. Dev must not control the real doors.
 - **Images:** `ghcr.io/pixelbar/pixel:<sha>` is #10. First apply may use a public GHCR tag. If the package is still private, the module accepts an optional registry (`container_registry_server` + username + Key Vault `ghcr-pull-token`). The image contains `content/` only.
 - **Logs:** Log Analytics, 30 days, until #7 decides retention for the privacy notice.
 - **CI:** `just tf-validate` format-checks and validates bootstrap and `envs/dev`. `terraform plan` on `envs/dev` uses GitHub Environment `dev` OIDC and **fails closed** if `ARM_*` is missing. No apply on merge.

docs/adr/0011-tailscale-sidecar.md

@@ -0,0 +1,28 @@
+# 0011. Tailscale sidecar: userspace SOCKS5 and localhost HA proxy
+
+* Status: accepted
+* Date: 2026-10-09
+* Issue: #74
+
+## Context
+
+ADR 0009 added an optional Tailscale sidecar (`tailscale_enabled`, userspace, no TUN) so Azure Pixel can reach Home Assistant without a public URL. The stub was not enough to turn on: Container Apps has no TUN, userspace does not install `100.x` routes, ACA injects `KUBERNETES_SERVICE_HOST` so official `containerboot` dies, and `--accept-routes` / `TS_ACCEPT_DNS` were the wrong defaults.
+
+Nabu Casa still works when the sidecar is off. Pixel’s HA client is unchanged: fail closed, not on `/healthz` / `/readyz`.
+
+## Decision
+
+- **One replica**, extra **containers** on it (`tailscale` + `ha-proxy`). Extra replicas would double-answer Discord and register a second tailnet node.
+- **Userspace** (`TS_USERSPACE=true`) plus **SOCKS5** on `127.0.0.1:1055`. No TUN, no VNet, no `--accept-routes`.
+- **`ha-proxy`** (pinned socat) listens on `127.0.0.1:8123` and SOCKS5s to `home_assistant_mesh_host:8123`. When Tailscale is on, Pixel’s `HOME_ASSISTANT_URL` is **`http://127.0.0.1:8123`**. Never a `*.ts.net` or `100.x` in the Pixel container (public DNS would blackhole via Azure egress).
+- **One URL.** No automatic fallback to Nabu Casa while the mesh is on (that would fail-open to the internet). Unset `home_assistant_url` when enabling Tailscale.
+- Clear **`KUBERNETES_SERVICE_HOST`** and **`TS_KUBE_SECRET`**. Advertise `tag:pixel-<env>`. Do not persist Tailscale state on Azure Files while #11 is open (ephemeral / OAuth-minted nodes).
+- **Do not ACA-probe** Tailscale `/healthz`. Pixel probes stay Discord-only. HA unavailability stays application fail-closed.
+- Enable only in **gitignored** `terraform.tfvars` after Key Vault `tailscale-auth-key` exists. Example tfvars stay `tailscale_enabled = false`.
+- Tailnet ACL: Pixel tags may open **tcp:8123** to the HA tag only. Dev must not be granted prod doors.
+
+## Consequences
+
+- Flipping `tailscale_enabled` in gitignored tfvars is the operator step; this module is safe to apply with it still false.
+- `ha-proxy` costs a full ACA consumption increment (0.25 vCPU / 0.5 Gi), same floor as Tailscale.
+- Application code does not change. Local/Mini Pixel still uses `.env` `HOME_ASSISTANT_URL` as today.

docs/architecture.md

@@ -188,7 +188,7 @@ type Feature = {
 
 Pixel can read and control a Home Assistant (HA) instance: lights, switches, doors. Unlike Discord this is not a place where commands come from. Pixel is the client, and features reach HA only through the core `Home` (`core/home.ts`), the same way they use the calendar. The adapter in `adapters/home-assistant/` is the **only** place that imports the official `home-assistant-js-websocket` library, and it plugs a backend into `Home` at startup.
 
-- **Connection:** `HOME_ASSISTANT_URL` and `HOME_ASSISTANT_TOKEN`, a long-lived token. Both unset means off, and only one set stops startup. It connects in the background and reconnects forever, so a slow or absent HA never holds up the bot, and it doesn't count towards `/healthz`: HA being down shouldn't restart Pixel. On Azure, prefer a **Tailscale sidecar** so the container reaches HA on the LAN (#9, #43). A **Nabu Casa** URL still works and is what local/dev use today.
+- **Connection:** `HOME_ASSISTANT_URL` and `HOME_ASSISTANT_TOKEN`, a long-lived token. Both unset means off, and only one set stops startup. It connects in the background and reconnects forever, so a slow or absent HA never holds up the bot, and it doesn't count towards `/healthz`: HA being down shouldn't restart Pixel. On Azure, prefer a **Tailscale sidecar** so the container reaches HA on the mesh (#74, [ADR 0011](adr/0011-tailscale-sidecar.md)): Pixel’s URL is `http://127.0.0.1:8123`. A **Nabu Casa** URL still works when the sidecar is off, and is what local use today.
 - **A non-admin token:** HA can't scope a token, so a long-lived token can do whatever its user can. Use a **non-admin** HA user: that blocks the admin-only commands. Pixel asks HA whose token it is (`auth/current_user`) and warns, in the logs and Sentry, if it belongs to an admin. `/admin status` shows the warning too. The real fence is Pixel's own allow-list of devices (below).
 - **Fails closed:** while HA isn't connected, or the token was refused, a call fails at once with a plain message. Nothing is queued, nothing is retried (a late unlock must never happen), and every call has a time limit. An outage is reported once, not on every command.
 - **No cache:** every read asks HA, so a change in HA shows on the next read.
@@ -401,7 +401,7 @@ Environment variables are validated by `config.ts` (zod). Nothing else reads `pr
 | `DISCORD_ANNOUNCE_TIMELINE_CHANNEL_ID` | | Optional. Timeline style: a new post for every open and close |
 | `DISCORD_ROLE_MEMBER`         |        | Optional. A Discord role name or ID that the `member` level is mirrored to. Unset means not mirrored |
 | `DISCORD_ROLE_FRIEND`         |        | Optional. A Discord role name or ID that the `friend` level is mirrored to. Unset means not mirrored |
-| `HOME_ASSISTANT_URL`          |        | Optional, with the token. Where Pixel reaches Home Assistant (http or https). On Azure prefer the Tailscale sidecar address; a Nabu Casa URL still works |
+| `HOME_ASSISTANT_URL`          |        | Optional, with the token. Where Pixel reaches Home Assistant (http or https). On Azure with Tailscale: `http://127.0.0.1:8123`. Off-mesh: a Nabu Casa URL |
 | `HOME_ASSISTANT_TOKEN`        |        | Optional, with the URL. A long-lived access token from a **non-admin** Home Assistant user. A secret |
 | `PIXEL_HOME_ASSISTANT_DIR`    |        | Default `config/home-assistant`. Holds `devices.yaml`, the allow-list of devices. Required when Home Assistant is set up |
 | `PIXEL_HOME_SYNC_MINUTES`     |        | Default `60`. How often the inventory (`inventory.yaml`, everything Home Assistant has: known, not usable) is refreshed. 0 means only at startup and on `/admin reload` |
@@ -435,8 +435,8 @@ There is **no database**. Access, schedules and (planned) account linking are fi
 - **Runtime files:** the Azure Files share is mounted at `/app/persist` as uid 1000 (`node`), covering `members.yaml`, `home-assistant/`, and `PIXEL_DATA_DIR` (`schedules.yaml` must persist; `home-switches.state` lost → doors start **off**; `space.state` and `announcements.state` only cost details). A container's own filesystem is thrown away on every deploy.
 - **Environments (temporary routing):** `dev` (Pixel Dev bot, test guild) is `infra/envs/dev`. `prod` (Pixel bot, Pixelbar guild) is #12. Separate bots, tokens and vaults. **`just deploy-dev`** (a local build) rolls Azure `dev` (one replica, then `just register`, then Sentry release). **Merges to `main`** publish GHCR only. CI does not deploy pull requests, and **does not deploy prod** until #12. This overrides the earlier `main` → `dev` plan until [ADR 0010](adr/0010-ghcr-cd.md) is reverted. Fail closed: never copy the Mini / Pixel Dev Discord token into prod.
 - **Terraform layout:** `infra/bootstrap` (state storage, GitHub OIDC, applied) then `infra/modules/pixel` and `infra/envs/dev` (#9). Secret values never go into Terraform state; apply can take them as ephemeral write-only inputs. Subscription `d150e252-e2f0-47fb-8a4a-c3f29e9aebd4`, West Europe. See [ADR 0008](adr/0008-terraform-bootstrap.md) and [ADR 0009](adr/0009-container-apps-dev.md). CD does **not** apply Terraform; it updates the Container App image.
-- **Home Assistant from Azure:** Tailscale sidecar into the space network is the preferred path (`tailscale_enabled`); `HOME_ASSISTANT_URL` can still be a Nabu Casa URL. Both URL and token or neither. Sidecar is userspace (no TUN). LAN MagicDNS through it is remaining work for #43.
-- **Cost (rough):** always-on `dev` is about €40–55/month West Europe without Tailscale, plus about €15 with the sidecar. Not a quote.
+- **Home Assistant from Azure:** Tailscale sidecar is the preferred path (`tailscale_enabled`, [ADR 0011](adr/0011-tailscale-sidecar.md) / #74): userspace SOCKS5 + `ha-proxy` on the same replica. When enabled, Pixel’s `HOME_ASSISTANT_URL` is `http://127.0.0.1:8123` (no Nabu Casa fallback). Off-mesh: a Nabu Casa URL still works. Both URL and token or neither. Do not ACA-probe Tailscale (`/readyz` is Discord-only). Enable only in gitignored tfvars after Key Vault `tailscale-auth-key` exists. Dev must not control the real doors.
+- **Cost (rough):** always-on `dev` is about €40–55/month West Europe without Tailscale, plus about €15 per extra 0.25 vCPU sidecar (Tailscale and `ha-proxy` each). Not a quote.
 - **CI (built):** [`.github/workflows/ci.yml`](../.github/workflows/ci.yml) runs on every PR and on pushes to `main`. It runs `just check` (lint, type-check, tests with coverage thresholds) and `just build`, uploads the coverage report, and checks that the Docker image builds. [`terraform.yml`](../.github/workflows/terraform.yml) runs `just tf-validate` (bootstrap and `envs/dev`) only when `infra/`, that workflow, or the `justfile` change, then `terraform plan` on `envs/dev` via GitHub Environment `dev` OIDC. Missing `ARM_*` **fails closed**. No apply on merge. Actions are pinned to commit SHAs.
 - **CD (built, temporary routing):** `just deploy-dev` for Azure `dev` (local GHCR push + `az` + `just register` + Sentry release). [`.github/workflows/cd.yml`](../.github/workflows/cd.yml) on `main` only: publish with `GITHUB_TOKEN` (`packages: write` on that job) and upload Sentry source maps (fails closed if org/token are missing). **No Azure login and no `pixel-prod` update** until #12. See [ADR 0010](adr/0010-ghcr-cd.md).
 
@@ -466,3 +466,4 @@ Record significant decisions as short ADRs in `docs/adr/NNNN-title.md`.
 | 0008 | Terraform bootstrap: remote state (Azure AD, no shared keys) and GitHub OIDC (UAMI per env) | accepted |
 | 0009 | Container Apps module + `dev`: one replica, required Azure Files volume, Key Vault, optional Tailscale sidecar | accepted |
 | 0010 | Temporary GHCR CD: local `just deploy-dev` → Azure `dev`; `main` publishes GHCR only (prod CD off until #12) | accepted |
+| 0011 | Tailscale sidecar: userspace SOCKS5 + localhost ha-proxy; Pixel URL is 127.0.0.1:8123 | accepted |

docs/runbook.md

@@ -161,7 +161,7 @@ Pixel talks to Home Assistant (HA) with a long-lived token from a **non-admin**
 
 **If Home Assistant is unreachable:** see [Incidents](#incidents).
 
-**How Pixel reaches HA:** locally, `HOME_ASSISTANT_URL` is usually the Nabu Casa cloud URL. On Azure the preferred path is a Tailscale sidecar so the container talks to HA on the LAN (#9, #43). A Nabu Casa URL still works if the sidecar is not there yet.
+**How Pixel reaches HA:** locally, `HOME_ASSISTANT_URL` is usually the Nabu Casa cloud URL. On Azure the preferred path is a Tailscale sidecar so the container talks to HA on the mesh (#74): Pixel uses `http://127.0.0.1:8123`. A Nabu Casa URL still works if the sidecar is off.
 
 ## Terraform bootstrap
 
@@ -193,9 +193,9 @@ az containerapp revision restart -g pixel-dev -n pixel-dev
 
 **See that there is exactly one replica:** `az containerapp replica list -g pixel-dev -n pixel-dev -o table`. If a command is answered twice, stop the extra instance (a laptop, or an old revision). Never raise `max_replicas`. Overlap during a deploy is #11.
 
-**Home Assistant:** Tailscale sidecar is preferred (`tailscale_enabled = true` plus Key Vault `tailscale-auth-key`). A Nabu Casa URL in `home_assistant_url` plus `home-assistant-token` still works when Tailscale is off. Set both URL and token, or neither. Dev must not point at the real doors.
+**Home Assistant:** Tailscale sidecar is preferred (`tailscale_enabled = true` in gitignored tfvars, plus Key Vault `tailscale-auth-key` and `home_assistant_mesh_host`). Pixel then uses `HOME_ASSISTANT_URL=http://127.0.0.1:8123`. A Nabu Casa URL in `home_assistant_url` plus `home-assistant-token` still works when Tailscale is off. Do not set both. `/readyz` stays Discord-only; HA down must not bounce the replica. Dev must not point at the real doors.
 
-**Rough cost:** about €40–55/month for always-on `dev` without Tailscale, plus about €15 with the sidecar. West Europe. Not a quote.
+**Rough cost:** about €40–55/month for always-on `dev` without Tailscale, plus about €15 per extra sidecar container (Tailscale and `ha-proxy`). West Europe. Not a quote.
 
 🚧 **Prod** is #12. `just deploy-dev` already publishes GHCR, rolls `pixel-dev`, registers commands, and creates a Sentry release when org/token are set. `main` publishes GHCR (and source maps) only. No Terraform apply on merge.
 
@@ -235,6 +235,8 @@ Every secret lives in the secret store for its environment (a local `.env` for d
 2. Put the new one in the secret store as `HOME_ASSISTANT_TOKEN`, and restart Pixel.
 3. `/admin status` should say Connected with no warning about an admin token. If it warns that the token belongs to an admin, the token is from the wrong user: redo it with the non-admin user.
 
+**Rotate the Tailscale auth key:** Tailscale admin → revoke the old OAuth client or tagged key, mint a new one (tagged `tag:pixel-dev` / `tag:pixel-prod`, ephemeral). Put it in Key Vault `tailscale-auth-key` and restart the Container App. HA commands fail closed until the sidecar rejoins; Discord stays up (`/readyz` is not Tailscale).
+
 **Rotate the Sentry DSN:** Sentry project → Settings → Client Keys: create a new key, disable the old one, update `SENTRY_DSN`, restart. Check that a test event arrives.
 
 **If a secret leaked:** rotate it first, then work out where it leaked from. Check the logs and Sentry for the time window. A leaked Discord token lets someone act as the bot in your server, so treat it as urgent.

infra/README.md

@@ -7,7 +7,7 @@ Terraform for Pixel on Azure. There is **no database**. Secret *values* never go
 | Path | What | Issue |
 | --- | --- | --- |
 | [`bootstrap/`](bootstrap/) | Remote state storage and GitHub → Azure OIDC. Apply this by hand, once. | #8 |
-| [`modules/pixel/`](modules/pixel/) | Container App, Azure Files volume, Key Vault, optional Tailscale sidecar. | #9 |
+| [`modules/pixel/`](modules/pixel/) | Container App, Azure Files volume, Key Vault, optional Tailscale sidecar + ha-proxy. | #9, #74 |
 | [`envs/dev/`](envs/dev/) | Dev root: Pixel Dev bot, test guild, into `pixel-dev`. | #9 |
 | `envs/prod` | Same module, Pixelbar guild. Not in this PR. | #12 |