リリース: フィードバックのキューに dead letter queue を設定する

README.md

@@ -20,7 +20,7 @@ single Worker.
 - **Cloudflare Workers** — `fetch` / `queue` / `scheduled` handlers
 - **Workers KV** — TTS cache metadata, config, and review read-state
 - **R2** — audio binaries and feedback images
-- **Cloudflare Queues** — `feedback-triage`
+- **Cloudflare Queues** — `feedback-triage` (+ `feedback-triage-dlq` as its dead letter queue)
 - **Workers AI** — feedback triage
 - **Google Cloud Text-to-Speech** — TTS synthesis (`Standard` voices, service-account auth)
 - **OpenAI** — the conversational agent
@@ -57,8 +57,14 @@ wrangler r2 bucket create trainlcd-tts-dev
 wrangler r2 bucket create trainlcd-uploads-dev
 # Queues
 wrangler queues create feedback-triage-dev
+wrangler queues create feedback-triage-dev-dlq   # dead letter queue (no consumer)
 ```
 
+The names above are the dev ones. Production uses the same set without the
+`-dev` suffix (`trainlcd-tts`, `trainlcd-uploads`, `feedback-triage`,
+`feedback-triage-dlq`); create those too if you are setting up prod from
+scratch. Both environments' resources already exist on the TrainLCD account.
+
 ### Setting secrets
 
 ```bash
@@ -264,6 +270,89 @@ completes in 5–17 s at 26–62 neurons per feedback, versus ~1 s and ~4.7 neur
 for the old 8B model. Queue consumers use `max_batch_size: 5`, so a batch stays
 well inside the invocation limit.
 
+### Dead letter queue
+
+`processFeedbackMessage()` rethrows on failure so the consumer can `retry()`,
+but retrying does not help when the cause is permanent — a credential that no
+longer grants access, a repo that was renamed. Without a dead letter queue the
+message is simply dropped once `max_retries: 3` is exhausted, and the feedback
+is lost for good.
+
+The consumers therefore declare `dead_letter_queue` (`feedback-triage-dlq`, and
+`feedback-triage-dev-dlq` for dev). The DLQ intentionally has **no consumer** —
+running the same handler against it would fail for the same reason.
+
+Do not look for the wiring on the DLQ itself. Moving a message into a DLQ is
+something Cloudflare does internally, not something the Worker sends, so a
+correctly configured DLQ still reports zero producers and zero consumers in
+`wrangler queues list`. The link lives on the **source** queue's consumer, and it
+only exists once the config has been deployed:
+
+```bash
+wrangler queues consumer list feedback-triage-dev   # or feedback-triage for prod
+# dead_letter_queue must name the DLQ; "-" means this environment is still
+# dropping messages once max_retries is exhausted.
+```
+
+**A DLQ is not archival storage.** Messages sitting in it expire on the queue's
+retention period, which both DLQs inherit from the account default (4 days on a
+paid plan, and not extendable beyond 24 h on the free plan). That is the replay
+deadline: once it passes the feedback is gone just as surely as it was before
+this queue existed. Check and extend it if an incident may outlast it:
+
+```bash
+# 1209600 = 14 days, the paid-plan maximum. The free tier is capped at 86400
+# (24 h) and rejects anything above it, so use that value instead on free.
+# Run this for the dev DLQ too — retention is per queue, and
+# feedback-triage-dev-dlq inherits nothing from the prod one.
+RETENTION=1209600
+for q in feedback-triage-dlq feedback-triage-dev-dlq; do
+  wrangler queues update "$q" --message-retention-period-secs "$RETENTION"
+done
+```
+
+`wrangler queues info` does not print the retention period (as of wrangler 4.103),
+so there is no CLI read-back for it — set it explicitly, or check the dashboard.
+
+To recover, fix the root cause first, then replay by temporarily attaching a
+consumer to the DLQ. Give that consumer its own dead letter queue — a replay
+consumer runs the same handler, so anything still failing would hit `max_retries`
+and be deleted outright, which is the exact loss this section exists to prevent.
+
+```bash
+# production. For dev: DLQ=feedback-triage-dev-dlq, SCRIPT=trainlcd-worker-dev
+DLQ=feedback-triage-dlq
+SCRIPT=trainlcd-worker
+RETENTION=1209600   # 86400 on the free tier, as above
+
+wrangler queues info "$DLQ"    # backlog size, current consumers
+
+# Catches whatever still fails on replay. Give it the same retention as the DLQ,
+# otherwise it silently falls back to the account default.
+wrangler queues create "$DLQ-quarantine" --message-retention-period-secs "$RETENTION"
+
+# --batch-size 5 matches the max_batch_size the regular consumer runs with; the
+# default of 10 is a lot for one invocation at 5–17 s of inference per message.
+wrangler queues consumer add "$DLQ" "$SCRIPT" \
+  --dead-letter-queue "$DLQ-quarantine" --batch-size 5
+
+# ...wait for the backlog to drain, then detach:
+wrangler queues consumer remove "$DLQ" "$SCRIPT"
+```
+
+The replay consumer is deliberately not declared in `wrangler.jsonc` — it exists
+only for the duration of an incident, so nothing will remove it for you. Leaving
+it attached means every later failure gets reprocessed by it instead of landing
+in the DLQ where you can see it.
+
+`$DLQ-quarantine` outlives the incident as well. `queues create` fails if it
+already exists, so on the next incident either reuse it (confirm it is empty
+first — anything left in it is unprocessed feedback) or `wrangler queues delete`
+it once you have dealt with whatever landed there.
+
+Note that DLQ messages carry the full feedback payload, so the DLQ is subject to
+the same handling rules as the private `TrainLCD/Issues` repo.
+
 ## Public repo routing
 
 Feedback Issues are always created in the private `TrainLCD/Issues` repo with

wrangler.jsonc

@@ -26,12 +26,22 @@
     { "binding": "UPLOAD_BUCKET", "bucket_name": "trainlcd-uploads-dev" }
   ],
 
+  // dead_letter_queue: max_retries を使い切ったメッセージの退避先。未設定だと
+  // GitHub 起票が恒久的に失敗する状況（PAT 失効など）でフィードバックが消えるため必須。
+  // DLQ 側に consumer は付けない（同じ処理を回すと同じ理由で落ちるだけ）。
+  // DLQ にも保持期限があり、期限を過ぎたメッセージは消える。復旧手順と
+  // 保持期限の延ばし方は README の「Dead letter queue」節を参照。
   "queues": {
     "producers": [
       { "binding": "FEEDBACK_QUEUE", "queue": "feedback-triage-dev" }
     ],
     "consumers": [
-      { "queue": "feedback-triage-dev", "max_batch_size": 5, "max_retries": 3 }
+      {
+        "queue": "feedback-triage-dev",
+        "max_batch_size": 5,
+        "max_retries": 3,
+        "dead_letter_queue": "feedback-triage-dev-dlq"
+      }
     ]
   },
 
@@ -109,7 +119,12 @@
           { "binding": "FEEDBACK_QUEUE", "queue": "feedback-triage" }
         ],
         "consumers": [
-          { "queue": "feedback-triage", "max_batch_size": 5, "max_retries": 3 }
+          {
+            "queue": "feedback-triage",
+            "max_batch_size": 5,
+            "max_retries": 3,
+            "dead_letter_queue": "feedback-triage-dlq"
+          }
         ]
       },
       "services": [{ "binding": "STATION_API", "service": "stationapi" }],