[oxoxDev]

Summary

• Stops the cron agent_job retry loop from retrying permanent billing user-states and reporting them to Sentry on every exhausted attempt — two sibling Sentry floods:
  • TAURI-RUST-514 — BYO provider 402 ("requires more credits", OpenRouter out of balance).
  • TAURI-RUST-BMW — managed-backend 400 USER_INSUFFICIENT_CREDITS ("Insufficient budget").
• Adds the is_insufficient_credits_event arm to the desktop before_send chain (app/src-tauri/src/lib.rs) — fix(inference): cap extraction max_tokens + classify provider 402 to stop the retry/Sentry storm (#3616) #3617 wired it into the standalone-CLI chain (src/main.rs) only.
• Extracts a shared is_insufficient_credits_message matcher (one source of truth with the before_send filter); reuses the existing provider::is_budget_exhausted_message for the budget case.

Problem

Both issues are the same leak path: every event is tagged domain=cron, operation=agent_job, failure=retries_exhausted. execute_job_with_retry reports the propagated billing error with a bare, unconditional report_error. Session-expired is halted-on-first right above it; the two billing states were not — so a permanent 402/400 is retried N times and reported each exhaustion.

The respective before_send filters do not catch the cron re-report:

• 514: is_insufficient_credits_event was wired into src/main.rs only; the shipped desktop core (in-process since fix(core,cef): run core in-process and stop orphaning CEF helpers on Cmd+Q #1061) uses app/src-tauri/src/lib.rs's chain, which lacked it.
• BMW: is_budget_event is tag-gated (failure=non_2xx + status=400) — tags the cron re-report (failure=retries_exhausted, no status) does not carry.

514 is also the live residual of #3617 (which capped the extraction path; these are the full-window cron agent requests #3617 did not cover).

Solution

• observability.rs — extract is_insufficient_credits_message(&str); is_insufficient_credits_event delegates to it. The matcher checks the credit phrase against the response body only (strips the formatted "): " status prefix) so the 402 reason phrase "Payment Required" can't masquerade as a body credit-signal (Codex P2).
• cron/scheduler.rs — add is_insufficient_credits_failure (402) and is_budget_exhausted_failure (400), both Agent-only, mirroring is_session_expired_failure. Break on the first occurrence; extend the post-loop guard to skip the report on either billing halt. Metadata-only debug breadcrumbs on the halt + suppression branches (CodeRabbit). Fixes the leak and the retry amplification at source, independent of the tag-gated before_send filters.
• app/src-tauri/lib.rs — add the is_insufficient_credits_event arm + metadata-only log to the desktop before_send, closing the fix(inference): cap extraction max_tokens + classify provider 402 to stop the retry/Sentry storm (#3616) #3617 drift.

Bucket (RCA-vs-suppress): unpreventable — a BYO account out of credit / a managed account out of budget has no local lever to fund it, so demotion is the complete answer for the billing error itself; the real lever is on the amplification (don't retry a permanent 402/400), which the cron halt fixes. No suppression on a data path. Matchers anchored on status + phrase, guarded by verbatim-body tests so wording drift fails CI.

Follow-up (out of scope): the two before_send chains (src/main.rs and app/src-tauri/src/lib.rs) can silently diverge — worth unifying into one shared builder later.

Submission Checklist

• Tests added or updated (happy path + at least one failure / edge case) per Testing Strategy
• Diff coverage ≥ 80% — Coverage Gate (diff-cover) green on this PR.
• N/A: behaviour-only change (Sentry noise routing) — no feature rows added/removed/renamed in docs/TEST-COVERAGE-MATRIX.md
• N/A: no coverage-matrix feature IDs touched by this change
• No new external network dependencies introduced (mock backend used per Testing Strategy)
• N/A: no release-cut surface touched (provider-error Sentry routing only)
• Linked issue closed via Closes #NNN in the ## Related section

Impact

• Desktop (in-process core): cron agent jobs that hit a permanent BYO 402 or managed-backend budget 400 now stop after the first attempt instead of looping the backoff, and the event is demoted from Sentry on the shipped build. Recoverable failures unchanged — they retry and report failure=retries_exhausted as before.
• No schema, migration, or API change. No new dependencies.

Related

• Closes: Cron agent-job insufficient-credits 402 floods Sentry (residual after #3617) (TAURI-RUST-514) #3912
• Closes: Cron agent-job budget-exhausted 400 floods Sentry (TAURI-RUST-BMW) #3929
• Sentry-Issue: TAURI-RUST-514
• Sentry-Issue: TAURI-RUST-BMW
• Follow-up PR(s)/TODOs: unify the src/main.rs and app/src-tauri/src/lib.rs before_send chains into one shared builder.

---

AI Authored PR Metadata (required for Codex/Linear PRs)

Linear Issue

• Key: N/A
• URL: N/A

Commit & Branch

• Branch: fix/514-cron-insufficient-credits
• Commit SHA: d8300ebb46e0e373e06337a1320bde5842f12ecc

Validation Run

• N/A: pnpm --filter openhuman-app format:check — no frontend/TS files changed
• N/A: pnpm typecheck — no TS files changed
• Focused tests: cargo test --lib -- is_insufficient_credits is_budget_exhausted_failure → 11 passed
• Rust fmt/check (if changed): cargo fmt + GGML_NATIVE=OFF cargo check (root) clean
• Tauri fmt/check (if changed): GGML_NATIVE=OFF pnpm rust:check (app/src-tauri) clean

Validation Blocked

• command: N/A
• error: N/A
• impact: N/A

Behavior Changes

• Intended behavior change: permanent BYO-provider 402 / managed-backend budget 400 in a cron agent job halts after one attempt and is kept out of Sentry on the desktop build.
• User-visible effect: none (background cron + observability routing only); recoverable failures unchanged.

Parity Contract

• Legacy behavior preserved: non-billing failures still retry and report failure=retries_exhausted; session-expired halt untouched; event-filter behavior unchanged except the Codex P2 body-scoping (stricter, not looser).
• Guard/fallback/dispatch parity checks: matchers require status anchor + genuine phrase; Agent-only scope (shell jobs keep retry semantics); both before_send chains now carry the credits arm.

Duplicate / Superseded PR Handling

• Duplicate PR(s): none (checked open + 30d-merged; fix(observability): drop cron agent-job session-expired events in before_send (TAURI-RUST-M) #3736 handles the sibling cron session-expired leak, disjoint region)
• Canonical PR: this
• Resolution: N/A

[coderabbitai]

📝 Walkthrough

Walkthrough

Adds a shared is_insufficient_credits_message predicate in observability.rs anchored to a 402 status shape and a provider credit phrase. The cron scheduler uses it to halt agent retries immediately on 402 insufficient credits or 400 budget exhausted, suppressing the residual report_error call. The Tauri shell before_send filter gains the matching event-drop arm that was previously missing from the desktop process.

Changes

Insufficient Credits (402/400) Cron Halt and Sentry Suppression

Layer / File(s)	Summary
is_insufficient_credits_message classifier and event filter refactor src/core/observability.rs	Adds public is_insufficient_credits_message predicate requiring both a 402 status shape and the provider credit-indicator phrase; refactors is_insufficient_credits_event to delegate to it for event.message and all event.exception value paths; extends unit tests for match, rejection, and delegation parity.
Scheduler 402/400 halt-on-first and suppress report_error src/openhuman/cron/scheduler.rs, src/openhuman/cron/scheduler_tests.rs	Adds is_insufficient_credits_failure (402, agent-only, checks last_agent_error then last_output) and is_budget_exhausted_failure (400, agent-only, same fallback); introduces credits_exhausted and budget_exhausted flags in execute_job_with_retry; breaks the retry loop early on either condition; gates the final report_error behind !credits_exhausted && !budget_exhausted. Tests cover primary match, last_output fallback, negative guards (canned messages, 500/400 unrelated shapes), and scope restriction to JobType::Agent.
Tauri shell before_send filter wiring app/src-tauri/src/lib.rs	Adds the missing is_insufficient_credits_event arm to the desktop process before_send filter, logging by event id and returning None to drop the event.

Sequence Diagram

sequenceDiagram
    participant Agent as Agent Job
    participant Scheduler as Cron Scheduler
    participant Classifiers as Billing Classifiers
    participant ErrorLog as Error Reporting
    participant Sentry as Sentry Client

    Agent->>Scheduler: execute_job_with_retry()
    Scheduler->>Agent: attempt retry
    Agent-->>Scheduler: fail with 402/400 error
    Scheduler->>Classifiers: is_insufficient_credits_failure()?
    Classifiers-->>Scheduler: true (or is_budget_exhausted)
    Scheduler->>Scheduler: set credits_exhausted=true
    Scheduler->>Scheduler: break retry loop
    Scheduler->>ErrorLog: skip report_error (billing halt)
    ErrorLog->>Sentry: log suppression breadcrumb (debug)
    Sentry-->>ErrorLog: acknowledged

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Suggested labels

agent

Suggested reviewers

• graycyrus

🐇 A 402 knocked, then a 400 came to stay,
Retries just multiplied grief day by day.
Now credits_exhausted and budgets must align,
The scheduler halts first, the before_send draws the line.
No flood in Sentry—just logs when they pay. 🌿

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	All code changes directly address the requirements in #3912: adding insufficient-credits detection to observability, halting cron retries on billing errors, and extending the before_send filter in desktop client.
Out of Scope Changes check	✅ Passed	All changes are directly related to addressing billing-state Sentry floods: observability matcher refactoring, cron scheduler retry logic, before_send filter alignment, and comprehensive test coverage.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title directly matches the PR's primary objective: stopping Sentry floods from billing-state errors (402 insufficient credits, 400 budget exhaustion) by preventing cron retries and filtering events.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2c43bd40eb

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Check the 402 body before suppressing cron failures

When this new message-level classifier is used by is_insufficient_credits_failure, any formatted provider error like provider API error (402 Payment Required): some unrelated condition is treated as insufficient credits: body_indicates_insufficient_credits also matches the reason phrase payment required, which is part of the status prefix rather than the response body. In that scenario the cron loop now breaks after one attempt and skips the retries_exhausted Sentry report even though the body did not contain a balance/credits signal, contradicting the intended 402+credit-phrase guard.

Useful? React with 👍 / 👎.

[oxoxDev]

Addressed in 5e4eb50 — is_insufficient_credits_message now slices off the formatted "): " status prefix and runs body_indicates_insufficient_credits against the body only, so the prefix's own "payment required" can't self-match. Added a regression assertion: (402 Payment Required): some unrelated condition → false; (402 Payment Required): … insufficient balance → true.

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/openhuman/cron/scheduler.rs`:
- Around line 408-421: Add debug logging to the insufficient_credits_failure
branch within the scheduler to aid production diagnostics. When the
is_insufficient_credits_failure check evaluates to true and credits_exhausted is
set, add a trace or debug log statement that includes stable grep-friendly
prefixes and correlation fields such as job_id, attempt number, and remaining
retries. Ensure the log only contains metadata (no raw provider response body)
and use consistent field names for correlation. Apply the same logging pattern
to any other related terminal branches or report-suppression paths around line
430 to maintain consistency across all new failure-handling flows.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 02590ba6-8ea5-4a6d-aa6a-3a5449c146f9

📥 Commits

Reviewing files that changed from the base of the PR and between d4db739 and 2c43bd4.

📒 Files selected for processing (4)

• app/src-tauri/src/lib.rs
• src/core/observability.rs
• src/openhuman/cron/scheduler.rs
• src/openhuman/cron/scheduler_tests.rs