Skip to content

fix(skills): fail fast and name the reason when a skill fetch is refused - #6508

Merged
senamakel merged 1 commit into
tinyhumansai:mainfrom
M3gA-Mind:fix/skills-install-fail-fast
Sep 23, 2026
Merged

senamakel merged 1 commit into
tinyhumansai:mainfrom
M3gA-Mind:fix/skills-install-fail-fast

Conversation

@M3gA-Mind

@M3gA-Mind M3gA-Mind commented Sep 23, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Problem

Every non-2xx produced the same string:

format!("fetch failed: {fetch_url} returned status {}", status.as_u16())

So "this host is rate limiting you, wait a moment" and "this host is not answering" reached the user identically, and Retry-After was never read at all — rg "429|Retry-After|retry_after" across skills/ops_install/ and skills/catalog/ returned nothing.

Meanwhile the catalogue install path passed timeout_secs: Some(60) while download.rs bounds its own probes to the same hosts at PROBE_TIMEOUT_SECS = 15.

One correction to the issue's reproduction, from the code: a 429 is a response and returns immediately. The 60-second failure in QA's log was therefore a transport timeout, not the 429 path — they observed the 429 separately via curl. Both are real; they are different failures. The timeout cut addresses the logged symptom, the Retry-After work addresses the throttling.

Solution

Kind, not just status. A 429 — or a 5xx carrying Retry-After — now reports as rate limiting with the host's own delay when it sends a parseable one. RATE_LIMITED_ERROR_PREFIX is exported so a caller can branch on the kind rather than parsing a status code back out of prose.

Header read before the body. response.bytes() takes response by value, so a later header read is not merely wrong — it does not compile. The same ordering silently drops the header on the model-call path (#6413, found independently by another worker within the hour). The comment notes that parallel rather than inventing a shared helper for two call sites in different crates and different loops.

Retry-After's HTTP-date form is deliberately not parsed. It would pull in a date parser to serve a form GitHub's raw CDN does not send, and the caller degrades correctly without it — the error still says the host is throttling, just without a specific delay.

Timeout scoped to the catalogue path only. CATALOG_INSTALL_TIMEOUT_SECS = 15 matches download::PROBE_TIMEOUT_SECS, with a comment saying why it matches so the next person tuning one knows the other exists. DEFAULT_INSTALL_TIMEOUT_SECS stays at 60 — that is the documented default for the public skills.install_from_url tool, and a scripted fetch is not a user waiting on a button.

Tests

Three cases, asserting the kind of failure rather than that install fails — the latter passes on the code this replaces:

case asserts
429 with Retry-After: 42 reports rate limiting and carries 42s
429 without Retry-After still reports rate limiting, does not fall through
404 is not reported as rate limiting, keeps the status wording

The 404 case exists because the rate-limit branch could otherwise widen to every non-2xx and silently lose the distinction again.

Revert-proved. Removing the branch fails both 429 cases with the pre-fix string verbatim, while the 404 case keeps passing:

a_throttled_host_reports_rate_limiting_with_its_retry_after       ... FAILED
  a throttled host must be reported as rate limiting,
  got: fetch failed: http://127.0.0.1:62946/SKILL.md returned status 429
a_throttled_host_without_retry_after_still_reports_rate_limiting ... FAILED
a_missing_skill_is_not_reported_as_rate_limiting                 ... ok

They live in a new ops_install_failure_tests.rs because ops_create_and_url_tests.rs hit the 750-line layout cap at 777. Extracted rather than trimmed, per the gate's own guidance, and both files land with headroom (663 / 124) rather than at the limit.

Scope — #6409 stays open

acceptance criterion status
1. Built-in skills bundled or proxied; no dependency on raw.githubusercontent.com at install not addressed → #6505
2. Install button shows installing / installed / failed states; failure shows the reason already worked before this PR — see below
3. 429 / 5xx surfaces quickly with the reason, honouring Retry-After addressed
4. Timeout reduced, distinguishes rate-limited from unreachable addressed

On criterion 2: SkillsExplorerTab.handleRegistryInstall already set an installing state and toasted the error with its reason, wired via pages/Skills.tsx, shipped in #3457 on 2026-06-06. The issue's "no UI feedback" line describes the 60-second wait before the failure surfaced, not missing UI. Corrected on the issue so nobody rewrites working code.

Criterion 1 is not a repair: the skill catalogue is remotely aggregated (skills/catalog/ops.rs:4), so source: "built-in" means built in to HermesHub, not ships with OpenHuman. Nothing about that path was ever local. Bundling or proxying is new capability needing decisions on which skills ship, how they update, and the licensing position on redistributing third-party content — filed as #6505 with those questions as its first acceptance criterion.

Impact

  • Rust core, skills install path. No frontend, no schema, no migration, no new dependency.
  • A user on a throttled route sees an actionable error in ~15s instead of an opaque one at 60s.
  • No behaviour change for a healthy fetch or for skills.install_from_url callers.

Related

Submission Checklist

  • Tests added or updated (happy path + at least one failure / edge case) — three failure-mode cases; revert-proof output above.
  • Diff coverage ≥ 80% — every added branch is exercised: the Some/None Retry-After arms and the non-rate-limited fallback each have a dedicated test. Full-suite diff-cover is CI's authority.
  • Coverage matrix updated — N/A: behaviour-only fix, no feature row added, removed or renamed.
  • All affected feature IDs from the matrix are listed under ## Related — N/A: no matrix row covers the skill install fetch.
  • No new external network dependencies introduced — N/A: this reduces reliance on one. Tests use a local wiremock host.
  • Manual smoke checklist updated if this touches release-cut surfaces — N/A: docs/RELEASE-MANUAL-SMOKE.md has no skill-install step.
  • Linked issue closed via Closes #NNN — deliberately not used; Refs #6409 only, because criterion 1 remains open. See Scope.

AI Authored PR Metadata

Linear Issue

  • Key: N/A
  • URL: N/A

Commit & Branch

  • Branch: fix/skills-install-fail-fast
  • Commit SHA: 59ab60073f46c5c7de0186d41271b51e04052b82

Validation Run

  • pnpm --filter openhuman-app format:check — N/A: no frontend files changed.
  • pnpm typecheck — N/A: no TypeScript changed.
  • Focused tests: the three cases above, 3 passed / 0 failed, plus the revert-proof.
  • Rust fmt/check (if changed): cargo fmt -p openhuman --check clean; layout gate passed; both CI clippy invocations (--features "$(product-features.sh)" and contributor default) clean with -D warnings.
  • Tauri fmt/check (if changed): N/A — crates/openhuman-app untouched.

Validation Blocked

Behavior Changes

  • Intended behavior change: a throttled skill host produces a distinct, actionable error, and the catalogue install gives up after 15s rather than 60s.
  • User-visible effect: Install failures surface roughly four times faster and say whether to retry shortly or that the host is unreachable.

Parity Contract

  • Legacy behavior preserved: a successful install is unchanged; non-429 failures keep their existing wording; skills.install_from_url keeps its documented 60s default and 600s ceiling.
  • Guard/fallback/dispatch parity checks: the rate-limit branch is narrow by construction — 429, or 5xx with a Retry-After — and the 404 test pins that it does not widen.

Duplicate / Superseded PR Handling

  • Duplicate PR(s): none
  • Canonical PR: this one
  • Resolution: N/A

Summary by CodeRabbit

  • New Features

    • Improved handling of throttled skill downloads, including clearer rate-limit errors and retry timing when available.
    • Reduced the catalog skill download timeout to 15 seconds.
  • Bug Fixes

    • Non-rate-limit HTTP failures, such as 404 errors, continue to show their specific status instead of being misclassified.
  • Tests

    • Added coverage for rate-limited responses, retry timing, and standard HTTP failures.

Installing a BUILT-IN skill fetches its SKILL.md live from
raw.githubusercontent.com. When that host is slow or throttling, the
install sat for a full 60 seconds behind a disabled button and then
failed with a string the UI could not act on (tinyhumansai#6409).

Two changes, both narrow.

Distinguish a throttled host from an unreachable one. Every non-2xx
previously produced the same `fetch failed: <url> returned status N`, so
"come back shortly" and "this host is not answering" were the same
message. A 429 — or a 5xx that carries Retry-After — now reports as rate
limiting and includes the host's own delay when it sends a parseable one.
`RATE_LIMITED_ERROR_PREFIX` is exported so a caller can branch on the
kind without parsing a status code back out of prose.

The header is read before anything consumes the response body, because
`response.bytes()` takes `response` by value and a later read is not
merely wrong but impossible. The same ordering silently drops the header
on the model-call path (tinyhumansai#6413, found independently); the comment notes
the parallel rather than inventing a shared helper for two call sites in
different crates.

Retry-After's HTTP-date form is deliberately not parsed: it would pull in
a date parser to serve a form GitHub's raw CDN does not send, and the
caller degrades correctly without it — the error still says the host is
throttling, just without a specific delay.

Cut the catalog install timeout from 60s to 15s, matching
`download::PROBE_TIMEOUT_SECS`, which bounds requests to the same hosts
in the same flow. `DEFAULT_INSTALL_TIMEOUT_SECS` is left at 60 — that is
the documented default for the public `skills.install_from_url` tool and
a scripted fetch is not a user waiting on a button.

Worth recording for whoever reads the issue: a 429 returns immediately,
so the reported 60-second hang was a transport timeout, not the 429 path.
QA saw the 429 separately via curl. The timeout cut addresses the logged
symptom; Retry-After addresses the throttling they observed.

The three new tests assert the KIND of failure, not that install fails —
the latter passes on the code this replaces. Reverting the branch fails
both 429 cases with `returned status 429` while the 404 case still
passes, so the branch discriminates rather than catching every non-2xx.
They live in their own file because the existing one reached the 750-line
layout cap.

Addresses tinyhumansai#6409 acceptance criteria 3 and 4. Criterion 1 — no dependency
on raw.githubusercontent.com at install time — is NOT addressed here and
is filed as tinyhumansai#6505; tinyhumansai#6409 stays open.

Refs tinyhumansai#6409
@M3gA-Mind
M3gA-Mind requested a review from a team September 23, 2026 01:56
@tinysweeper

tinysweeper Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

Tiny Sweeper review

Tiny Sweeper reviewed this change across 6 lane(s) and found 0 active actionable finding(s). Detailed lane evidence and any incomplete work are listed below.

State: Reviewing pending checks
Priority: none
Reviewed head: 59ab60073f46
Updated: 1790128772 (Unix time)

Review snapshot

Change surface Files Review signal Count
Production 3 Active findings 0
Tests 2 Noted findings 0
Documentation 0 Resolved findings 0
Configuration 0 Pending checks/questions 5

Completeness: Complete
Test assessment: No supported feature-to-test mapping was available; this does not mean tests are absent or passed.

What changed

The review could not produce a supported behavioral summary; inspect the cited changed surface and lane details below.

Features

None identified with supported citations.

Tests

No supported feature-to-test mapping was produced. Test execution is not inferred.

Findings

No active actionable findings.

Pending checks: Rust E2E (mock backend), Build Playwright E2E Artifact, E2E (Playwright / web lane), Desktop E2E (full suite, 3 OS), Rust Feature-Gate Smoke (gates off)

Before merge

  • Wait for Rust E2E (mock backend), Build Playwright E2E Artifact, E2E (Playwright / web lane), Desktop E2E (full suite, 3 OS), Rust Feature-Gate Smoke (gates off).

How this fits together

flowchart LR
  n0["install_from_catalog<br/>changed"]:::changed
  n1["url_validation<br/>changed"]:::changed
  n2["install_workflow_from_url<br/>changed"]:::changed
  n3["redact_url<br/>changed"]:::changed
  n4["make_legacy_skill<br/>changed"]:::changed
  n5["read_workflow_resource"]:::impacted
  n6["install_workflow_from_url_with_home"]:::impacted
  n7["format"]:::impacted
  n8["CatalogEntry"]:::impacted
  n9["execute"]:::impacted
  n10["...l_is_idempotent_when_skill_already_exists"]:::impacted
  n0 -->|calls| n2
  n0 -->|calls| n7
  n0 -->|uses| n8
  n2 -->|calls| n6
  n3 -->|calls| n7
  n4 -->|calls| n7
  n5 -->|calls| n7
  n6 -->|uses| n1
  n6 -->|calls| n3
  n6 -->|calls| n7
  n9 -->|calls| n0
  n9 -->|calls| n7
  n10 -->|calls| n6
  n10 -->|tests| n6
  n10 -->|calls| n7
  n10 -->|tests| n7
  classDef changed fill:#0d4429,stroke:#238636,color:#e6edf3
  classDef impacted fill:#161b22,stroke:#6e7681,color:#c9d1d9
  classDef flagged fill:#5a1e02,stroke:#d93f0b,color:#ffffff
  classDef blocking fill:#67060c,stroke:#f85149,color:#ffffff
Loading
Agent review details

critique

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Lane summary: Reviewed 5 files; 0 findings. _The code index is behind this pull request (indexed at `d7a097b5afc1`), so retrieved context may be out of date._ _3 memory call(s) failed (model: cortex: v1/recall: timed out after 10s), so this review saw part of what the engine holds._

security

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Lane summary: Reviewed 5 files; 0 findings. _The code index is behind this pull request (indexed at `d7a097b5afc1`), so retrieved context may be out of date._ _3 memory call(s) failed (model: cortex: v1/recall: timed out after 10s), so this review saw part of what the engine holds._

tests

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Lane summary: The change adds rate-limit detection to the SKILL.md fetch path, distinguishing throttled hosts from unreachable ones by inspecting the HTTP status and Retry-After header, and reduces the catalog-install timeout from 60s to 15s. Tests for the new failure-modelling logic are included and pin the contract. No behavioural regression or uncovered branch was identified. _The code index is behind this pull request (indexed at `d7a097b5afc1`), so retrieved context may be out of date._ _3 memory call(s) failed (model: cortex: v1/recall: timed out after 10s), so this review saw part of what the engine holds._

commits

  • Conclusion: Neutral
  • Scope reviewed: all assigned evidence
  • Lane summary: Nothing sensitive found in what this pull request commits.

description

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Lane summary: This change improves error reporting for skill install failures by distinguishing rate-limited hosts from unreachable ones, reading Retry-After headers, and reducing the catalogue install timeout from 60s to 15s. The tests cover the new failure branches. The change is safe to merge. _The code index is behind this pull request (indexed at `d7a097b5afc1`), so retrieved context may be out of date._ _3 memory call(s) failed (model: cortex: v1/recall: timed out after 10s), so this review saw part of what the engine holds._

e2e

  • Conclusion: Neutral
  • Scope reviewed: all assigned evidence
  • Lane summary: This change adds distinguishable error messages for rate-limited vs. unreachable hosts during SKILL.md install, reduces the catalog install timeout from 60s to 15s, and adds unit tests for the new rate-limit error paths. The behavioural changes are covered by the new unit tests; no e2e test coverage gap exists for these changes. Waiting on end-to-end jobs: `Rust E2E (mock backend)`, `Build Playwright E2E Artifact`, `E2E (Playwright / web lane)`, `Desktop E2E (full suite, 3 OS)`, `Rust Feature-Gate Smoke (gates off)`.
  • Unresolved questions/checks: Rust E2E (mock backend), Build Playwright E2E Artifact, E2E (Playwright / web lane), Desktop E2E (full suite, 3 OS), Rust Feature-Gate Smoke (gates off)
Evidence and run details
  • Models: ladder/vectors, gpt-5.6-luna, deepseek/deepseek-v4-flash
  • Spend: $0.017150
  • Tokens: 298061 input · 5061 output · 13984 cached · 779 embedding
Head State Pass summary
59ab60073f46 pending 0 active finding(s), 0 resolved finding(s) (at 1790128772)

tinysweeper 0.1.0

@coderabbitai

coderabbitai Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 14a08e92-64c3-4f6f-b893-c03f07902884

📥 Commits

Reviewing files that changed from the base of the PR and between 0f1ecc9 and 59ab600.

📒 Files selected for processing (5)
  • crates/openhuman-core/src/skills/catalog/ops.rs
  • crates/openhuman-core/src/skills/ops_install.rs
  • crates/openhuman-core/src/skills/ops_install/fetch.rs
  • crates/openhuman-core/src/skills/ops_install_failure_tests.rs
  • crates/openhuman-core/src/skills/ops_tests.rs

Included review availability: Your plan provides up to 10 included reviews per hour; 2 remain after this review.


📝 Walkthrough

Walkthrough

The install path now uses a 15-second catalog timeout. Fetch handling classifies HTTP 429 and eligible 5xx responses as rate-limited, includes parsed retry delays, and preserves status errors for other failures. Tests cover these cases.

Changes

Install fetch reliability

Layer / File(s) Summary
Catalog install timeout
crates/openhuman-core/src/skills/catalog/ops.rs
Catalog installs use the private CATALOG_INSTALL_TIMEOUT_SECS constant set to 15 seconds.
Rate-limit error handling and validation
crates/openhuman-core/src/skills/ops_install/fetch.rs, crates/openhuman-core/src/skills/ops_install.rs, crates/openhuman-core/src/skills/ops_install_failure_tests.rs, crates/openhuman-core/src/skills/ops_tests.rs
Fetch handling parses delta-seconds Retry-After headers. HTTP 429 and eligible 5xx responses use the rate-limited error prefix. Tests cover retry delays, missing headers, and 404 responses.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix · Severity of issue fixed: Medium

Suggested reviewers: senamakel

Merge Risk: ⚪ Minimal · up to 59ab6

Rate-limited and ordinary fetch failures retain the intended behavior, with no actionable merge-blocking issue identified.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Linked Issues check ⚠️ Warning PR #6508 implements part of #6409. It sets the catalogue install timeout to 15 seconds and classifies HTTP 429 responses, plus 5xx responses with a parseable Retry-After, as rate limited. It include… Implement the #6409 requirement that built-in skills do not depend on live raw.githubusercontent.com fetches. Bundle the skill content or proxy it through a cached TinyHumans service. Add regression coverage for the selected delivery path…
Docstring Coverage ⚠️ Warning Docstring coverage is 75.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 8 functions across 5 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: skill fetch failures now fail faster and identify refusal reasons, including rate limiting.
Out of Scope Changes check ✅ Passed The changed files support #6409. They adjust catalogue and fetch timeouts, classify rate-limited fetch failures, expose the error prefix to callers, and add focused failure tests. No unrelated change …
Full details: Linked Issues check

Explanation

PR #6508 implements part of #6409. It sets the catalogue install timeout to 15 seconds and classifies HTTP 429 responses, plus 5xx responses with a parseable Retry-After, as rate limited. It includes the delay when available. Tests cover 429 with and without Retry-After and a 404 fallback. The direct-fetch implementation remains: install_workflow_from_url fetches the catalog entry URL, and the issue requires built-in skills to be bundled or served through a cached TinyHumans backend. The added tests do not cover the 5xx Retry-After branch.

Resolution

Implement the #6409 requirement that built-in skills do not depend on live raw.githubusercontent.com fetches. Bundle the skill content or proxy it through a cached TinyHumans service. Add regression coverage for the selected delivery path and for 5xx responses with Retry-After.

  • Fix all pre-merge checks with AI

A rabbit found a timeout bright,
Fifteen seconds trimmed the night.
When throttled clouds say, “Wait,”
Retry seconds mark the gate.
Clear errors help paws proceed.

Comment @coderabbitai help to get the list of available commands.

✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Commit to this branch
  • Create a new PR

@tinysweeper tinysweeper Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tinysweeper found nothing blocking. Approving.

             $0.0171 · 298,061 in / 5,061 out · 13,984 cached (5%) · ladder/vectors, gpt-5.6-luna, deepseek/deepseek-v4-flash · 779 embedded
critique:    $0.0053 · 123,119 in / 1,526 out · 8,423 cached (7%)  · gpt-5.6-luna
security:    $0.0065 · 119,271 in / 1,639 out · 5,561 cached (5%)  · gpt-5.6-luna
tests:       $0.0016 · 17,935 in  / 100 out   · 0 cached (0%)      · deepseek/deepseek-v4-flash
description: $0.0009 · 10,437 in  / 68 out    · 0 cached (0%)      · deepseek/deepseek-v4-flash
e2e:         $0.0020 · 21,971 in  / 91 out    · 0 cached (0%)      · deepseek/deepseek-v4-flash

@tinysweeper tinysweeper Bot added the priority: p3 Whenever. Cosmetic, a nicety, or a cleanup with no user visible effect. label Sep 23, 2026
@senamakel
senamakel merged commit 605bcc3 into tinyhumansai:main Sep 23, 2026
32 of 35 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

priority: p3 Whenever. Cosmetic, a nicety, or a cleanup with no user visible effect.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Skills: BUILT-IN skills are fetched live from NousResearch GitHub raw at install; 60 s silent hang on 429, no UI feedback

2 participants