Skip to content

perf(skills): discover installed skills once per workflow lookup - #5523

Closed
mysma-9403 wants to merge 1 commit into
tinyhumansai:mainfrom
mysma-9403:perf/skills-registry-single-discover
Closed

mysma-9403 wants to merge 1 commit into
tinyhumansai:mainfrom
mysma-9403:perf/skills-registry-single-discover

Conversation

@mysma-9403

@mysma-9403 mysma-9403 commented Aug 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • get_workflow_with_profile — the resolution seam behind run_workflow / describe_workflow — walked the installed-skill roots twice per call: once inside load_workflows_with_profile, and again to resolve a display-name in the fallback branch.
  • Extract discover_all (prune + one discovery walk) and definitions_from_discovered (builtins + parse each discovered skill); discover once and feed both the id lookup and the name fallback from that single list.
  • Behavior-preserving refactor — the parsed list and resolution order are unchanged.

Problem

Resolving one skill by id parses the whole catalogue and, on the fallback path, re-walks the roots:

let workflows = load_workflows_with_profile(workspace_dir, profile_skills_root); // discover walk #1 + parse all
if let Some(exact) = workflows.iter().rev().find(|s| s.definition.id == id) { return Some(exact.clone()); }
// display-name fallback:
let slug = discover_workflows_with_profile(/* … */)   // discover walk #2 — from scratch
    .into_iter().find(|w| w.scope == WorkflowScope::Profile && w.name == id)…;

discover_workflows_with_profile already reads every skill's WORKFLOW.md/SKILL.md frontmatter; load_workflows_with_profile then reads each one again to parse its definition; and the name-fallback discovers a second full time. So a single run_workflow pays O(N_installed_skills) filesystem reads, with the fallback path re-walking the roots it just walked.

Solution

Split the work so discovery happens once and is reused:

  • discover_all(workspace_dir, profile_skills_root) — prune (before discovery, unchanged ordering) + one discover_workflows_with_profile walk, returning the lightweight Workflow entries.
  • definitions_from_discovered(&[Workflow]) — builtins prepended + parse each discovered skill's definition.
  • load_workflows_with_profile now delegates to both (behavior identical).
  • get_workflow_with_profile calls discover_all once, derives the parsed list from it, and — on the name-fallback — reuses that same discovery instead of walking the roots again.

This is a pure refactor: the parsed workflow list is byte-identical (same builtins, same discovery, same per-skill parse), the exact-id then display-name resolution order is unchanged, and the fallback reads the same discovery it did before — computed once rather than twice. It removes one full root walk per lookup (a third markdown read of every installed skill on the fallback path).

Submission Checklist

  • Tests added or updated (happy path + at least one failure / edge case) — behavior-preserving refactor; covered by the existing resolution suite, which now exercises both extracted helpers: get_workflow_with_profile_resolution_matrix (id resolution + precedence), get_profile_workflow_resolves_distinct_display_name (the reused name fallback), profile_workflow_exact_id_overrides_builtin (builtin vs discovered), skill_md_only_install_resolves_by_dir_slug_not_frontmatter_name (slug vs frontmatter name), and prune_removes_legacy_bundled_only (prune-before-discover ordering). All 15 skills::registry tests pass.
  • Diff coverage ≥ 80% — the two extracted helpers and the rewritten lookup are exercised by the resolution suite above (both are on the call path of every get_workflow* / load_workflows* test).
  • Coverage matrix updated — N/A: behaviour-preserving performance refactor, no matrix row added/removed/renamed.
  • All affected feature IDs from the matrix are listed under ## Related — N/A.
  • No new external network dependencies introduced.
  • Manual smoke checklist updated if this touches release-cut surfaces — N/A: internal resolution path, identical behavior.
  • Linked issue closed via Closes #NNN — N/A: no tracking issue (self-identified hot-path redundancy).

Impact

  • Runtime: desktop/core — every run_workflow / describe_workflow (and any get_workflow caller). Removes a redundant full discovery walk per lookup; the win scales with the number of installed skills.
  • Compatibility: none — resolution results and precedence are unchanged.

Related

  • Closes: N/A
  • Follow-up PR(s)/TODOs: the primary lookup still parses every discovered skill's definition to match one id; resolving a single skill lazily (parse only the matched directory) is a larger change with a subtler correctness argument (workflow.toml ids can differ from the dir slug) and is intentionally out of scope here.

AI Authored PR Metadata (required for Codex/Linear PRs)

Linear Issue

  • Key: N/A
  • URL: N/A

Commit & Branch

  • Branch: perf/skills-registry-single-discover

Summary by CodeRabbit

  • Bug Fixes

    • Improved workflow discovery to consistently identify applicable workflows.
    • Removed outdated workflows from loading results.
    • Improved profile display-name resolution when viewing workflow details.
  • Performance

    • Streamlined workflow loading to avoid repeated discovery, providing more efficient results.

@mysma-9403
mysma-9403 requested review from a team and a lite review from Copilot August 12, 2026 12:48
@coderabbitai

coderabbitai Bot commented Aug 12, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: 7b9ab7da-f711-4e3b-b65d-4e9845fd1a3f

📥 Commits

Reviewing files that changed from the base of the PR and between fa044d3 and 0a43c90.

📒 Files selected for processing (1)
  • src/openhuman/skills/registry.rs
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/openhuman/skills/registry.rs

Included review availability: Your plan provides up to 10 included reviews per hour; 7 remain after this review.


📝 Walkthrough

Walkthrough

Workflow discovery now runs through a shared helper. The registry reuses discovered workflows for definition loading and profile display-name resolution, removing repeated root scans.

Changes

Workflow discovery and lookup

Layer / File(s) Summary
Centralized workflow discovery
src/openhuman/skills/registry.rs
The registry discovers applicable workflows once, prunes legacy entries, and builds definitions from the shared results.
Profile lookup reuse
src/openhuman/skills/registry.rs
Profile-aware lookup reuses discovered workflows for definition selection and display-name resolution. Scope filtering and slug fallback remain unchanged.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to 0a43c

This change removes a redundant skill-discovery pass while preserving workflow resolution order and results. No actionable merge-blocking risk remains beyond normal checks and review.

Poem

I hop through workflows, tidy and bright,
One discovery pass makes the path right.
Profiles find names without scanning anew,
Legacy trails vanish from view.
Squeak, says the rabbit, the registry grew!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: discovering installed skills once per workflow lookup to remove redundant scanning.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 4 functions across 1 files.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Warning

Your free Security trial is over. An organization admin can upgrade to Advanced for continuous pull request security review or dismiss this notice.


Comment @coderabbitai help to get the list of available commands.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors the skills/workflow registry lookup path to avoid re-discovering installed skill roots multiple times during a single get_workflow_with_profile call (the core resolution seam behind run_workflow / describe_workflow). This reduces redundant filesystem walks while preserving the existing resolution order and outputs.

Changes:

  • Extracts a shared discovery helper (discover_all) that prunes legacy bundled skills and performs a single discovery walk.
  • Extracts definitions_from_discovered to parse WorkflowDefinitions from an already-discovered list, keeping built-ins prepended.
  • Updates get_workflow_with_profile to reuse the single discovery result for both exact-id lookup and display-name fallback.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@mysma-9403

Copy link
Copy Markdown
Contributor Author

CI note — the two red Rust lanes are pre-existing on main, not from this PR

  • Rust Feature-Gate Smoke (gates off) — its scoped cargo test --no-default-features --lib step fails on two tests in config/migration_helpers (migrate_hermes_apply_imports_markdown_entries, migrate_openclaw_apply_imports_markdown_entries_into_target_workspace), which panic with no EmbeddingHost installed — the host must call memory::embedding_host::set_embedding_host during startup wiring. Those tests are untouched by this PR. I reproduced them on a clean checkout of main with cargo test --no-default-features --lib config::migration_helpers::ops (2/8 fail) — it's fallout from the tinymemory v0.3.0 extraction, surfacing only in the gates-off test run (a plain cargo check --no-default-features on main compiles clean). Older-base PRs branched before it merged still pass this lane.
  • Rust Core Coverage (cargo-llvm-cov) — the chronic pre-existing red on Rust PRs.

Rust Quality (fmt, clippy) passes, which is the signal that this PR itself compiles and lints clean under the product feature set. The change here is confined to its own module and unrelated to either red lane.

senamakel added a commit that referenced this pull request Aug 14, 2026
`main` currently fails `cargo check --locked --features <product> --tests` in
three independent ways. Each was hidden behind the previous one, and none is
visible to CI Lite, which scopes Rust work to changed areas — so a submodule
advance that breaks an unrelated file, a stale lockfile, and integration tests
that no changed-file heuristic selects can all land green and stay green.

Every PR then inherits all three and looks individually broken. #5519, #5521,
#5523 and #5533 are currently red on `Rust Core Coverage` for this reason.

1. `flows/n8n_import.rs` — `WorkflowGraph` gained an `agents` field when
   tinyflows advanced (#5537), and the struct literal here was not updated.
   Filled with `Vec::new()`, matching the adjacent `inputs`: an n8n workflow
   has no agent declaration, so an import brings none across. Inventing agents
   the source never described would be worse than declaring none.

2. Root `Cargo.lock` — stale against the manifest, so `--locked` refuses
   outright. Regenerated; no `--workspace` sweep.

3. `Config::cli_inference_snapshot` was introduced as `pub(crate)` in
   2c7142c. `Config` is built with struct-literal syntax by EIGHT integration
   tests, which are external crates, and that syntax requires every field to be
   visible — so one crate-private field makes the whole struct unconstructible
   from outside:

     agent_retrieval_e2e        keyring_secretstore_e2e
     json_rpc_e2e               keyring_secretstore_fresh_e2e
     memory_golden_fixture_e2e  memory_roundtrip_e2e
     memory_sync_pipeline_e2e   memory_tree_summarizer_e2e

   Restored to `pub`, along with the type it names and a public path to it.

   This does partially relax "keep CLI override module export-only", so to be
   explicit about the tradeoff: that commit's goal was module structure, and
   breaking `Config`'s external constructibility reads as collateral rather
   than intent. Only the type appearing in `Config`'s public field list becomes
   public; every function in the module stays crate-internal, so it remains
   export-only. `#[serde(skip)]` and `#[schemars(skip)]` are what keep the
   field off the wire and out of the JSON schema — visibility was never doing
   that work.

   The alternative was rewriting eight test files to build by mutation. That is
   more churn, and it leaves the trap armed for the ninth.

Verified: `cargo check --features <product> --tests` and
`cargo clippy --features <product> --lib -- -D warnings` both clean.

Co-authored-by: Medulla <medulla@tinyhumans.ai>
`get_workflow_with_profile` — the resolution seam behind `run_workflow` /
`describe_workflow` — walked the skill roots twice per call: once inside
`load_workflows_with_profile` (which discovers, then parses every skill's
definition) and again for the display-name fallback, which re-ran
`discover_workflows_with_profile` from scratch only to map a frontmatter name
back to its slug.

Extract two helpers — `discover_all` (prune + a single discovery walk) and
`definitions_from_discovered` (builtins + parse each discovered skill) — and
have `get_workflow_with_profile` discover once, feeding both the parsed-
definition lookup and the name fallback from the same list. `load_workflows_with_profile`
now delegates to the same helpers, so its behavior is unchanged.

This is a behavior-preserving refactor: the parsed workflow list is identical
(same builtins, same discovery, same per-skill parse), and the name fallback
reads the same discovery it did before — just computed once instead of twice.
The prune still runs before discovery in both paths. Removes one full root walk
per lookup (a third markdown read of every installed skill on the fallback
path).

Covered by the existing resolution suite, which exercises both extracted
helpers: `get_workflow_with_profile_resolution_matrix`,
`get_profile_workflow_resolves_distinct_display_name` (the reused name
fallback), `profile_workflow_exact_id_overrides_builtin`,
`skill_md_only_install_resolves_by_dir_slug_not_frontmatter_name`, and
`prune_removes_legacy_bundled_only` (prune-before-discover ordering).

Claude-Session: https://claude.ai/code/session_01ACB4Ugi5pJMQqoCbZnVo6f
@M3gA-Mind
M3gA-Mind force-pushed the perf/skills-registry-single-discover branch from bee05ca to 0a43c90 Compare September 1, 2026 22:47
@coderabbitai

coderabbitai Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

@tinysweeper tinysweeper Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tinysweeper found nothing blocking. Approving.

             $0.0049 · 42,514 in / 845 out · 9,294 cached (22%) · openrouter/openai/text-embedding-3-small, deepseek/deepseek-v4-flash, z-ai/glm-5.2 · 197 embedded
critique:    $0.0009 · 12,707 in / 79 out  · 0 cached (0%)      · deepseek/deepseek-v4-flash
security:    $0.0026 · 11,919 in / 599 out · 9,294 cached (78%) · z-ai/glm-5.2
tests:       $0.0009 · 12,599 in / 106 out · 0 cached (0%)      · deepseek/deepseek-v4-flash
description: $0.0004 · 5,289 in  / 61 out  · 0 cached (0%)      · deepseek/deepseek-v4-flash

@tinysweeper

tinysweeper Bot commented Sep 1, 2026

Copy link
Copy Markdown

How this change flows

2 changed behaviours across 13 relationships. 6 surrounding behaviours are shown (60 graph nodes walked). 38 further behaviours left out to keep the diagram readable.

flowchart LR
  n0["get_workflow_with_profile<br/>changed"]:::changed
  n1["load_workflows_with_profile<br/>changed"]:::changed
  n2["discover_workflows_with_profile"]:::impacted
  n3["is_workspace_trusted"]:::impacted
  n4["WorkflowDefinition"]:::impacted
  n5["get_workflow"]:::impacted
  n6["spawn_workflow_run_background_with_profile"]:::impacted
  n7["execute"]:::impacted
  n0 -->|calls| n1
  n0 -->|calls| n2
  n0 -->|calls| n3
  n0 -->|uses| n4
  n1 -->|calls| n2
  n1 -->|calls| n3
  n1 -->|uses| n4
  n5 -->|calls| n0
  n5 -->|uses| n4
  n6 -->|calls| n0
  n7 -->|calls| n0
  n7 -->|calls| n2
  n7 -->|calls| n3
  classDef changed fill:#0d4429,stroke:#238636,color:#e6edf3
  classDef impacted fill:#161b22,stroke:#6e7681,color:#c9d1d9
  classDef flagged fill:#5a1e02,stroke:#d93f0b,color:#ffffff
  classDef blocking fill:#67060c,stroke:#f85149,color:#ffffff
Loading

Green: changed behaviour. Grey: surrounding behaviour. Arrows name the call, use, implementation, or test relationship. Orange: has findings. Red: has a finding that blocks the merge.

tinysweeper 0.1.0

@tinysweeper tinysweeper Bot added the priority: p3 Whenever. Cosmetic, a nicety, or a cleanup with no user visible effect. label Sep 1, 2026
@M3gA-Mind

Copy link
Copy Markdown
Collaborator

Maintainer review — and a disclosure: I pushed to this branch

First, the disclosure. Before our fleet's rules changed mid-session to review-only for external contributions, I rebased this branch onto current main and force-pushed it to mysma-9403:perf/skills-registry-single-discover. That was me, not you:

  • old head bee05cab → new head 0a43c906
  • It is a pure rebase. Your commit is unchanged in content (author still you, message untouched); only its parent moved from 2826259d to current main. git diff bee05cab..0a43c906 on the contents of src/openhuman/skills/registry.rs is empty.
  • Nothing else was added, and no other file was touched.

Apologies for touching your branch without asking. If you have local work on that branch, git fetch && git reset --hard origin/perf/skills-registry-single-discover will pick up the rebase; nothing of yours was dropped. I have not approved or merged anything.

Why it was worth doing. The PR was MERGEABLE but its CI was from 2026-08-12 and red, and the two red tests were nothing to do with your change:

openhuman::config::migration_helpers::ops::tests::migrate_hermes_apply_imports_markdown_entries
openhuman::config::migration_helpers::ops::tests::migrate_openclaw_apply_imports_markdown_entries_into_target_workspace
  panicked: "no EmbeddingHost installed — the host must call
             memory::embedding_host::set_embedding_host during startup wiring"

That is config/migration_helpers, and this PR only touches skills/registry.rs. It was a stale-base failure, not a defect in your work. Fresh CI is now running against current main.

Assessment of the change itself

The change is real and the reasoning in the description holds up. get_workflow_with_profile did walk the roots twice — once inside load_workflows_with_profile and again for the display-name fallback — and the split into discover_all + definitions_from_discovered removes the second walk without moving the prune or changing resolution order. The rebase was clean, which is itself some evidence the seam you picked is a stable one.

Two things a reviewer will want to confirm, neither of which is an objection:

  1. prune_legacy_default_workflows now runs once per get_workflow_with_profile instead of twice. That is the point of the change and the prune is documented as idempotent, so this should be fine — but it is the one observable behavioural difference, and it is worth saying out loud rather than leaving it under "behavior-preserving".
  2. discover_all returns Vec<Workflow> and definitions_from_discovered takes &[Workflow], so the fallback path now reuses entries parsed under the same trust decision as the id lookup. Previously the second walk re-evaluated is_workspace_trusted. Same answer in practice; just worth a sentence in the description.

What is left for you

Nothing, unless CI comes back red on the new run — watch https://github.com/tinyhumansai/openhuman/actions/runs/33567965724. I am not approving this; a maintainer reviews and merges.

@senamakel senamakel closed this Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

priority: p3 Whenever. Cosmetic, a nicety, or a cleanup with no user visible effect.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants