Repository navigation
feat(skills): activation persistence, latency instrumentation, registry browse cache + pagination - #4288
Conversation
… add latency tracking for tool chains
… improved performance
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughAdds durable Composio connection caching, skill browse caching, client-side catalog pagination, per-thread skill tool-chain latency tracking, and E2E coverage for activation persistence and chat invocation. ChangesDurable Composio Connection Cache
Skill Registry Browse Cache and Client-Side Pagination
Skill Tool-Chain Latency Tracker
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Comment |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e35b767898
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
Caution Failed to replace (edit) comment. This is likely due to insufficient permissions or the comment being deleted. Error details |
There was a problem hiding this comment.
Actionable comments posted: 8
🧹 Nitpick comments (1)
app/src/services/api/skillRegistryApi.test.ts (1)
168-221: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winSolid cache coverage; consider two gaps.
The cache-hit, concurrent-dedup, force-refresh and invalidation cases are well covered. Two behaviors documented on the cache remain untested and would be cheap to add with fake timers / a rejected mock:
- TTL expiry: after advancing past
BROWSE_CACHE_TTL_MS(usevi.useFakeTimers()+vi.advanceTimersByTime), the nextbrowse()should re-fetch.- Failure isolation: a rejected
callCoreRpcshould leave the cache empty (and clearbrowseInflight) so the next call retries rather than serving a poisoned/empty result.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@app/src/services/api/skillRegistryApi.test.ts` around lines 168 - 221, The cache tests in skillRegistryApi.test.ts cover hits, concurrent dedupe, force refresh, and invalidation, but they still miss TTL expiry and failure recovery. Add a test around skillRegistryApi.browse using fake timers to advance past BROWSE_CACHE_TTL_MS and assert the next call re-fetches, and add a rejected callCoreRpc case to verify browse does not retain a poisoned cache entry and that browseInflight is cleared so a later browse retries. Use the existing browse and invalidateSkillBrowseCache helpers and the callCoreRpc mock to locate the behavior under test.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@app/src/lib/ai/skillToolChainLatency.ts`:
- Around line 26-27: The `toolCount` field in `SkillToolChainLatency` is
mislabeled because `noteToolCall()` only keys off `threadId`, so it tracks total
calls per thread rather than distinct tools. Update the contract to match the
behavior by either renaming `toolCount` and the related
`noteToolCall()`/consumer usage to reflect “tool call count,” or extend the
tracking to accept a tool identifier and de-duplicate if distinct-tool counting
is intended. Make sure the `SkillToolChainLatency` class and any downstream
logging/telemetry code use the same meaning consistently.
In `@app/src/lib/composio/connectionCache.ts`:
- Around line 100-104: `readConnectionCache()` is caching an expired snapshot in
`memory`, which later lets `writeConnectionCache()` merge against stale
`toolkits`/`catalog` data and revive it. Update the
`readConnectionCache`/`writeConnectionCache` flow in `connectionCache.ts` so
expired disk reads do not become the merge base for subsequent writes; only
populate `memory` when the cached entry is still fresh, or otherwise treat the
cache as empty for merges. Ensure the `useComposioIntegrations()` path cannot
persist partial writes over an expired snapshot.
In `@app/src/lib/composio/hooks.ts`:
- Around line 138-154: The post-invalidation cache writes in the Composio hooks
are racy because in-flight refresh/poll callbacks can still call setConnections
and writeConnectionCache after clearConnectionCache() has run. Add a
config-generation or request-token guard in the relevant hook paths (around the
refresh/poll logic in hooks.ts) so only results from the currently active
Composio client are allowed to commit state. Use the existing refresh/poll
callbacks that write freshConnections/toolkits/catalog to compare against the
latest generation before mutating cache or state, and ignore stale responses
from previous tenants.
In `@app/src/providers/__tests__/ChatRuntimeProvider.test.tsx`:
- Around line 1455-1475: The `console.warn` spy in the
`ChatRuntimeProvider.test.tsx` latency tests is being restored inside each test,
so a failing assertion can leave the spy active for later tests. Move the
restore into the shared cleanup path used by these tests (for example the common
`afterEach`/test teardown alongside `vi.useRealTimers()`), and remove the
per-test `warnSpy.mockRestore()` calls. Keep the existing `vi.spyOn(console,
'warn')` usage in the affected test cases (`renderProvider`, `onToolCall`,
`onDone`) but ensure the original `console.warn` implementation is always
restored even when an expectation throws.
In `@app/src/providers/ChatRuntimeProvider.tsx`:
- Around line 521-523: The latency chain is keyed too broadly in
ChatRuntimeProvider, causing parallel turns that share a thread_id to overwrite
each other’s measurements. Update the tool-chain tracking in the affected turn
flow to use request_id as the primary key instead of event.thread_id, and make
sure both noteToolCall and finishChain operate on the same request-scoped
identifier. Apply the same change consistently in the other affected call sites
so each forked request maintains its own elapsed time and tool count.
- Around line 291-294: The provider-level skill latency tracker persists across
turns, but the disconnect reconciliation path can tear down in-flight turns
without clearing its state, causing stale startedAt/toolCount to leak into the
next tool call. Update the socket-disconnect teardown logic in
ChatRuntimeProvider to reset or finish the tracker whenever turns are abandoned,
using the existing skillLatencyRef and its tracker methods (finishChain/reset)
so each new thread/tool chain starts cleanly.
In `@app/test/e2e/specs/skill-activate-invoke-chat.spec.ts`:
- Around line 123-131: The Composio execute verification in the skill
activate/invoke chat spec is only warning on a missing tool call, so the test
can still pass without the expected routing. Update the check around
waitForToolCallInMockLog and toolHit in skill-activate-invoke-chat.spec.ts to
fail the test immediately when the /agent-integrations/composio/execute call is
not observed, using a hard assertion instead of console.warn so the
activate→invoke path is enforced.
In `@app/test/e2e/specs/skill-activation-persistence.spec.ts`:
- Around line 62-67: The simulated restart in simulateRestart() only remounts
routes within the same renderer, so it never validates the durable cache path.
Update the e2e flow in skill-activation-persistence.spec.ts to force a true
webview reload/relaunch, or explicitly clear the connectionCache.ts module-level
singleton before the second assertion, so readConnectionCache() cannot satisfy
the test from in-memory state populated by writeConnectionCache().
---
Nitpick comments:
In `@app/src/services/api/skillRegistryApi.test.ts`:
- Around line 168-221: The cache tests in skillRegistryApi.test.ts cover hits,
concurrent dedupe, force refresh, and invalidation, but they still miss TTL
expiry and failure recovery. Add a test around skillRegistryApi.browse using
fake timers to advance past BROWSE_CACHE_TTL_MS and assert the next call
re-fetches, and add a rejected callCoreRpc case to verify browse does not retain
a poisoned cache entry and that browseInflight is cleared so a later browse
retries. Use the existing browse and invalidateSkillBrowseCache helpers and the
callCoreRpc mock to locate the behavior under test.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 5dc84bda-6cdb-4dad-b47a-80acd588ec96
📒 Files selected for processing (14)
app/src/components/skills/SkillsExplorerTab.tsxapp/src/components/skills/__tests__/SkillsExplorerTab.test.tsxapp/src/lib/ai/skillToolChainLatency.test.tsapp/src/lib/ai/skillToolChainLatency.tsapp/src/lib/composio/connectionCache.test.tsapp/src/lib/composio/connectionCache.tsapp/src/lib/composio/hooks.test.tsapp/src/lib/composio/hooks.tsapp/src/providers/ChatRuntimeProvider.tsxapp/src/providers/__tests__/ChatRuntimeProvider.test.tsxapp/src/services/api/skillRegistryApi.test.tsapp/src/services/api/skillRegistryApi.tsapp/test/e2e/specs/skill-activate-invoke-chat.spec.tsapp/test/e2e/specs/skill-activation-persistence.spec.ts
…torage and enhance tool-chain latency tracking
|
Pushed Summary of fixes
Re the |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@app/test/e2e/specs/skill-activation-persistence.spec.ts`:
- Around line 100-105: The persistence check is using a suffix match on
localStorage keys, which can accidentally read a stale connection blob from
another user. Update the e2e test in skill-activation-persistence.spec.ts to
resolve the active user id first and then read and assert against the exact
`${userId}:composio:connections:v1` entry from browser.execute, instead of using
Object.keys(...).find(...) with `endsWith('composio:connections:v1')`.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 41f7c292-c7ef-4391-80e2-a9cb37d1d127
📒 Files selected for processing (9)
app/src/lib/ai/skillToolChainLatency.tsapp/src/lib/composio/connectionCache.test.tsapp/src/lib/composio/connectionCache.tsapp/src/lib/composio/hooks.test.tsapp/src/lib/composio/hooks.tsapp/src/providers/ChatRuntimeProvider.tsxapp/src/providers/__tests__/ChatRuntimeProvider.test.tsxapp/test/e2e/specs/skill-activate-invoke-chat.spec.tsapp/test/e2e/specs/skill-activation-persistence.spec.ts
🚧 Files skipped from review as they are similar to previous changes (7)
- app/test/e2e/specs/skill-activate-invoke-chat.spec.ts
- app/src/providers/tests/ChatRuntimeProvider.test.tsx
- app/src/providers/ChatRuntimeProvider.tsx
- app/src/lib/ai/skillToolChainLatency.ts
- app/src/lib/composio/hooks.test.ts
- app/src/lib/composio/hooks.ts
- app/src/lib/composio/connectionCache.ts
…data is accessed correctly
M3gA-Mind
left a comment
There was a problem hiding this comment.
Reviewed and verified: correct, well-tested, CI green/clearing. Approving. Thanks!
…try browse cache + pagination (tinyhumansai#4288)\n\n
Summary
Problem
Issue #4273 (Skills page — end-to-end completion) called out gaps in the skill lifecycle: activation state not surviving restart, no E2E for the activate→invoke-from-chat path, and no visibility into the 60s latency target. Separately, the registry tab was slow (full ~90k-entry catalog re-fetched + re-parsed on every mount) and the catalog UI dead-ended at 60 entries with a "refine your search" hint but no pagination control.
Solution
app/src/lib/composio/connectionCache.ts— synchronouslocalStorage+ in-memory mirror (mirrors the siblingcatalogCache.ts), namespaced byOPENHUMAN_ACTIVE_USER_IDand inert when no user is identified (read→null, write→no-op) so two unidentified sessions can never share a blob.useComposioIntegrationsseeds initial state from it and writes on every successful fetch/poll; evicted on direct-mode key removal and tenant switch. Not persisted as a Redux slice deliberately — coupling the Provider-less hook to the store would have broken itsrenderHooktests.app/src/lib/ai/skillToolChainLatency.ts— pure, injectable-clock tracker wired intoChatRuntimeProvider's existingonToolCall/onDone/onError. Non-enforcing: did not clamp the 120s tool timeout to 60s (that would cut off legitimately long chains = breaking).skillRegistryApi.browse()gains an in-memory TTL cache + shared in-flight promise;force_refreshand a newinvalidateSkillBrowseCache()drop it. Not localStorage — the payload far exceeds the ~5MB quota.SkillsExplorerTabretains the full fetched list and pages through it client-side viavisibleCount(no extra RPC), replacing the contradictory "refine your search" copy with acommon.showMorebutton (no new i18n keys).Submission Checklist
pnpm test:coveragelocally to confirm before merge. New TS lines are covered by the added unit tests; the 2 E2E specs do not count toward diff-cover.docs/TEST-COVERAGE-MATRIX.md(latency is observability-only →N/A).docs/RELEASE-MANUAL-SMOKE.md.Closes #NNNin the## RelatedsectionImpact
Related
limit/offsetonskill_registry_browse; install-without-full-catalog-refetch; poll-pause-on-tab-hidden;search()result cache; tile virtualization; clear connection cache on logout.Summary by CodeRabbit
New Features
Bug Fixes