Skip to content

feat: harden provider seam diagnostics#55

Merged
omarluq merged 1 commit into
mainfrom
feat/runtime-seam-hardening
May 28, 2026
Merged

feat: harden provider seam diagnostics#55
omarluq merged 1 commit into
mainfrom
feat/runtime-seam-hardening

Conversation

@omarluq

@omarluq omarluq commented May 28, 2026

Copy link
Copy Markdown
Owner

Summary

  • add richer provider response/error diagnostics
  • dispatch provider response/error lifecycle handlers and emit hook diagnostics
  • cover provider response, retryable error metadata, and hook failure diagnostics

Validation

  • mise exec -- go test ./internal/assistant
  • mise exec -- task ci
  • cr review --agent -t uncommitted --base main (timed out before findings)

Note: docs/refactoring/* remain local/uncommitted and are not part of this PR.

@coderabbitai

coderabbitai Bot commented May 28, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Summary by CodeRabbit

  • New Features

    • Enhanced diagnostic capture for provider lifecycle events with detailed response metrics and error metadata
    • Added hook error tracking in error diagnostics
  • Tests

    • Added comprehensive test coverage for provider lifecycle diagnostics emission
    • Added tests verifying error metadata and hook failure tracking

Walkthrough

The PR refactors provider diagnostics by introducing shared payload builders and integrating them into the unified lifecycle event dispatch pipeline. providerBaseDiagnostics centralizes common fields, while specialized functions append role-specific metrics. Event emission now dispatches via a unified lifecycle handler and always emits accompanying diagnostics. Three tests verify successful response, error metadata, and hook error capture in diagnostic payloads.

Changes

Provider Lifecycle Diagnostics

Layer / File(s) Summary
Diagnostics payload structure
internal/assistant/lifecycle_diagnostics.go
providerBaseDiagnostics centralizes common fields from the completion request/model/session. providerHookDiagnostics, providerResponseDiagnostics, and providerErrorDiagnostics are updated to reuse the base structure and append role-specific metrics (hook mutation counts, response/token/thinking metrics, error details and retryability).
Lifecycle event emission integration
internal/assistant/runtime_events.go
emitProviderResponse and emitProviderError are refactored to dispatch via runtime.dispatchLifecycle instead of observational dispatch, and both now emit lifecycle diagnostics through runtime.emitLifecycleDiagnostics with the corresponding diagnostic payloads.
Diagnostic payload test coverage
internal/assistant/provider_seam_hardening_test.go
Three integration tests verify provider response and error diagnostics: successful responses include provider/model identifiers and response/token metrics, error payloads capture retryability and error metadata (code, status, message), and hook errors are tracked in the diagnostic payload when a provider error hook fails via Lua.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • omarluq/librecode#52: Earlier "provider/tool runtime seams" work that introduced the lifecycle diagnostics framework and dispatchLifecycle wiring that this PR builds upon and extends.

Poem

🐰 A refactor most neat, diagnostics now share,
Base builders and pipelines, care everywhere,
From response to error, hooks caught in the flow,
Shared fields, shared tests—a cohesive tableau! ✨

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title 'feat: harden provider seam diagnostics' accurately summarizes the main change: adding richer diagnostics and lifecycle handling for provider interactions.
Description check ✅ Passed The description is directly related to the changeset, detailing the addition of richer diagnostics, lifecycle handler dispatch, and hook error capture.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/runtime-seam-hardening

Comment @coderabbitai help to get the list of available commands and usage tips.

@sonarqubecloud

Copy link
Copy Markdown

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 85.45455% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 62.02%. Comparing base (f7116f4) to head (ba07b4e).

Files with missing lines Patch % Lines
internal/assistant/lifecycle_diagnostics.go 82.85% 3 Missing and 3 partials ⚠️
internal/assistant/runtime_events.go 90.00% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #55      +/-   ##
==========================================
+ Coverage   61.96%   62.02%   +0.06%     
==========================================
  Files         174      174              
  Lines       17304    17348      +44     
==========================================
+ Hits        10722    10760      +38     
- Misses       5525     5528       +3     
- Partials     1057     1060       +3     
Flag Coverage Δ
unittests 62.02% <85.45%> (+0.06%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
internal/assistant/provider_seam_hardening_test.go (2)

67-70: ⚡ Quick win

Use oops.Wrapf pattern for contextual errors instead of constructing with Errorf here.

This repo’s Go guideline prefers chaining contextual metadata with Wrapf(err, "message") in oops-enabled packages; please switch this test error construction to that pattern for consistency.

As per coding guidelines, **/*.go: Use oops.In("domain").Code("code").Wrapf(err, "message") for contextual errors in packages that already use samber/oops.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/assistant/provider_seam_hardening_test.go` around lines 67 - 70,
Replace the direct Errorf construction with the oops.Wrapf pattern: create a
base error (e.g., errors.New or fmt.Errorf("service unavailable")) and then call
oops.In("assistant").Code("responses_status").Wrapf(baseErr, "service
unavailable") to attach the contextual metadata; update the symbol providerErr
to be assigned from that Wrapf chain (using the same In/Code chain).

17-125: ⚡ Quick win

Prefer a table-driven test for these provider diagnostic scenarios.

These cases cover one core behavior surface and share setup/assertion shape; a table-driven structure will reduce duplication and make future scenario additions safer.

As per coding guidelines, **/*_test.go: Prefer table-driven tests for core behavior and regression tests for terminal rendering bugs.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/assistant/provider_seam_hardening_test.go` around lines 17 - 125,
These three tests (TestRuntime_ProviderLifecycleEmitsDiagnostics,
TestRuntime_ProviderErrorDiagnosticsIncludeRetryAndErrorMetadata,
TestRuntime_ProviderErrorDiagnosticsCaptureHookErrors) exercise the same
provider-diagnostics behavior and should be combined into a single table-driven
test: create a slice of cases that configures the staticCompletionClient
(result/err), optional extension load (hook error case), expected diagnostic
topic (string(extension.LifecycleAfterProviderResponse)+"_diagnostic" or
string(extension.LifecycleProviderError)+"_diagnostic"), and expected assertions
(hook_count, provider, model, token counts, error metadata, hook_errors, etc.),
then loop cases calling newTestRuntimeWithManager,
collectRuntimeDiagnosticPayloads, runtime.Prompt and run the appropriate
assertions per case; keep helpers newTestRuntimeWithManager,
collectRuntimeDiagnosticPayloads and runtime.Prompt as-is and reference their
names in the table entries to reduce duplication and make adding scenarios
simpler.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@internal/assistant/provider_seam_hardening_test.go`:
- Around line 67-70: Replace the direct Errorf construction with the oops.Wrapf
pattern: create a base error (e.g., errors.New or fmt.Errorf("service
unavailable")) and then call
oops.In("assistant").Code("responses_status").Wrapf(baseErr, "service
unavailable") to attach the contextual metadata; update the symbol providerErr
to be assigned from that Wrapf chain (using the same In/Code chain).
- Around line 17-125: These three tests
(TestRuntime_ProviderLifecycleEmitsDiagnostics,
TestRuntime_ProviderErrorDiagnosticsIncludeRetryAndErrorMetadata,
TestRuntime_ProviderErrorDiagnosticsCaptureHookErrors) exercise the same
provider-diagnostics behavior and should be combined into a single table-driven
test: create a slice of cases that configures the staticCompletionClient
(result/err), optional extension load (hook error case), expected diagnostic
topic (string(extension.LifecycleAfterProviderResponse)+"_diagnostic" or
string(extension.LifecycleProviderError)+"_diagnostic"), and expected assertions
(hook_count, provider, model, token counts, error metadata, hook_errors, etc.),
then loop cases calling newTestRuntimeWithManager,
collectRuntimeDiagnosticPayloads, runtime.Prompt and run the appropriate
assertions per case; keep helpers newTestRuntimeWithManager,
collectRuntimeDiagnosticPayloads and runtime.Prompt as-is and reference their
names in the table entries to reduce duplication and make adding scenarios
simpler.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 6cbdb0a3-1310-4c57-9e01-f18cdf4b65a7

📥 Commits

Reviewing files that changed from the base of the PR and between f7116f4 and ba07b4e.

📒 Files selected for processing (3)
  • internal/assistant/lifecycle_diagnostics.go
  • internal/assistant/provider_seam_hardening_test.go
  • internal/assistant/runtime_events.go

@omarluq
omarluq merged commit a48e258 into main May 28, 2026
13 checks passed
@omarluq
omarluq deleted the feat/runtime-seam-hardening branch May 28, 2026 23:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants