Repository navigation
Error handling/4 telemetry and e2e - #197
Conversation
⚡ A2UI Composer PR PreviewYour automated preview is successfully live (commit
|
There was a problem hiding this comment.
Code Review
This pull request introduces a centralized diagnostic logging and error-handling framework across the A2UI composer workspace, including a new ErrorLogger service, real-time schema validation in the Monaco editor, and debounced error overlays in the Angular, React, and Lit adapters. The review feedback highlights several opportunities for improvement: deduplicating consecutive identical marker sets in the Monaco editor to prevent log flooding, cleaning up duplicate mock properties in tests, using the correct warning-level logging method in raw-frame.ts, escaping a dot in a regular expression in errors.ts, and preserving the error class name alongside the message in the ErrorLogger service.
95ef832 to
4b4d114
Compare
076fffb to
6ffe7c3
Compare
suyangw-g
left a comment
There was a problem hiding this comment.
Review Summary
Scoped note: #197 stacks on #196, so its diff against
maincontains the whole 4-PR series. This review covers the incremental delta (git diff pr-196..pr-197— 51 files, +1414/−309). My 27 comments on #196 still apply to the shared base and aren't repeated here.
The console.* → ErrorLogger migration is thorough and mechanically clean — ~30 call sites converted, withTag() applied consistently, and the accompanying spec updates are diligent. Extracting handleConsoleLog() in HostCommunication removes real duplication, the new errors.spec.ts and ga4-usage-tracking.service.spec.ts suites are well built, and the monaco-editor.spec.ts marker rewrite (real IMarker shapes via a factory instead of loose casts) is a solid improvement. It's also good to see e2e coverage added for the error surfaces I flagged as missing on #196.
I'm requesting changes because the headline feature doesn't work end-to-end, and CI is still red.
🔴 Blockers
-
CI is still failing — the watchdog fix is half-applied. This PR correctly switched
errorLogger.error→errorLogger.warnfor the unresponsive branch, but neither branch setslevel, and theIFRAME_UNRESPONSIVE_ERROR:prefix was never added. Bothraw-frame.spec.tswatchdog tests still fail. →raw-frame.ts:243,raw-frame.ts:248 -
error_categoryis never populated, so the telemetry dimension is dead on arrival.errors.ts:92is the only production caller and omits it, so GA4 receivesevent_label: 'UNKNOWN_ERROR'anderror_category: 'UNKNOWN_ERROR'for 100 % ofcomposer_errorevents. The specs pass only because they supply it by hand. →errors.ts:92 -
The dedupe signature silently drops most error telemetry.
${sourceTag}:${line}:${column}:${invalidProp}— all three coordinates areundefinedfor any error that isn't a Monaco marker (storage, startup, share, bridge…), so they all collapse to[Shell]:undefined:undefined:undefinedand only the first one per 5-second window is ever reported. The signature also excludesmessage. →errors.ts:87 -
The Errors badge now counts
infoandlogmessages. The trigger moved fromCONSOLE_LOGenvelopes toerrorLogger.errorStream$with no level filter, while this same PR converts ~15console.logcalls intologger.info. The badge will be non-zero on every cold start from routine startup chatter. →composer-workspace.ts:85 -
A new e2e test asserts nothing. The test named "captures cross-frame preview errors, renders non-crashing 350ms debounced UI error overlay, and assigns
[Previewer]log provenance" has zero assertions and ends with// Ignored to avoid brittleness. It can never fail. →json-error-handling.e2e.ts:181
⚠️ Privacy concern worth a decision before merge
ga4-usage-tracking.service.ts:316 sends params.message verbatim to Google Analytics. Composer error messages routinely embed user content — JSON fragments from the user's own layout, file paths, API error bodies. GA4's terms prohibit PII. I'd send category + coordinates only, or truncate hard and strip quoted literals.
🟠 Significant, non-blocking
- Telemetry is emitted from a UI component.
Errorsis a dockview panel — if it's never mounted, nocomposer_errorevents are sent at all. This belongs inErrorLoggeror a root-provided subscriber. - Type error in a spec:
ComposerErrorTelemetryParams.messageis required, butga4-usage-tracking.service.spec.ts:500and:525omit it. Vitest doesn't typecheck, so it passes at runtime whiletsc/lint would reject it. - Test-only scaffolding in production code:
window.a2uiCatalogManagement = thisplus a publichandshakeHistoryIndexsignal whose JSDoc says "Used by integration tests". It extends an existinga2uiHostCommunicationprecedent, but that precedent is worth containing rather than growing. waitForTimeoutis reintroduced in the very PR that removes it elsewhere —json-error-handling.e2e.ts:89(1000 ms inbeforeEach) andrenderer-integration-and-telemetry.e2e.ts:33(150 ms insidewaitForPreviewSettled).- Third copy of the same pattern: "hydrate from
getHistoryBuffer()+ dedupe viaWeakSet" now lives inevents.ts,data-model.ts, and partiallycatalog-management.ts. data-model.ts:114deep-compares withJSON.stringify, which is key-order sensitive — the repo already hasstableStringify.errors.ts:90— the 5 ssetTimeoutis never cleared on destroy.errors.ts:82—/instance\.([^ ]+) /irequires a trailing space, so a message ending in the property name never matches.
📋 Documentation
No Fixes #… link, and the CHANGELOG / documentation boxes are unchecked despite adding a new abstract method (trackComposerError) to the exported UsageTrackingService — a breaking change for any external subclass — plus a new exported interface and a breaking CatalogSchemaResolver constructor signature. The e2e checkbox is also unchecked while the PR adds a 253-line e2e suite and rewrites three others.
Blockers 2 and 3 are the ones I'd most want resolved: as written, the telemetry this PR exists to add would produce a single undifferentiated UNKNOWN_ERROR bucket with most events missing. ErrorLogItem.sourceTag already carries [Monaco] / [ChatParser] / [Previewer] / [Shell] / [Storage], so a small mapping function would give you a genuinely useful dimension.
0f7467f to
e7ae1e1
Compare
2125041 to
e9268f0
Compare
There was a problem hiding this comment.
Two blockers on the new code since my last pass, both in host-communication.ts. Everything else from my original review verified as addressed — the reporter extraction, the PII removal, the timer elimination and the e2e de-flaking all landed well.
f85350e to
7b81750
Compare
Deduplicate consecutive Monaco editor marker emissions via sorted signature checks to prevent log flooding in the Errors panel. Remove redundant mock properties in catalog schema resolver tests. Direct watchdog 15-second unresponsiveness logs to errorLogger.warn. Escape literal dot in instance property regex in ErrorLogger errors. Include error names in ErrorLogger normalizeUnknown message formatting. Update React preview date input filling and Data Model timing in E2E. TAG=agy CONV=28b993f1-8dc7-4320-8ac3-2ba70da6f758
Document the rationale for Monaco editor marker deduplication to explain why composite signatures prevent flooding the ErrorLogger during language server validation cycles. Add unit tests verifying marker change emission, duplicate suppression, and URI filtering. TAG=agy CONV=28b993f1-8dc7-4320-8ac3-2ba70da6f758
Remove explicit level properties ('error', 'warn', 'info', 'log') from
ErrorLogger.withTag methods so caller payloads let normalizeItem apply
the respective default level cleanly.
Update ScreenshotCaptureService unit tests to assert without requiring
an explicit level in logger warning calls. Add unit tests verifying
tagged logger methods delegate without passing explicit level keys.
TAG=agy
CONV=28b993f1-8dc7-4320-8ac3-2ba70da6f758
Remove .github/workflows/playwright-ci.yaml as end-to-end tests are already executed in composer_build_and_test.yml. TAG=agy CONV=28b993f1-8dc7-4320-8ac3-2ba70da6f758
…porting Increase Playwright assertion timeout for invalid JSON error snackbar to 8000ms to allow sufficient headroom for the 3000ms debounce.
Update e2e test to click the Go to line snackbar action and verify cursor navigation in Monaco editor without switching to Errors tab. TAG=agy CONV=28b993f1-8dc7-4320-8ac3-2ba70da6f758
Update Playwright selector in json-error-handling.e2e.ts to match the Errors tab action button by regex pattern, accommodating the accessible name defined by aria-label. TAG=agy CONV=28b993f1-8dc7-4320-8ac3-2ba70da6f758
Route MCP server connection, tool invocation, and bridge execution failures through ErrorLogger instead of unlogged exceptions. Provide getHistoryBuffer on HostCommunication to preserve non-destructive buffer inspection alongside consumeEnvelopeHistory.
6321ec8 to
c76c07d
Compare
Validate origin for early CONSOLE_LOG messages before iframe target registration to prevent unauthorized error injection into the Errors panel and telemetry. Also support Monaco missing-property markers in ErrorTelemetryReporter while excluding untrusted preview logs. Consolidate preview error and logging source tags to [Preview] across host communication, raw frame, error telemetry, and tests. Replace destructive consumeEnvelopeHistory with non-destructive getEnvelopeHistory across debug panels so multiple panels hydrate without clearing shared buffer state.
a305793 to
fd294e7
Compare
Description
This is the 4th of a stack of 4 PRs to improve error handling.
console.log,console.warn, etc... with usages of theErrorLoggerservicePre-launch Checklist