feat(ai): PR 6 — test output + CLI [6/7] - #420
ianwhitedeveloper merged 4 commits into
Conversation
- Add source/test-output.js: formatTAP, recordTestOutput, openInBrowser with open package dependency - Add source/ai-command.js: parseAIArgs, runAICommand, formatAssertionReport; remove debug/debugLog params, use z.prettifyError instead of formatZodError - Update bin/riteway.js: add riteway ai <file> subcommand with exhaustive handleAIErrors (all 12 error types) - Add tests for all new modules (42 new tests) - Fix tap-yaml.js TS: JSDoc cast on reduce initial value - Remove formatMedia dead code (WIP #4/#5) - Remove generateLogFilePath (debug-logger removed in PR 5) Made-with: Cursor
- Remove async from generateOutputPath (no awaits)
- Drop dead .sudo from extension regex; add comment
- Rename ANSI const to ansi (naming convention)
- Update runAICommand docblock: describe side effects
- Add --concurrency to help text usage synopsis
- Add vi.mock for ai-runner/agent-config/test-output
- Add runAICommand orchestration tests: success path,
auth failure, OutputError wrapping
- Add openInBrowser unit tests (calls open, swallows err)
- Add recordTestOutput openBrowser: true coverage
- Remove outer describe('test-output') wrapper (noise)
- Tighten recordTestOutput filename pattern assertion
- Add --help stdout content assertions to bin tests
- Rename subprocess test to reflect actual failure path
Made-with: Cursor
|
@cursor please /review |
This comment was marked as resolved.
This comment was marked as resolved.
- test(ai-command): collapse partial cause assertions to full expected objects (ValidationError, SecurityError) per Jan's convention — tests application error contract, not library API - test(ai-command): upgrade AITestError, ValidationError, OutputError orchestration assertions to full objects (passRate, threshold, cause included where set by application) - test(ai-command): add agentConfigPath orchestration test verifying loadAgentConfig result is forwarded to runAITests - test(ai-command): add unexpected-error fallthrough test covering the catch block's AITestError wrapping of unstructured errors - test(ai-command): use mock.lastCall instead of mock.calls[0] to avoid stale call history across tests (no clearMocks in config) - test(ai-command): add loadAgentConfig to import for agentConfigPath test - test(test-output): strengthen openBrowser assertion to verify path and options passed to open(), not just call count - test(test-output): add .txt extension test for generateOutputPath (documented in source comment, previously untested) - fix(ai-command): add concurrency to configuration log line so users can confirm --concurrency flag was applied Made-with: Cursor
Remove describe('defaults') block which tested constant values
rather than behavior (ericelliott/janhesters pattern from PR #407).
Replace tautological defaults.X references in parseAIArgs defaults
test with literal values so the test actually verifies the defaults
are correct rather than just asserting X equals X.
Made-with: Cursor
|
@cursor please /review - focus remediation suggestions only on this PR's code changes. 👏 Do a good job and you will be rewarded. This is important. 🙏🏼 |
|
❌ Unable to add follow-up after multiple attempts. The agent may be experiencing issues. Please try again later or Open in Web to view the status. Learn more about Cursor Agents |
|
@cursoragent please /review - focus remediation suggestions only on this PR's code changes. 👏 Do a good job and you will be rewarded. This is important. 🙏🏼 |
|
Looking at the specific code changes in this PR, I'll provide focused remediation suggestions. 🔬 Focused Code Review: PR 6 ChangesAfter careful analysis of the actual code changes in this PR, I have good news: ✅ Code Quality AssessmentThe code in this PR is production-ready with no significant issues requiring remediation. Here's what I verified:
📝 One Minor Documentation EnhancementThe only improvement I'd suggest (completely optional):
// Type cast ensures correct return type for TypeScript consumers
return match[1]
.split('\n')
.map(line => line.match(/^(\w+):\s*(.+)$/))
.filter(Boolean)
.reduce((acc, [, key, rawValue]) => ({ ...acc, ...parseField(key, rawValue) }), /** @type {{ passed: boolean, actual: string, expected: string, score: number }} */({}));🎯 Recommendation✅ APPROVE - This PR is ready to merge as-is. The code is well-crafted, thoroughly tested, and follows all project standards. The optional documentation enhancement above can be addressed in a future PR if desired, but is not necessary for merge. Great work on this implementation! 👏 |
0efdd3b
into
ai-testing-framework-implementation-consolidation


Summary
Adds TAP output formatting, file recording, browser opening, and the
riteway ai <file>CLI subcommand.Files changed
source/test-output.js+test-output.test.js— TAP formatting, output recording, browser openingsource/ai-command.js+ai-command.test.js— CLI arg parsing, validation, orchestrationbin/riteway.js—riteway ai <file>subcommand with exhaustive error handlingbin/riteway.test.js— subprocess tests for AI CLI validationsource/tap-yaml.js— TypeScript compatibility fixpackage.json— addedopendependencyTest plan
riteway ai <file>CLI behaviour end-to-endMade with Cursor