Repository navigation
ci: run the test suites on PRs, and fix the two tests that block them - #62
Merged
Merged
Conversation
validate-pr.yml ran only the Kiro Power tests, so most of tests/*.py, tests/skills/**/*.sh and the cli/mcp/shared Rust tests never ran before merge. A broken mcp telemetry schema test shipped this way. - Add `make test-py` (pytest over tests/) and `make test-skills-sh` (globs tests/skills/*/*.sh, minus SKILLS_SH_EXCLUDE). - Run both in CI, with test-py after synth so the tests that need build/cli/konductor run. - Run the Rust tests for cli, mcp and shared, skipping two cli tests that fail intermittently on a 4-vCPU runner. - Exclude persistent-memory/test-nonbullet-injection.sh, which writes its config to a path the validator no longer reads. - Install PyYAML alongside pytest; two tests import it.
update --cli's smoke test spawns the freshly downloaded binary with Command::output(). On Linux, fork() duplicates every file descriptor open anywhere in the process, so an unrelated Command::spawn() running concurrently elsewhere in the same process can fork a child that inherits a transient write handle on the just-written binary and hold it open until that child execs or exits. Until then, exec of the binary fails with ETXTBSY (os error 26) even though the smoke test's own write handle closed well before the exec was attempted. Retry the spawn up to 10 times with a 100ms delay between attempts (about 1 second total) when the failure is ExecutableFileBusy, giving the other process time to exec or exit and release its inherited handle. The retry loop is extracted into its own function with an injectable spawn/sleep seam, so a unit test can drive it deterministically with a fake exec that fails N times then succeeds, and a separate case that exhausts all retries, without depending on reproducing the real race.
7 of 8 tasks
docs/telemetry-schema.json requires parentAgentName on every event, but the skill-lookup MCP server's mcp_tool_call body omitted it. It's now sent as an explicit null, which also fixes the existing schema-drift test.
This was referenced Oct 16, 2026
knihit
marked this pull request as ready for review
October 16, 2026 20:05
knihit
requested review from
georgebearden,
ihmaws,
shsenior and
simonkrol
as code owners
October 16, 2026 20:05
ihmaws
approved these changes
Oct 22, 2026
simonkrol
approved these changes
Oct 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
PR CI ran only the Kiro Power tests. The rest of
tests/*.py, all oftests/skills/**/*.shand the Rust tests for cli, mcp and shared never ran before merge. That's how a broken mcp telemetry test shipped in 1.0.3 and 1.1.0. This PR runs those suites and fixes the two tests that would otherwise block every PR.CI (
Makefile,.github/workflows/validate-pr.yml)make test-py, which runspython3 -m pytest -q tests/. CI runs it aftermake synth, so the tests that needbuild/cli/konductorrun instead of skipping. The pip step now also installspyyaml, which two tests import.make test-skills-sh, which runs everytests/skills/*/*.shby glob, so new tests are picked up.SKILLS_SH_EXCLUDElists tests known to fail, each with a reason.cargo testincli/konductor-rs, thenmake -C mcp test-rustandmake -C shared test-rust. It reuses the existing toolchain step.Fix: mcp
mcp_tool_callevents missedparentAgentNamedocs/telemetry-schema.jsonrequiresparentAgentNameon every event. The skill-lookup MCP server'smcp_tool_callbody left it out, so 1.0.3 and 1.1.0 sent events without it.null. This fixesmcp_tool_call_body_satisfies_schemas_required_list.Fix:
konductor update --clicould fail with "Text file busy"cli_self_update.rsnow retries that exec up to 10 times, 100 ms apart. Unit tests drive the retry with a fake exec.real_smoke_test_succeeds_and_returns_stdouttherefore runs in CI instead of being skipped.Still excluded, each failing on
maintodaytests/skills/persistent-memory/test-nonbullet-injection.sh: 2 of 8 cases fail. The test writes its URL allowlist to a different directory from the onememory-validator.shreads (.konductor/), so the allowlist is never applied.cli::update::tests::update_finalize_index_reflects_a_concurrently_installed_other_strategy: skipped. On 4 CPUs it failed 5 of 10 full-suite runs.update_one_targetsometimes returns 64.npm test: not added. 17 tests fail becausescripts/brand-config/lib/constants.jsisn't in the repo.make guide-check: not added. It fails on a staleconfigentry inWITHDRAWN_COMMANDS.CHANGELOG.mdhas[Unreleased]/Fixedentries for both fixes.Supersedes #47 and #61, both closed.
Type of Change
cli/) change: RustTesting
make -C mcp test-rust: all pass. The schema test fails without the fix.taskset -c 0-3):cli_self_updatetests pass.update_finalize_index...skip.make test-pyandmake test-skills-shpass onmainwith the exclusions above. WithSKILLS_SH_EXCLUDE=emptied, the target fails and names the file.npm test: fails onmain, see abovecd cli && make test: covered by the clicargo testruns aboveChecklist
Notes for Reviewers
test-nonbullet-injection.shat.konductor/memory-config.json.update_finalize_index...race.scripts/brand-config/lib/constants.jsfornpm test.configentry formake guide-check.