You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
serve: exit nonzero when the backend server dies in --foreground mode #73
Problem
On ROCm tank, rocm serve ornith --foreground (user unit rocm-serve-ornith.service, Restart=on-failure) kept running after its backend died. On Mon 10/19 at about 6:34 PM PT the journal logged Error: Lemonade packaged llama-server exited with status signal: 9 (SIGKILL), but the rocm process (and its rocm daemon child) stayed up with nothing listening on port 11435. systemd saw the unit as active, never restarted it, and the factory's triage model was down for about 2.7 days.
Expected
In --foreground mode, when the backend server exits for any reason (signal or nonzero status), rocm serve exits nonzero after cleaning up its children, so the service manager's restart policy applies.
Acceptance criteria
A test (or scripted repro) kills the backend with SIGKILL and asserts rocm serve --foreground exits nonzero within a few seconds and leaves no orphaned backend or daemon process.
A clean backend shutdown still exits 0 when the user stopped it (SIGTERM/SIGINT to rocm serve).
The exit is logged with the backend's exit status/signal.
Optional: --restart-backend N to retry in-process before giving up (off by default).
Context: mikeroySoft/factory triage outage; companion factory tickets cover health checks and alerting.
Triage: The issue has a clear problem statement (rocm serve --foreground keeps running after the backend dies, defeating systemd's Restart=on-failure), a precise expected behavior, and multiple observable acceptance criteria (nonzero exit on backend death, exit 0 on clean user-initiated shutdown, logging of backend exit status/signal, no orphaned processes). The optional --restart-backend item is explicitly marked optional. It is a contained bug fix within this repository that does not touch release, signing, or security policy, so it does not require human judgment.
Agent brief: Make rocm serve --foreground exit nonzero when the backend server dies for any reason (signal or nonzero status), after cleaning up its child processes (backend + daemon), and log the backend's exit status/signal. It must still exit 0 on a clean user-initiated shutdown (SIGTERM/SIGINT sent to rocm serve). Add a test or scripted repro that SIGKILLs the backend and asserts rocm serve --foreground exits nonzero within a few seconds and leaves no orphaned backend or daemon process, plus a clean-shutdown case asserting exit 0. Optional (off by default): add --restart-backend N to retry the backend in-process before giving up. Verify with: start rocm serve <service> --foreground, then kill -9 <backend-pid> and assert the parent process exits nonzero and no orphan processes remain; separately send SIGTERM/SIGINT to rocm serve and assert exit 0.
rocm serve --foreground now retains and reaps the managed supervisor child, detects its real exit status, logs an unexpected exit, cleans up the managed process tree and any daemon started by that invocation, and returns nonzero.
The vLLM supervisor now writes its backend child's ExitStatus before returning nonzero, so signal deaths such as SIGKILL are present in the engine log instead of being collapsed to exit code 1.
SIGINT/SIGTERM sent to an attached serve now follows the explicit user-stop path, performs the same cleanup, and returns zero. Ctrl-D detach behavior remains unchanged.
Added Linux/GPU-independent Cucumber scenarios serve-27 and serve-28. They use a real shell process tree to verify backend SIGKILL failure, signal logging, clean user SIGTERM, bounded exit time, and absence of backend/supervisor/daemon orphans.
Updated rocm serve help, README, testing docs, and manual-testing instructions.
Fixed gate blockers encountered while completing the required full gate: PATH serialization for TheRock archive tests, a stale-port ComfyUI test race, non-UTF-8 system-tar diagnostics escaping Rust test capture, and an ETXTBSY race in the system SDK fake enumerator.
Eight signed-off commits are on agent/73; .factory-prompt.md remains untracked and uncommitted as required.
cargo test -p rocm-engine-vllm: 108 tests passed across 3 suites.
cargo test -p rocm-core uv_archive_extraction_never_writes_outside_the_staging_dir: passed; captured output is valid UTF-8.
cargo test -p rocm-core --lib system_sdk::tests::probe_collects_deduplicated_gfx_targets_dropping_gfx000: passed.
/home/mike/.local/share/uv/tools/factory/bin/python -m factory gate --report .factory/gate-report-73.md: PASS for conflict markers, clippy, tests, smoke, and leak scan.
Still unverified
The host has no ROCm/amdgpu runtime, so live Lemonade and vLLM backend SIGKILLs on GPU hardware were not exercised. The committed E2E fixture covers the same parent/supervisor/backend/daemon lifecycle with real Unix processes.
Next
Dispatcher may push agent/73 and open the PR. CI or a ROCm host should run the live manual SIGKILL reproduction if hardware validation is required.
rocm serve --foreground now retains and reaps the managed supervisor child, detects its real exit status, logs an unexpected exit, cleans up the managed process tree and any daemon started by that invocation, and returns nonzero.
The vLLM supervisor now writes its backend child's ExitStatus before returning nonzero, so signal deaths such as SIGKILL are present in the engine log instead of being collapsed to exit code 1.
SIGINT/SIGTERM sent to an attached serve now follows the explicit user-stop path, performs the same cleanup, and returns zero. Ctrl-D detach behavior remains unchanged.
Added Linux/GPU-independent Cucumber scenarios serve-27 and serve-28. They use a real shell process tree to verify backend SIGKILL failure, signal logging, clean user SIGTERM, bounded exit time, and absence of backend/supervisor/daemon orphans.
Updated rocm serve help, README, testing docs, and manual-testing instructions.
Fixed gate blockers encountered while completing the required full gate: PATH serialization for TheRock archive tests, a stale-port ComfyUI test race, non-UTF-8 system-tar diagnostics escaping Rust test capture, and an ETXTBSY race in the system SDK fake enumerator.
Eight signed-off commits are on agent/73; .factory-prompt.md remains untracked and uncommitted as required.
cargo test -p rocm-engine-vllm: 108 tests passed across 3 suites.
cargo test -p rocm-core uv_archive_extraction_never_writes_outside_the_staging_dir: passed; captured output is valid UTF-8.
cargo test -p rocm-core --lib system_sdk::tests::probe_collects_deduplicated_gfx_targets_dropping_gfx000: passed.
/home/mike/.local/share/uv/tools/factory/bin/python -m factory gate --report .factory/gate-report-73.md: PASS for conflict markers, clippy, tests, smoke, and leak scan.
Still unverified
The host has no ROCm/amdgpu runtime, so live Lemonade and vLLM backend SIGKILLs on GPU hardware were not exercised. The committed E2E fixture covers the same parent/supervisor/backend/daemon lifecycle with real Unix processes.
Next
Dispatcher may push agent/73 and open the PR. CI or a ROCm host should run the live manual SIGKILL reproduction if hardware validation is required.
Factory handoff request 73/2: a human decision is needed on #73.
No decision owner could be determined (unassigned); a maintainer should claim it.
Question
How should #73 proceed? It escalated with: PR #74: CI failed (Test (affected crates), build-and-test, windows-build-and-test, Coverage (rocm-dash crates, ratcheted)); factory-approved label removed. Automatic recovery stopped because the manager's 1 allowed round(s) are exhausted.
Evidence (bounded links; worker logs and the escalation packet stay on the runner)
Proposed next step
Read the evidence, then amend the ticket and relabel ready-for-agent for another attempt, fix the branch by hand, request changes on the PR, or close it. Replying here is context for humans; nothing here retries, approves or merges.
Routing
decision_owner: absent — ticket body has no **Decision owner** section
initiative: skipped — initiative owner only routes requirements, not ci
configuration: absent — no [collaboration] section in .factory.toml; legacy behaviour unchanged
To claim or reassign this decision, set a **Decision owner** section (a GitHub login) in the issue body; the next pass reflects it and never overwrites it.
rocm serve --foreground now retains and reaps the managed supervisor child, detects its real exit status, logs an unexpected exit, cleans up the managed process tree and any daemon started by that invocation, and returns nonzero.
The vLLM supervisor now writes its backend child's ExitStatus before returning nonzero, so signal deaths such as SIGKILL are present in the engine log instead of being collapsed to exit code 1.
SIGINT/SIGTERM sent to an attached serve now follows the explicit user-stop path, performs the same cleanup, and returns zero. Ctrl-D detach behavior remains unchanged.
Added Linux/GPU-independent Cucumber scenarios serve-27 and serve-28. They use a real shell process tree to verify backend SIGKILL failure, signal logging, clean user SIGTERM, bounded exit time, and absence of backend/supervisor/daemon orphans.
Updated rocm serve help, README, testing docs, and manual-testing instructions.
Fixed gate blockers encountered while completing the required full gate: PATH serialization for TheRock archive tests, a stale-port ComfyUI test race, non-UTF-8 system-tar diagnostics escaping Rust test capture, and an ETXTBSY race in the system SDK fake enumerator.
Eight signed-off commits are on agent/73; .factory-prompt.md remains untracked and uncommitted as required.
cargo test -p rocm-engine-vllm: 108 tests passed across 3 suites.
cargo test -p rocm-core uv_archive_extraction_never_writes_outside_the_staging_dir: passed; captured output is valid UTF-8.
cargo test -p rocm-core --lib system_sdk::tests::probe_collects_deduplicated_gfx_targets_dropping_gfx000: passed.
/home/mike/.local/share/uv/tools/factory/bin/python -m factory gate --report .factory/gate-report-73.md: PASS for conflict markers, clippy, tests, smoke, and leak scan.
Still unverified
The host has no ROCm/amdgpu runtime, so live Lemonade and vLLM backend SIGKILLs on GPU hardware were not exercised. The committed E2E fixture covers the same parent/supervisor/backend/daemon lifecycle with real Unix processes.
Next
Dispatcher may push agent/73 and open the PR. CI or a ROCm host should run the live manual SIGKILL reproduction if hardware validation is required.
rocm serve --foreground now retains and reaps the managed supervisor child, detects its real exit status, logs an unexpected exit, cleans up the managed process tree and any daemon started by that invocation, and returns nonzero.
The vLLM supervisor now writes its backend child's ExitStatus before returning nonzero, so signal deaths such as SIGKILL are present in the engine log instead of being collapsed to exit code 1.
SIGINT/SIGTERM sent to an attached serve now follows the explicit user-stop path, performs the same cleanup, and returns zero. Ctrl-D detach behavior remains unchanged.
Added Linux/GPU-independent Cucumber scenarios serve-27 and serve-28. They use a real shell process tree to verify backend SIGKILL failure, signal logging, clean user SIGTERM, bounded exit time, and absence of backend/supervisor/daemon orphans.
Updated rocm serve help, README, testing docs, and manual-testing instructions.
Fixed gate blockers encountered while completing the required full gate: PATH serialization for TheRock archive tests, a stale-port ComfyUI test race, non-UTF-8 system-tar diagnostics escaping Rust test capture, and an ETXTBSY race in the system SDK fake enumerator.
Eight signed-off commits are on agent/73; .factory-prompt.md remains untracked and uncommitted as required.
cargo test -p rocm-engine-vllm: 108 tests passed across 3 suites.
cargo test -p rocm-core uv_archive_extraction_never_writes_outside_the_staging_dir: passed; captured output is valid UTF-8.
cargo test -p rocm-core --lib system_sdk::tests::probe_collects_deduplicated_gfx_targets_dropping_gfx000: passed.
/home/mike/.local/share/uv/tools/factory/bin/python -m factory gate --report .factory/gate-report-73.md: PASS for conflict markers, clippy, tests, smoke, and leak scan.
Still unverified
The host has no ROCm/amdgpu runtime, so live Lemonade and vLLM backend SIGKILLs on GPU hardware were not exercised. The committed E2E fixture covers the same parent/supervisor/backend/daemon lifecycle with real Unix processes.
Next
Dispatcher may push agent/73 and open the PR. CI or a ROCm host should run the live manual SIGKILL reproduction if hardware validation is required.
Factory handoff request 73/4: a human decision is needed on #73.
No decision owner could be determined (unassigned); a maintainer should claim it.
Question
How should #73 proceed? It escalated with: review requested changes after manager FIX. Automatic recovery stopped because the manager's 3 allowed round(s) are exhausted.
Evidence (bounded links; worker logs and the escalation packet stay on the runner)
Proposed next step
Read the evidence, then amend the ticket and relabel ready-for-agent for another attempt, fix the branch by hand, request changes on the PR, or close it. Replying here is context for humans; nothing here retries, approves or merges.
Routing
decision_owner: absent — ticket body has no **Decision owner** section
initiative: skipped — initiative owner only routes requirements, not implementation
configuration: absent — no [collaboration] section in .factory.toml; legacy behaviour unchanged
To claim or reassign this decision, set a **Decision owner** section (a GitHub login) in the issue body; the next pass reflects it and never overwrites it.
rocm serve --foreground retains and reaps its managed supervisor, treats an unexpected backend/supervisor exit as an error, logs the exit status or signal, and cleans up its managed process tree and daemon.
SIGINT/SIGTERM sent to an attached serve follows the explicit user-stop path, cleans up, and exits zero; Ctrl-D detach remains unchanged.
The vLLM supervisor logs its backend child's ExitStatus before returning an error.
Linux/GPU-independent Cucumber scenarios serve-27 and serve-28 exercise real backend/supervisor/daemon processes for SIGKILL failure and clean SIGTERM shutdown.
Preserved the current-base serve-23 and dash-35 coverage while removing stale-base changes unrelated to issue 73.
Made the ComfyUI stale-state test deterministic after the required full gate exposed its released-port race: port 0 is always unreachable and tests the intended stopped-state branch without racing another listener.
Nine signed-off commits are on agent/73; .factory-prompt.md, the gate report, and this handoff remain uncommitted.
cargo test -p rocm --bin rocm comfyui::tests::status_reports_stopped_when_saved_comfyui_pid_is_gone: passed.
/home/mike/.local/share/uv/tools/factory/bin/python -m factory gate --report .factory/gate-report-73.md: PASS for conflict markers, clippy, tests, smoke, and leak scan.
Still unverified
This host has no ROCm/amdgpu runtime, so live Lemonade and vLLM backend SIGKILL behavior was not exercised on GPU hardware.
serve-27 proves the final log drain and process lifecycle with a real Unix process tree, but its fixture emits the signal: 9 (SIGKILL) line itself; it does not independently validate Lemonade's production signal-formatting text.
Factory handoff request 73/5: a human decision is needed on #73.
No decision owner could be determined (unassigned); a maintainer should claim it.
Question
How should #73 proceed? It escalated with: REVISE verdict after 3 review round(s). Automatic recovery stopped because no manager is configured, so nothing recovers automatically.
Evidence (bounded links; worker logs and the escalation packet stay on the runner)
Proposed next step
Read the evidence, then amend the ticket and relabel ready-for-agent for another attempt, fix the branch by hand, request changes on the PR, or close it. Replying here is context for humans; nothing here retries, approves or merges.
Routing
decision_owner: absent — ticket body has no **Decision owner** section
initiative: skipped — initiative owner only routes requirements, not implementation
configuration: absent — no [collaboration] section in .factory.toml; legacy behaviour unchanged
To claim or reassign this decision, set a **Decision owner** section (a GitHub login) in the issue body; the next pass reflects it and never overwrites it.
rocm serve --foreground retains and reaps its managed supervisor, treats an unexpected backend/supervisor exit as an error, logs the exit status or signal, and cleans up its managed process tree and daemon.
SIGINT/SIGTERM sent to an attached serve follows the explicit user-stop path, cleans up, and exits zero; Ctrl-D detach remains unchanged.
The vLLM supervisor logs its backend child's ExitStatus before returning an error.
Linux/GPU-independent Cucumber scenarios serve-27 and serve-28 exercise real backend/supervisor/daemon processes for SIGKILL failure and clean SIGTERM shutdown.
Foreground cleanup now preserves a daemon started by that invocation when another managed service is still running or automations are enabled. Two Linux process-lifecycle regression tests cover both shared-daemon cases.
/home/mike/.local/share/uv/tools/factory/bin/python -m factory gate --report .factory/gate-report-73.md: PASS for conflict markers, clippy, tests, smoke, and leak scan.
Still unverified
The gate reports ROCm and amdgpu unavailable, so live Lemonade and vLLM backend SIGKILL behavior was not exercised on GPU hardware.
serve-27 proves the final log drain and process lifecycle with a real Unix process tree, but its fixture emits the signal: 9 (SIGKILL) line itself; it does not independently validate Lemonade's production signal-formatting text.
Factory handoff request 73/6: a human decision is needed on #73.
No decision owner could be determined (unassigned); a maintainer should claim it.
Question
How should #73 proceed? It escalated with: PR #74: gate failed after rebase onto moved main; factory-approved label removed. Automatic recovery stopped because the manager's 3 allowed round(s) are exhausted.
Evidence (bounded links; worker logs and the escalation packet stay on the runner)
Proposed next step
Read the evidence, then amend the ticket and relabel ready-for-agent for another attempt, fix the branch by hand, request changes on the PR, or close it. Replying here is context for humans; nothing here retries, approves or merges.
Routing
decision_owner: absent — ticket body has no **Decision owner** section
initiative: skipped — initiative owner only routes requirements, not implementation
configuration: absent — no [collaboration] section in .factory.toml; legacy behaviour unchanged
To claim or reassign this decision, set a **Decision owner** section (a GitHub login) in the issue body; the next pass reflects it and never overwrites it.
Activity
mikeroysoft-agent commentedon Oct 22, 2026
Triage: The issue has a clear problem statement (rocm serve --foreground keeps running after the backend dies, defeating systemd's Restart=on-failure), a precise expected behavior, and multiple observable acceptance criteria (nonzero exit on backend death, exit 0 on clean user-initiated shutdown, logging of backend exit status/signal, no orphaned processes). The optional --restart-backend item is explicitly marked optional. It is a contained bug fix within this repository that does not touch release, signing, or security policy, so it does not require human judgment.
Agent brief: Make
rocm serve --foregroundexit nonzero when the backend server dies for any reason (signal or nonzero status), after cleaning up its child processes (backend + daemon), and log the backend's exit status/signal. It must still exit 0 on a clean user-initiated shutdown (SIGTERM/SIGINT sent torocm serve). Add a test or scripted repro that SIGKILLs the backend and assertsrocm serve --foregroundexits nonzero within a few seconds and leaves no orphaned backend or daemon process, plus a clean-shutdown case asserting exit 0. Optional (off by default): add--restart-backend Nto retry the backend in-process before giving up. Verify with: startrocm serve <service> --foreground, thenkill -9 <backend-pid>and assert the parent process exits nonzero and no orphan processes remain; separately send SIGTERM/SIGINT torocm serveand assert exit 0.mikeroysoft-agent commentedon Oct 22, 2026
Factory dispatcher escalating: PR #74: CI failed (Third-party notices current, clippy);
factory-approvedlabel removed.Escalation packet:
/home/mike/.local/share/district/roots/rocm-cli/.factory/escalations/73.mdWorker handoff notes:
Issue 73 handoff
Changed
rocm serve --foregroundnow retains and reaps the managed supervisor child, detects its real exit status, logs an unexpected exit, cleans up the managed process tree and any daemon started by that invocation, and returns nonzero.ExitStatusbefore returning nonzero, so signal deaths such as SIGKILL are present in the engine log instead of being collapsed to exit code 1.serve-27andserve-28. They use a real shell process tree to verify backend SIGKILL failure, signal logging, clean user SIGTERM, bounded exit time, and absence of backend/supervisor/daemon orphans.rocm servehelp, README, testing docs, and manual-testing instructions.ETXTBSYrace in the system SDK fake enumerator.agent/73;.factory-prompt.mdremains untracked and uncommitted as required.Verified
cargo xtask e2e -- -n 'serve-(27|28)': 2 scenarios, 7 steps passed.cargo test -p rocm-engine-vllm: 108 tests passed across 3 suites.cargo test -p rocm-core uv_archive_extraction_never_writes_outside_the_staging_dir: passed; captured output is valid UTF-8.cargo test -p rocm-core --lib system_sdk::tests::probe_collects_deduplicated_gfx_targets_dropping_gfx000: passed./home/mike/.local/share/uv/tools/factory/bin/python -m factory gate --report .factory/gate-report-73.md: PASS for conflict markers, clippy, tests, smoke, and leak scan.Still unverified
Next
agent/73and open the PR. CI or a ROCm host should run the live manual SIGKILL reproduction if hardware validation is required.mikeroysoft-agent commentedon Oct 22, 2026
Factory dispatcher escalating: PR #74: CI failed (Test (affected crates), build-and-test, windows-build-and-test, Coverage (rocm-dash crates, ratcheted));
factory-approvedlabel removed.Escalation packet:
/home/mike/.local/share/district/roots/rocm-cli/.factory/escalations/73.mdWorker handoff notes:
Issue 73 handoff
Changed
rocm serve --foregroundnow retains and reaps the managed supervisor child, detects its real exit status, logs an unexpected exit, cleans up the managed process tree and any daemon started by that invocation, and returns nonzero.ExitStatusbefore returning nonzero, so signal deaths such as SIGKILL are present in the engine log instead of being collapsed to exit code 1.serve-27andserve-28. They use a real shell process tree to verify backend SIGKILL failure, signal logging, clean user SIGTERM, bounded exit time, and absence of backend/supervisor/daemon orphans.rocm servehelp, README, testing docs, and manual-testing instructions.ETXTBSYrace in the system SDK fake enumerator.agent/73;.factory-prompt.mdremains untracked and uncommitted as required.Verified
cargo xtask e2e -- -n 'serve-(27|28)': 2 scenarios, 7 steps passed.cargo test -p rocm-engine-vllm: 108 tests passed across 3 suites.cargo test -p rocm-core uv_archive_extraction_never_writes_outside_the_staging_dir: passed; captured output is valid UTF-8.cargo test -p rocm-core --lib system_sdk::tests::probe_collects_deduplicated_gfx_targets_dropping_gfx000: passed./home/mike/.local/share/uv/tools/factory/bin/python -m factory gate --report .factory/gate-report-73.md: PASS for conflict markers, clippy, tests, smoke, and leak scan.Still unverified
Next
agent/73and open the PR. CI or a ROCm host should run the live manual SIGKILL reproduction if hardware validation is required.mikeroysoft-agent commentedon Oct 22, 2026
Factory handoff request
73/2: a human decision is needed on #73.No decision owner could be determined (unassigned); a maintainer should claim it.
Question
How should #73 proceed? It escalated with: PR #74: CI failed (Test (affected crates), build-and-test, windows-build-and-test, Coverage (rocm-dash crates, ratcheted));
factory-approvedlabel removed. Automatic recovery stopped because the manager's 1 allowed round(s) are exhausted.Evidence (bounded links; worker logs and the escalation packet stay on the runner)
Proposed next step
Read the evidence, then amend the ticket and relabel
ready-for-agentfor another attempt, fix the branch by hand, request changes on the PR, or close it. Replying here is context for humans; nothing here retries, approves or merges.Routing
**Decision owner**sectionTo claim or reassign this decision, set a
**Decision owner**section (a GitHub login) in the issue body; the next pass reflects it and never overwrites it.mikeroysoft-agent commentedon Oct 23, 2026
Factory dispatcher escalating: review requested changes after manager FIX.
Escalation packet:
/home/mike/.local/share/district/roots/rocm-cli/.factory/escalations/73.mdWorker logs:
/home/mike/.local/share/district/roots/rocm-cli/.factory/logs/73-attempt-6.logWorker handoff notes:
Issue 73 handoff
Changed
rocm serve --foregroundnow retains and reaps the managed supervisor child, detects its real exit status, logs an unexpected exit, cleans up the managed process tree and any daemon started by that invocation, and returns nonzero.ExitStatusbefore returning nonzero, so signal deaths such as SIGKILL are present in the engine log instead of being collapsed to exit code 1.serve-27andserve-28. They use a real shell process tree to verify backend SIGKILL failure, signal logging, clean user SIGTERM, bounded exit time, and absence of backend/supervisor/daemon orphans.rocm servehelp, README, testing docs, and manual-testing instructions.ETXTBSYrace in the system SDK fake enumerator.agent/73;.factory-prompt.mdremains untracked and uncommitted as required.Verified
cargo xtask e2e -- -n 'serve-(27|28)': 2 scenarios, 7 steps passed.cargo test -p rocm-engine-vllm: 108 tests passed across 3 suites.cargo test -p rocm-core uv_archive_extraction_never_writes_outside_the_staging_dir: passed; captured output is valid UTF-8.cargo test -p rocm-core --lib system_sdk::tests::probe_collects_deduplicated_gfx_targets_dropping_gfx000: passed./home/mike/.local/share/uv/tools/factory/bin/python -m factory gate --report .factory/gate-report-73.md: PASS for conflict markers, clippy, tests, smoke, and leak scan.Still unverified
Next
agent/73and open the PR. CI or a ROCm host should run the live manual SIGKILL reproduction if hardware validation is required.mikeroysoft-agent commentedon Oct 23, 2026
Factory dispatcher escalating: review requested changes after manager FIX.
Escalation packet:
/home/mike/.local/share/district/roots/rocm-cli/.factory/escalations/73.mdWorker logs:
/home/mike/.local/share/district/roots/rocm-cli/.factory/logs/73-attempt-7.logWorker handoff notes:
Issue 73 handoff
Changed
rocm serve --foregroundnow retains and reaps the managed supervisor child, detects its real exit status, logs an unexpected exit, cleans up the managed process tree and any daemon started by that invocation, and returns nonzero.ExitStatusbefore returning nonzero, so signal deaths such as SIGKILL are present in the engine log instead of being collapsed to exit code 1.serve-27andserve-28. They use a real shell process tree to verify backend SIGKILL failure, signal logging, clean user SIGTERM, bounded exit time, and absence of backend/supervisor/daemon orphans.rocm servehelp, README, testing docs, and manual-testing instructions.ETXTBSYrace in the system SDK fake enumerator.agent/73;.factory-prompt.mdremains untracked and uncommitted as required.Verified
cargo xtask e2e -- -n 'serve-(27|28)': 2 scenarios, 7 steps passed.cargo test -p rocm-engine-vllm: 108 tests passed across 3 suites.cargo test -p rocm-core uv_archive_extraction_never_writes_outside_the_staging_dir: passed; captured output is valid UTF-8.cargo test -p rocm-core --lib system_sdk::tests::probe_collects_deduplicated_gfx_targets_dropping_gfx000: passed./home/mike/.local/share/uv/tools/factory/bin/python -m factory gate --report .factory/gate-report-73.md: PASS for conflict markers, clippy, tests, smoke, and leak scan.Still unverified
Next
agent/73and open the PR. CI or a ROCm host should run the live manual SIGKILL reproduction if hardware validation is required.mikeroysoft-agent commentedon Oct 24, 2026
Factory handoff request
73/4: a human decision is needed on #73.No decision owner could be determined (unassigned); a maintainer should claim it.
Question
How should #73 proceed? It escalated with: review requested changes after manager FIX. Automatic recovery stopped because the manager's 3 allowed round(s) are exhausted.
Evidence (bounded links; worker logs and the escalation packet stay on the runner)
Proposed next step
Read the evidence, then amend the ticket and relabel
ready-for-agentfor another attempt, fix the branch by hand, request changes on the PR, or close it. Replying here is context for humans; nothing here retries, approves or merges.Routing
**Decision owner**sectionTo claim or reassign this decision, set a
**Decision owner**section (a GitHub login) in the issue body; the next pass reflects it and never overwrites it.1 remaining item
mikeroysoft-agent commentedon Oct 24, 2026
Factory dispatcher escalating: REVISE verdict after 3 review round(s).
Escalation packet:
/home/mike/.local/share/district/roots/rocm-cli/.factory/escalations/73.mdWorker logs:
/home/mike/.local/share/district/roots/rocm-cli/.factory/logs/73-attempt-6.logWorker handoff notes:
Issue 73 handoff
Changed
rocm serve --foregroundretains and reaps its managed supervisor, treats an unexpected backend/supervisor exit as an error, logs the exit status or signal, and cleans up its managed process tree and daemon.ExitStatusbefore returning an error.serve-27andserve-28exercise real backend/supervisor/daemon processes for SIGKILL failure and clean SIGTERM shutdown.rocm servehelp, README, testing documentation, and manual verification instructions.serve-23anddash-35coverage while removing stale-base changes unrelated to issue 73.agent/73;.factory-prompt.md, the gate report, and this handoff remain uncommitted.Verified
cargo xtask e2e -- -n 'serve-(23|27|28)|dash-35': 4 scenarios, 15 steps passed.cargo test -p rocm --bin rocm comfyui::tests::status_reports_stopped_when_saved_comfyui_pid_is_gone: passed./home/mike/.local/share/uv/tools/factory/bin/python -m factory gate --report .factory/gate-report-73.md: PASS for conflict markers, clippy, tests, smoke, and leak scan.Still unverified
serve-27proves the final log drain and process lifecycle with a real Unix process tree, but its fixture emits thesignal: 9 (SIGKILL)line itself; it does not independently validate Lemonade's production signal-formatting text.review_exit:1and published no code finding; no source change can address that reviewer-process failure.Next
agent/73and refresh PR agent/73: serve: exit nonzero when the backend server dies in --foreground mode #74. CI or a ROCm host should run the documented live backend SIGKILL reproduction if hardware validation is required.mikeroysoft-agent commentedon Oct 24, 2026
Factory handoff request
73/5: a human decision is needed on #73.No decision owner could be determined (unassigned); a maintainer should claim it.
Question
How should #73 proceed? It escalated with: REVISE verdict after 3 review round(s). Automatic recovery stopped because no manager is configured, so nothing recovers automatically.
Evidence (bounded links; worker logs and the escalation packet stay on the runner)
Proposed next step
Read the evidence, then amend the ticket and relabel
ready-for-agentfor another attempt, fix the branch by hand, request changes on the PR, or close it. Replying here is context for humans; nothing here retries, approves or merges.Routing
**Decision owner**sectionTo claim or reassign this decision, set a
**Decision owner**section (a GitHub login) in the issue body; the next pass reflects it and never overwrites it.mikeroysoft-agent commentedon Oct 24, 2026
Factory dispatcher escalating: PR #74: gate failed after rebase onto moved main;
factory-approvedlabel removed.Escalation packet:
/home/mike/.local/share/district/roots/rocm-cli/.factory/escalations/73.mdWorker handoff notes:
Issue 73 handoff
Changed
rocm serve --foregroundretains and reaps its managed supervisor, treats an unexpected backend/supervisor exit as an error, logs the exit status or signal, and cleans up its managed process tree and daemon.ExitStatusbefore returning an error.serve-27andserve-28exercise real backend/supervisor/daemon processes for SIGKILL failure and clean SIGTERM shutdown.rocm servehelp, README, testing documentation, and manual verification instructions.agent/73;.factory-prompt.md, the gate report, and this handoff remain uncommitted.Verified
cargo test -p rocm --bin rocm cleanup_preserves_its_daemon -- --nocapture: 2 passed.cargo test -p rocm --bin rocm service_launch::tests::attached_lifecycle: 9 passed.cargo fmt --all -- --check: passed.cargo xtask e2e -- -n 'serve-(27|28)': 2 scenarios, 7 steps passed./home/mike/.local/share/uv/tools/factory/bin/python -m factory gate --report .factory/gate-report-73.md: PASS for conflict markers, clippy, tests, smoke, and leak scan.Still unverified
serve-27proves the final log drain and process lifecycle with a real Unix process tree, but its fixture emits thesignal: 9 (SIGKILL)line itself; it does not independently validate Lemonade's production signal-formatting text.Next
agent/73and refresh PR agent/73: serve: exit nonzero when the backend server dies in --foreground mode #74. CI or a ROCm host should run the documented live backend SIGKILL reproduction if hardware validation is required.mikeroysoft-agent commentedon Oct 24, 2026
Factory handoff request
73/6: a human decision is needed on #73.No decision owner could be determined (unassigned); a maintainer should claim it.
Question
How should #73 proceed? It escalated with: PR #74: gate failed after rebase onto moved main;
factory-approvedlabel removed. Automatic recovery stopped because the manager's 3 allowed round(s) are exhausted.Evidence (bounded links; worker logs and the escalation packet stay on the runner)
Proposed next step
Read the evidence, then amend the ticket and relabel
ready-for-agentfor another attempt, fix the branch by hand, request changes on the PR, or close it. Replying here is context for humans; nothing here retries, approves or merges.Routing
**Decision owner**sectionTo claim or reassign this decision, set a
**Decision owner**section (a GitHub login) in the issue body; the next pass reflects it and never overwrites it.