Skip to content

Flaky test: status_reports_stopped_when_saved_comfyui_pid_is_gone races other tests for its "unused" port #89

Closed

Description

@mikeroySoft

Problem
comfyui::tests::status_reports_stopped_when_saved_comfyui_pid_is_gone (apps/rocm/src/comfyui.rs, ~line 2736 on main 800f373) fails at random under the parallel test runner.

The test saves a ComfyUI state with pid: 0 and a URL on unused_local_port(), then asserts render_status reports status: stopped. unused_local_port() (~line 2306) binds 127.0.0.1:0, reads the port, and drops the listener, so the port is free again before the test uses it. Other tests in the same binary call unused_local_port() and start listeners (~lines 3976, 4071, 4141, 4291, 4683). When one of them grabs the same port, the status probe connects, ComfyUI reads as starting or running, and the assertion fails.

It passes 10/10 on a quiet run but failed three times on 2026-10-24, in the post-rebase gates for #17 (PR #25) and #73 (PR #74), and escalated both.

Fix
Use port 0 in this test (let port = 0;). Nothing can listen on port 0, so connecting to 127.0.0.1:0 always fails. This is the same one-line change as commit a29b7e10 ("test(comfyui): avoid stale-status port race"). A worker made that commit during #73, but review dropped it as out of scope. Change only this test. Leave the other unused_local_port() callers alone; they need a real port.

Acceptance criteria

  • status_reports_stopped_when_saved_comfyui_pid_is_gone no longer calls unused_local_port(). It uses a port that can never accept a connection (port 0), and its existing assertions are unchanged (status: stopped, next step: rocm comfyui start, not starting or running).
  • The test passes 20 times in a row:
    for i in $(seq 20); do cargo test -p rocm status_reports_stopped_when_saved_comfyui_pid_is_gone || exit 1; done
    
  • The full suite still passes: cargo test -p rocm and cargo clippy --workspace --all-targets -- -D warnings.
  • Check that the test body no longer contains unused_local_port: awk '/fn status_reports_stopped_when_saved_comfyui_pid_is_gone/,/^ }$/' apps/rocm/src/comfyui.rs | grep -c unused_local_port prints 0.

Out of scope

  • Changing unused_local_port() itself or any other test.
  • Any non-test code.

Estimated size: 1 line, 1 file.

Activity

  1. added
    bugSomething isn't working
    needs-triageMaintainer needs to evaluate this issue
    on Oct 25, 2026
  2. mikeroysoft-agent commented on Oct 25, 2026

    @mikeroysoft-agent
    Collaborator

    Triage: The issue is fully specified: it has a clear problem statement (flaky test racing on an 'unused' port), a precise one-line fix (use port 0 instead of unused_local_port()), explicit scope boundaries (change only this test, leave other callers alone, no non-test code), and observable acceptance criteria with exact verification commands. This is a low-risk, contained test-only change with no design judgment, release, signing, or security implications.

    Agent brief: In apps/rocm/src/comfyui.rs, change the test fn status_reports_stopped_when_saved_comfyui_pid_is_gone so it no longer calls unused_local_port(); instead use a port that can never accept a connection (let port = 0;), leaving the existing assertions unchanged (status: stopped, next step: rocm comfyui start, not starting or running). Do not modify unused_local_port() or any other test, and touch no non-test code. Verify with: awk '/fn status_reports_stopped_when_saved_comfyui_pid_is_gone/,/^ }$/' apps/rocm/src/comfyui.rs | grep -c unused_local_port (must print 0); then for i in $(seq 20); do cargo test -p rocm status_reports_stopped_when_saved_comfyui_pid_is_gone || exit 1; done; then cargo test -p rocm; then cargo clippy --workspace --all-targets -- -D warnings.

  3. added
    ready-for-agentFully specified and ready for an AFK agent
    and removed
    needs-triageMaintainer needs to evaluate this issue
    on Oct 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

bugSomething isn't workingready-for-agentFully specified and ready for an AFK agent

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions