Skip to content

usdrecord keeps waiting after its Arras render worker exits #309

Open

Description

@nicolaspopravka

During a benchmark run with the MoonRay Hydra delegate in OpenMoonRay v2026.29.1 and OpenUSD 24.08 on Rocky Linux 8.10.
The Moana Island worker exited with signal 9. Arras reported the failure, but usdrecord stayed alive without its render worker for about 2 hours 46 minutes, until the benchmark's own deadline stopped it. No Moana image was produced. I don't have evidence establishing why the worker died in this run.

I reproduced this behavior separately on OpenUSD 23.08, using McUsd under Xvfb/Mesa and deliberately killing the Arras worker (execComp). No OOM or GPU was needed:

Error: Arras session stopped, compExited: mcrt exited due to signal 9
RESULT: CLIENT STILL ALIVE 150s after its render worker died

Control probe and logs. The probe script starts usdrecord on McUsd, waits for the worker, kills it while the frame is still rendering, and observes the client.

In hdMoonray, ArrasRenderer::statusHandler logs stopped/stopping and sets the connection state to disconnected, but leaves the unfinished frame waiting for completion. Without a final image, isFrameComplete() remains false and usdrecord keeps waiting for convergence.

I would expect usdrecord to stop waiting and report the failed render when Arras reports that the worker has stopped. One failed render should not block the rest of a batch until an external timeout.

The reason the worker stopped and usdrecord's handling of an aborted recording are separate questions. This report is about the client continuing to wait after Arras has already reported the stopped session.

Benchmark issue and investigation.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions