Fix macOS Process hangs caused by stopped children by steveisok · Pull Request #135323 · dotnet/runtime

steveisok · 2026-10-07T01:51:36Z

On macOS, waitid(WEXITED | WNOHANG | WNOWAIT) can report a stopped child even though waitpid(WNOHANG) cannot reap it. The SIGCHLD handler repeatedly observes the same child while holding shared process-management locks, blocking other Process operations.

Fall back to scanning all tracked children when the reported child cannot be reaped, rather than continuing the no-progress loop. This guard lives in the shared Unix implementation; successful reaping retains the existing fast path, and the scan reuses the existing nonblocking fallback for unknown children.

Add isolated regression coverage for synchronous and asynchronous exit waits while another child is stopped.
Restore class-level parallelism for Process tests on macOS by removing the temporary serialization safeguard.
Preserve the existing RemoteExecutor isolation changes and timeouts.

Resolves #135294

Note

This pull request description and code changes were generated with GitHub Copilot assistance.

Fall back to scanning tracked children when waitid reports a child that cannot be reaped, avoiding a SIGCHLD livelock with stopped children on macOS. Add isolated synchronous and asynchronous regression coverage and restore class-level test parallelism on macOS. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

azure-pipelines · 2026-10-07T01:52:21Z

Azure Pipelines: Successfully started running 3 pipeline(s). 13 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run.

dotnet-policy-service · 2026-10-07T01:52:45Z

Tagging subscribers to this area: @dotnet/area-system-diagnostics-process
See info in area-owners.md if you want to be subscribed.

jkotas · 2026-10-07T02:12:57Z

cc @tmds

tmds

LGTM!

tmds · 2026-10-07T07:19:09Z

+                            // The child may be unmanaged, or stopped rather than exited (macOS waitid
+                            // can report stopped children despite WEXITED). Scan all managed children


suggestion for between the parentheses:

"on macOS, WaitIdAnyExitedNoHangNoWait can report stopped children, despite not including WSTOPPED when calling waitid"

adamsitnik

The fix is very simple and provides reliable test 👍

Before I hit the approve button, I need to get a final confirmation of how exactly it works (I need to understand risk, as I want to backport it to .NET 11)

Big thanks for your help @steveisok and @tmds !

adamsitnik · 2026-10-07T12:52:18Z

+
+                try
+                {
+                    Assert.Equal(0, Interop.Sys.Kill(stoppedPid, Interop.Sys.GetPlatformSIGSTOP()));


I wanted to suggest using Process.Signal(PosixSignal) here but then I realized that PosixSignal dos not expose SIGSTOP ;)

adamsitnik · 2026-10-07T12:54:43Z

+        [PlatformSpecific(TestPlatforms.OSX)]
+        [InlineData(false)]
+        [InlineData(true)]
+        public void WaitForExit_StoppedChild_DoesNotPreventReapingOtherChildren(bool useAsync)


At the first look, this test seems to be a hack. But after studying my own #133948 I think it's currently the best way to test the hang in deterministic way without spawning too many processes. So it's great!👍

adamsitnik · 2026-10-07T12:57:56Z

-                            // unlikely: This is not a managed Process, so we are not responsible for reaping.
-                            // Fall back to checking all Processes.
+                            // The child may be unmanaged, or stopped rather than exited (macOS waitid
+                            // can report stopped children despite WEXITED). Scan all managed children


I would extend what @tmds has suggested with the mention of SA_NOCLDSTOP which is the thing I've used in #128598 hoping it's going to be enough

Suggested change

// can report stopped children despite WEXITED). Scan all managed children

// can report stopped children despite not including WSTOPPED when calling waitid

// and using SA_NOCLDSTOP when registering the signal handler). Scan all managed children

adamsitnik · 2026-10-07T13:29:01Z

                    {
-                        if (s_childProcessWaitStates.TryGetValue(pid, out ProcessWaitState? pws))
+                        if (s_childProcessWaitStates.TryGetValue(pid, out ProcessWaitState? pws) &&
+                            pws.TryReapChild(configureConsole))


@tmds @steveisok Just to make sure I have good understanding of this fix, as I would like to backport it to .NET 11.

When registering SIGCHILD, we let OS know we are not interested in SIGSTOP by using SA_NOCLDSTOP:

runtime/src/native/libs/System.Native/pal_signal.c

Line 510 in 122c28f

installed = InstallSignalHandler(SIGCHLD, SA_RESTART | SA_NOCLDSTOP);

macOS ignores it and as soon we SIGSTOP a child (for example, by using Process.Kill(entireProcessTree: true), we receive SIGCHILD.

In SIGCHILD handler, we check for the ID of the child process that has exited and has not been reaped. On macOS, the pid of the stopped process is returned, we fail to reap it and keep looping forever.

runtime/src/native/libs/System.Native/pal_process.c

Line 1185 in 122c28f

while (CheckInterrupted(result = waitid(P_ALL, 0, &siginfo, WEXITED | WNOHANG | WNOWAIT)));

With this fix, we are going to read the pid of the process that was stopped, fail to reap it and fall back to checking all other running child processes for exit. We won't loop forever, but we also won't reap it.

For every next SIGCHILD, we are going to repeat that, as the signal won't be reaped (marked as observed).

If the SIGSTOP was sent by Process.Kill(entireProcessTree: true), the process is going to be terminated soon, and then we are going to reap it and stop falling to checking all child processes?

That is correct.

To avoid the checkAll behavior during the Process.Kill, we'd need to consume the STOPPED state.

Yes, that matches the fix’s behavior. While the stopped status remains observable, subsequent SIGCHLD callbacks can take the scan-all fallback. Once the child is killed and reaped, it no longer causes that fallback. The key is releasing the shared locks so process-tree termination and unrelated Process operations can continue. As tmds notes, consuming the stopped status would avoid those repeated scans; this fix leaves that status untouched.

Document the omitted WSTOPPED option and the SIGCHLD handler's SA_NOCLDSTOP registration without changing behavior. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

dotnet-policy-service Bot assigned steveisok Oct 7, 2026

github-actions Bot added the area-System.Diagnostics.Process label Oct 7, 2026

steveisok requested a review from a team October 7, 2026 01:59

jkotas requested a review from adamsitnik October 7, 2026 02:11

This was referenced Oct 7, 2026

HttpMetricsTest_Http20.TimeInQueue_RecordedForNewConnectionsOnly test failure #122522

Open

System.Net.Sockets.Tests tests fail with ObjectDisposedException #126183

Open

SslStreamTlsResumeTests.DifferentEncryptionPolicy_NoResume failing #133418

Open

tmds approved these changes Oct 7, 2026

View reviewed changes

tmds reviewed Oct 7, 2026

View reviewed changes

adamsitnik reviewed Oct 7, 2026

View reviewed changes

Clarify macOS stopped-child reaping comment

6e21914

Document the omitted WSTOPPED option and the SIGCHLD handler's SA_NOCLDSTOP registration without changing behavior. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

This was referenced Oct 8, 2026

Checkout failure: "Git fetch failed with exit code 128" dotnet/arcade#9009

Open

Unable to checkout dotnet/dnceng#4115

Open

[wasm] OpenQA.Selenium.WebDriverTimeoutException: timeout: Timed out receiving message from renderer #117486

Open

Navigation Menu

Repository navigation

Fix macOS Process hangs caused by stopped children - #135323

Fix macOS Process hangs caused by stopped children#135323
steveisok wants to merge 2 commits into
dotnet:mainfrom
steveisok:steveisok-macos-process-hangs

steveisok commented Oct 7, 2026

Uh oh!

azure-pipelines Bot commented Oct 7, 2026

Uh oh!

dotnet-policy-service Bot commented Oct 7, 2026

Uh oh!

jkotas commented Oct 7, 2026

Uh oh!

tmds left a comment

Uh oh!

tmds Oct 7, 2026 •

edited

Loading

Uh oh!

adamsitnik left a comment

Uh oh!

adamsitnik Oct 7, 2026

Uh oh!

adamsitnik Oct 7, 2026

Uh oh!

adamsitnik Oct 7, 2026

Uh oh!

adamsitnik Oct 7, 2026

Uh oh!

tmds Oct 7, 2026

Uh oh!

steveisok Oct 7, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		// The child may be unmanaged, or stopped rather than exited (macOS waitid
		// can report stopped children despite WEXITED). Scan all managed children

	// can report stopped children despite WEXITED). Scan all managed children
	// can report stopped children despite not including WSTOPPED when calling waitid
	// and using SA_NOCLDSTOP when registering the signal handler). Scan all managed children

Conversation

steveisok commented Oct 7, 2026

Uh oh!

azure-pipelines Bot commented Oct 7, 2026

Uh oh!

dotnet-policy-service Bot commented Oct 7, 2026

Uh oh!

jkotas commented Oct 7, 2026

Uh oh!

tmds left a comment

Choose a reason for hiding this comment

Uh oh!

tmds Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adamsitnik left a comment

Choose a reason for hiding this comment

Uh oh!

adamsitnik Oct 7, 2026

Choose a reason for hiding this comment

Uh oh!

adamsitnik Oct 7, 2026

Choose a reason for hiding this comment

Uh oh!

adamsitnik Oct 7, 2026

Choose a reason for hiding this comment

Uh oh!

adamsitnik Oct 7, 2026

Choose a reason for hiding this comment

Uh oh!

tmds Oct 7, 2026

Choose a reason for hiding this comment

Uh oh!

steveisok Oct 7, 2026

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

tmds Oct 7, 2026 •

edited

Loading