Skip to content

Fix multithreaded shutdown thread safety issues - #5960

Merged
martin-frbg merged 9 commits into
OpenMathLib:developfrom
ngoldbaum:shutdown-release-race
Aug 7, 2026
Merged

martin-frbg merged 9 commits into
OpenMathLib:developfrom
ngoldbaum:shutdown-release-race

Conversation

@ngoldbaum

@ngoldbaum ngoldbaum commented Aug 4, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #5954.

The first commit adds a failing test that stress-tests multithreaded shutdown.

The later commits fixes the bug by adding the missing locking that only alloc_mmap currently has to a centralized helper shared by all the allocator backends and adds locking for the windows allocator case.

To fully fix this I also needed to add some more shutdown logic that is needed to prevent the cleanup logic (which acquires locks) from firing if shutdown is in progress. It uses RtlDllShutdownInProgress for this, there's also an explanatory comment.

@ngoldbaum
ngoldbaum force-pushed the shutdown-release-race branch 2 times, most recently from f8ff54e to 6d48fdb Compare August 5, 2026 00:14
@ngoldbaum
ngoldbaum force-pushed the shutdown-release-race branch from 6d48fdb to 6e5c525 Compare August 5, 2026 01:17
@ngoldbaum
ngoldbaum marked this pull request as ready for review August 5, 2026 14:41
@martin-frbg

Copy link
Copy Markdown
Collaborator

Jenkins error (on zarch) is

make[1]: Entering directory '/home/alfred/jenkins/workspace/OpenBLAS-Z_PR-5960/cpp_thread_test'
Makefile:48: *** missing 'endif'.  Stop.

@martin-frbg martin-frbg added this to the 0.3.35 milestone Aug 7, 2026
@martin-frbg
martin-frbg merged commit a692300 into OpenMathLib:develop Aug 7, 2026
106 checks passed
ViralBShah added a commit to JuliaLang/julia that referenced this pull request Oct 26, 2026
…#64249)

* openblas: Carry upstream fix for allocator contention between threads

OpenBLAS's default (non-TLS) buffer allocator, which Julia uses,
serializes threads that call BLAS concurrently. Small level-2 calls such
as `trsv` from several threads then run slower than the same calls made
one after another, and so do parallel solves with one supernodal CHOLMOD
factor, which make many such calls. Carry the five commits of
OpenMathLib/OpenBLAS#6105, which remove that contention and fix a crash
once more buffers are in use than the main table holds, until a release
includes them.

The commits apply after the winexit and cancellation patches. The only
change from upstream: the overflow area's release table keeps 0.3.34's
`malloc`, since upstream's `calloc` goes with shutdown bookkeeping from
OpenMathLib/OpenBLAS#5960 that 0.3.34 lacks.

This covers the source build. The binaries get the same patch through
JuliaPackaging/Yggdrasil#15321 and a new `OpenBLAS_jll`.

Assisted-by: Claude Code (Opus 5.5)

* openblas: Update OpenBLAS_jll to 0.3.34+1

Use the build of OpenBLAS 0.3.34 that carries the allocator contention
fix from the previous commit, so that the binaries get it too.

Assisted-by: Claude Code (Opus 5.5)
KristofferC pushed a commit to JuliaLang/julia that referenced this pull request Oct 28, 2026
…#64249)

* openblas: Carry upstream fix for allocator contention between threads

OpenBLAS's default (non-TLS) buffer allocator, which Julia uses,
serializes threads that call BLAS concurrently. Small level-2 calls such
as `trsv` from several threads then run slower than the same calls made
one after another, and so do parallel solves with one supernodal CHOLMOD
factor, which make many such calls. Carry the five commits of
OpenMathLib/OpenBLAS#6105, which remove that contention and fix a crash
once more buffers are in use than the main table holds, until a release
includes them.

The commits apply after the winexit and cancellation patches. The only
change from upstream: the overflow area's release table keeps 0.3.34's
`malloc`, since upstream's `calloc` goes with shutdown bookkeeping from
OpenMathLib/OpenBLAS#5960 that 0.3.34 lacks.

This covers the source build. The binaries get the same patch through
JuliaPackaging/Yggdrasil#15321 and a new `OpenBLAS_jll`.

Assisted-by: Claude Code (Opus 5.5)

* openblas: Update OpenBLAS_jll to 0.3.34+1

Use the build of OpenBLAS 0.3.34 that carries the allocator contention
fix from the previous commit, so that the binaries get it too.

Assisted-by: Claude Code (Opus 5.5)
(cherry picked from commit 3e610d6)
KristofferC pushed a commit to JuliaLang/julia that referenced this pull request Oct 28, 2026
…#64249)

* openblas: Carry upstream fix for allocator contention between threads

OpenBLAS's default (non-TLS) buffer allocator, which Julia uses,
serializes threads that call BLAS concurrently. Small level-2 calls such
as `trsv` from several threads then run slower than the same calls made
one after another, and so do parallel solves with one supernodal CHOLMOD
factor, which make many such calls. Carry the five commits of
OpenMathLib/OpenBLAS#6105, which remove that contention and fix a crash
once more buffers are in use than the main table holds, until a release
includes them.

The commits apply after the winexit and cancellation patches. The only
change from upstream: the overflow area's release table keeps 0.3.34's
`malloc`, since upstream's `calloc` goes with shutdown bookkeeping from
OpenMathLib/OpenBLAS#5960 that 0.3.34 lacks.

This covers the source build. The binaries get the same patch through
JuliaPackaging/Yggdrasil#15321 and a new `OpenBLAS_jll`.

Assisted-by: Claude Code (Opus 5.5)

* openblas: Update OpenBLAS_jll to 0.3.34+1

Use the build of OpenBLAS 0.3.34 that carries the allocator contention
fix from the previous commit, so that the binaries get it too.

Assisted-by: Claude Code (Opus 5.5)
(cherry picked from commit 3e610d6)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Windows shutdown may race triggering crashes or hangs on workloads that oversaturate the CPU

2 participants