Skip to content

Matrix operations give wrong answers when called from more than one thread - #41

Open
Rafael-SOWNet wants to merge 1 commit into
asc-community:masterfrom
Rafael-SOWNet:thread-safe-matrix-ops
Open

Rafael-SOWNet wants to merge 1 commit into
asc-community:masterfrom
Rafael-SOWNet:thread-safe-matrix-ops

Conversation

@Rafael-SOWNet

Copy link
Copy Markdown
Member

Fixes #40.

Two process-wide statics are read and written with no synchronisation, so matrix operations called from more than one thread come back with wrong numbers. Nothing throws and no result is malformed, which is the worst way for this to fail.

Found from the other side: AngouriMath builds its symbolic matrices on GenTensor, and a determinant taken on a background thread was numerically wrong (AngouriMath#1219).

The scratch-matrix pool

SquareMatrixFactory.GetMatrix hands every caller the same GenTensor for a given size, and the caller writes into it — Inversion.GetCofactorMatrix(t, temp, ...) fills the matrix it is handed. Two threads at the same size overwrite each other's minors between the write and the read.

Sixty 5×5 int matrices, computed once sequentially for truth, then rebuilt and recomputed under Parallel.For:

disagreements
DeterminantLaplace 53 of 60
Adjoint 60 of 60

[ThreadStatic] fixes it: the buffer is scratch, so there is nothing to share between threads in the first place. The lock goes with it, since nothing outside the owning thread can see the list any more — and that lock was incomplete anyway, because the list was indexed after it was released, so a concurrent Add could reallocate underneath the reader.

DeterminantGaussianSafeDivision is unaffected; it works on its own copy. There is a test recording that, so the boundary stays measured rather than assumed.

The compiled-operation cache

ExpressionCompiler.storage is a plain Dictionary written by whichever thread asks first. Reached cold from several threads it fails with the runtime's own diagnostic:

Operations that change non-concurrent collections must have exclusive access.
A concurrent update was performed on this collection and corrupted its state.

followed by a lookup missing a key that had been written. ConcurrentDictionary fixes it. A duplicate compilation under GetOrAdd is harmless, since two compilations of one key produce delegates that do the same thing.

One note on the test, because it nearly fooled me. My first version computed the expected values sequentially and only then went parallel — which populated every key before the threads started, so the parallel pass only read a finished dictionary and the test passed against the broken code. The cache holds a handful of entries, so any warm-up hides the defect completely. The test has none now.

Performance

Single-threaded, best of 15 runs, two independent runs of each arm on one machine:

ms before after
Laplace 6×6 0.00870 / 0.00875 0.00838 / 0.00825
Laplace 9×9 4.430 / 4.395 4.201 / 4.197
Adjoint 6×6 0.00808 / 0.00799 0.00821 / 0.00783

The Laplace determinant comes out about 5% faster. Getting there took one correction worth stating: reading a [ThreadStatic] field costs a thread-local lookup, and the naive version, which did that at every one of the n! nodes of the recursion, measured +10% — a real regression I would have shipped if I had not measured it. The pool is taken once at the top and threaded through instead; sizes only shrink on the way down, so growing it once covers every level. That also replaces the old per-node Count check and shared-list index with a single local index, which is where the 5% comes from.

221 of 221 existing tests pass, plus the four added here.

I would rather you carried the fix than AngouriMath carried a lock around your API, but I appreciate the repository has been quiet since 2022 — so if you would prefer not to take this, say so and I will guard it on our side instead.

🤖 Generated with Claude Code

https://claude.ai/code/session_012sonx8iAspMiwRwokT1Ura

…hread

Two process-wide statics are read and written without synchronisation, and
both fail silently rather than throwing.

The scratch-matrix pool behind DeterminantLaplace and Inverse hands every
caller the same GenTensor for a given size, and the caller writes into it --
GetCofactorMatrix fills the matrix it is given. Two threads taking a
determinant of the same size are handed the same buffer and overwrite each
other's minors between the write and the read. Sixty 5x5 integer matrices,
computed once sequentially for truth and then again under Parallel.For:

    DeterminantLaplace    53 of 60 wrong
    Adjoint               60 of 60 wrong

Nothing throws and no result is malformed. They are simply different numbers.
Making the pool [ThreadStatic] fixes it; the buffer is scratch, so there is
nothing to share between threads in the first place. The lock inside GetMatrix
goes with it, since nothing outside the owning thread can now see the list --
and that lock was in any case incomplete, because the list was indexed after
it had been released, so a concurrent Add could reallocate underneath the
reader.

DeterminantGaussianSafeDivision is unaffected: it works on its own copy and
never asks the pool for anything. A test records that, so the boundary stays
recorded rather than assumed.

The compiled-operation cache behind PiecewiseAdd and its siblings is a plain
Dictionary written from whichever thread asks first. Reached cold from several
threads it fails with the runtime's own diagnostic -- "Operations that change
non-concurrent collections must have exclusive access. A concurrent update was
performed on this collection and corrupted its state" -- and then with a lookup
missing a key that had been written. A ConcurrentDictionary fixes it, and a
duplicate compilation under GetOrAdd is harmless because two compilations of
one key produce delegates that do the same thing.

The first version of that test computed its expected values sequentially and
only then went parallel, which populated every key before the threads started
and passed against the broken code. It has no warm-up now.

Determinant, single-threaded, best of 15 x 20 iterations, two runs each:

                    before             after
    Laplace 6x6     0.00870 0.00875    0.00838 0.00825
    Laplace 9x9     4.430   4.395      4.201   4.197
    Adjoint 6x6     0.00808 0.00799    0.00821 0.00783

Laplace comes out about 5% faster. Reading a [ThreadStatic] field costs a
thread-local lookup, and doing that at every one of the n! nodes of the Laplace
recursion measured +10%, so the pool is taken once at the top and threaded
through the recursion; sizes only shrink on the way down. That also replaces
the old per-node Count check and shared-list index with one local index, which
is where the 5% comes from.

221 of 221 existing tests pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012sonx8iAspMiwRwokT1Ura
@Rafael-SOWNet

Copy link
Copy Markdown
Member Author

The red UnitTests check is not this branch. It is the coverage job, and it fails before a single line of code is compiled:

error : Microsoft.NET.Test.Sdk doesn't support net6.0 and has not been tested with it.
        Consider upgrading your TargetFramework to net8.0 or later.
Unable to create dependency graph file for project 'UnitTests/UnitTests.csproj'.
Cannot add package reference.
##[error]Process completed with exit code 1

That job runs an unpinned dotnet add package Microsoft.NET.Test.Sdk, which today resolves to 18.10.0, and 18.10.0 refuses a net6.0 test project. It is bit rot from the passage of time rather than from any diff — nothing here touches a .csproj or a target framework.

The three jobs that actually build and run the code are green:

SUCCESS  UnitTests (ubuntu-latest)
SUCCESS  UnitTests (windows-latest)
SUCCESS  UnitTests (macos-latest)

Two ways out if you want that job green again: pin the package in the workflow, or set SuppressTfmSupportBuildErrors in UnitTests.csproj. Both are unrelated to this change, so I have deliberately not put either in here — say the word and I will open it separately.

Rafael-SOWNet added a commit to asc-community/AngouriMath that referenced this pull request Sep 9, 2026
#1230)

MathS.Multithreading documents concurrent use as supported, and matrix
operations do not honour it. The cause is not in this repository: GenericTensor
1.0.4 keeps two process-wide mutable statics, reported upstream as
asc-community/GenericTensor#40 with a fix in
asc-community/GenericTensor#41. This is what stands in
until a release carrying that fix exists.

Measured on c2c8eb8, forty 4x4 matrices with non-polynomial entries, each
computed once sequentially and then rebuilt and recomputed under Parallel.For:

    Determinant   38 of 40 disagree
    Inverse       18 of 40 disagree
    Adjugate      11 of 40 disagree
    m + m, m - m, PointwiseMultiplication
                  throws, on a corrupted Dictionary

Nothing about the first three looks wrong. They are well-formed entities of the
right shape carrying another computation's values.

The two statics need opposite treatments, because only one can be closed
without a lock.

The scratch-matrix pool behind DeterminantLaplace and Adjoint hands every
caller the same tensor for a given size and the caller writes into it. There is
nothing to warm and no way to avoid it from out here, so those three call sites
take a lock. That serialises determinants and inverses of matrices the
polynomial elimination declines -- PolynomialDeterminant.Of runs first and
never reaches the pool, which is why the entries in the test are sines and
cosines, and why a polynomial matrix was not affected at all.

The compiled-operation cache behind the elementwise operators is an
unsynchronised Dictionary, so it is only unsafe while being filled; once an
entry is there the reads are pure. That one needs no lock, because the set of
entries this library can ever ask for is fixed: Entity.Matrix rejects any
tensor that is not rank 2, and every call here uses the default single-threaded
mode, so the only reachable keys are addition, subtraction and multiplication
at rank 2. Filling all three once under Lazy leaves the dictionary read-only,
at no steady-state cost.

Answers are unchanged, so there is no BREAKING-CHANGES entry: a single-threaded
caller computed the same values before and computes them now.

The tests rebuild their matrices inside each pass, which is load-bearing rather
than tidiness. Determinant, Inverse and Adjugate are cached lazy properties, so
reusing the instances lets the parallel pass read what the sequential pass
already computed -- a green test that ran the operation once. The elementwise
test has no sequential pass at all for the mirror-image reason: a warm-up
populates every cache key before the threads start and passes against the
unguarded code.

Full suite green: 8499 and 1485.


Claude-Session: https://claude.ai/code/session_012sonx8iAspMiwRwokT1Ura

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Determinant and Inverse return wrong values when called from more than one thread

1 participant