Skip to content

Fix double GC relocation of overlapping byrefs - #135890

Merged
jkoritzinsky merged 3 commits into
dotnet:mainfrom
jkoritzinsky:dev/jkoritzinsky/overlapping-byref-gc-fix
Oct 17, 2026
Merged

jkoritzinsky merged 3 commits into
dotnet:mainfrom
jkoritzinsky:dev/jkoritzinsky/overlapping-byref-gc-fix

Conversation

@jkoritzinsky

Copy link
Copy Markdown
Member

Summary

Report each physical byref slot only once when scanning an explicit-layout byref-like value type. Overlapping fields currently cause the shared scanner to report the same slot multiple times, so a compacting GC can relocate an already-relocated interior pointer again.

This is the standalone GC fix requested in review of #133659. It is not debugger-specific: an ordinary by-value ref-struct argument reproduces the corruption on Windows x86 with DOTNET_GCStress=1.

  • Track enclosing overlapping layouts and check earlier fields for the exact slot address, not the referenced object value. The check is stack-only and allocation-free; document its quadratic cost for rare overlapping layouts.
  • Use the already-loaded value-type field handle rather than triggering type loading during GC enumeration.
  • Add a targeted test to the existing ref-field tests, asserting all four expected values through both overlapping views. A non-inlined callee preserves the by-value argument across the JIT transition.

No debugger-specific or cDAC changes are included.

Validation

Windows x86 Checked runtime, Release libraries, at main base 6b1cd4b85c4b:

Check Unfixed main Fixed
Reviewer's repro, normal 1 2 3 4 1 2 3 4
Reviewer's repro, DOTNET_GCStress=1 0 3801155 7536732 2949173 1 2 3 4
Ref-field tests, normal 6/6 passed 6/6 passed
Ref-field tests, DOTNET_GCStress=1 New regression fails: expected 1, actual 6881312 (exit 101) 6/6 passed (exit 100)
Existing inline-array tests, normal / DOTNET_GCStress=1 Not run 11/11 passed in each mode (exit 100)

The regression assembly and CoreLib are byte-identical between the unfixed and fixed runs.

Build commands and results:

.\build.cmd clr+libs+host -a x86 -rc Checked -lc Release
# Passed; 0 errors, 3 existing MsQuic NU1902 warnings.
.\build.cmd clr.native+clr.corelib+clr.nativecorelib -c Checked -a x86
# Passed; 0 warnings, 0 errors.
.\src\tests\build.cmd -GenerateLayoutOnly x86 Checked /p:HostConfiguration=Debug
# Passed before and after the fix.
.\src\tests\build.cmd -Test Loader\classloader\RefFields\Validate.csproj x86 Checked /p:HostConfiguration=Debug
# Passed; 1 selected test project, 0 warnings/errors.
.\dotnet.cmd build -c Checked src\tests\Loader\classloader\InlineArray\InlineArrayValid.csproj /p:TargetArchitecture=x86 /p:BuildAsStandalone=true
# Passed; 0 warnings/errors.

Tests were run with CORE_ROOT set to artifacts\tests\coreclr\windows.x86.Checked\Tests\Core_Root, using & "$env:CORE_ROOT\corerun.exe" with Validate.dll and InlineArrayValid.dll from their respective output directories, once without DOTNET_GCStress and once with it set to 1. Other architectures/platforms were not run.

Resolves #135245

Note

This implementation and PR description were prepared with GitHub Copilot.

Deduplicate physical byref slots in explicit-layout value types without allocating during GC. Preserve non-loading field lookups and document the rare-layout quadratic scan. Add a targeted by-value regression in the existing ref-field tests.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @agocke
See info in area-owners.md if you want to be subscribed.

Nest ValueClassByRefSearchContext inside ByRefPointerOffsetsReporter without changing its scanning behavior.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Comment thread src/tests/Loader/classloader/RefFields/Validate.cs
Comment thread src/coreclr/vm/siginfo.cpp Outdated
Reuse the value-type reporter in overlap-probe mode without building nested overlap lists or recursively checking overlaps. Add nested-layout regression coverage with distinct and shared backing arrays.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Safety-critical GC scanning changes need runtime-owner review of scanner invariants and platform coverage beyond Windows x86.

No open issues 🎉

What changed in this PR

Fixes double GC relocation by reporting each physical byref slot once in overlapping ref-struct layouts, independently of debugger functionality.

Changes:

  • Deduplicates overlapping slots using allocation-free traversal.
  • Uses already-loaded field type handles during GC scanning.
  • Adds regression coverage for nested overlaps and shared backing arrays.

Validation reported in the PR covers Windows x86; no builds or tests were run during this review.

File Description
src/​tests/​Loader/​classloader/​RefFields/​Validate.cs Tests overlapping and nested byrefs through by-value arguments.
src/​coreclr/​vm/​siginfo.cpp Prevents duplicate slot reporting and avoids type loading during scanning.

Review effort: Balanced

@jkoritzinsky

Copy link
Copy Markdown
Member Author

/ba-g failures unrelated

@jkoritzinsky
jkoritzinsky merged commit 6fe708a into dotnet:main Oct 17, 2026
106 of 108 checks passed
@github-project-automation github-project-automation Bot moved this to Done in AppModel Oct 17, 2026
@dotnet-milestone-bot dotnet-milestone-bot Bot added this to the 12.0-preview1 milestone Oct 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Debugger func-eval: a by-value ref struct with overlapping ref fields is reported to the GC twice, so a compacting GC relocates the byref twice

3 participants