.NET 11 regression: static SAFEARRAY copy-back discards existing VARIANT byref pointers

[MichalStrehovsky]

Description

IDispatch copy-back of ref object[] into an existing FADF_STATIC SAFEARRAY(VARIANT) loses VT_BYREF element references. Managed code sees the referenced integer 4 and assigns integer 7, but .NET 11 overwrites the VARIANT slot with VT_I4 instead of updating the caller's referenced integer. .NET 10 preserves VT_BYREF|VT_I4 and writes 7 through the reference.

Reproduction steps

Create an empty directory outside the runtime repository and save the following as Repro.csproj and Program.cs. No external native library or NuGet package is required; the native-facing entry point is provided through UnmanagedCallersOnly or Windows built-in COM interop.

Repro.csproj

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net10.0</TargetFramework>
    <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
    <Nullable>enable</Nullable>
  </PropertyGroup>
</Project>

Program.cs

// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.

// DispatchInfo::MarshalParamManagedToNativeRef (src/coreclr/vm/dispatchinfo.cpp) copies a managed
// array back into a caller-owned FADF_STATIC SAFEARRAY, and passes TRUE to
// GetInstantiatedSafeArrayMethod. That argument is bHeterogeneous; bNativeDataValid stays FALSE.
// VariantArrayElementMarshaler therefore zeroes every existing VARIANT before converting, which
// discards the VT_BYREF pointer that the caller supplied. The referenced native integer is left
// unchanged instead of being written through.

using System;
using System.Runtime.InteropServices;

[assembly: ComVisible(true)]

internal static unsafe class Program
{
    private const ushort VT_I4 = 3;
    private const ushort VT_VARIANT = 12;
    private const ushort VT_ARRAY = 0x2000;
    private const ushort VT_BYREF = 0x4000;
    private const ushort FADF_STATIC = 0x0002;
    private const ushort DispatchMethod = 1;

    private static int Main()
    {
        Console.WriteLine(RuntimeInformation.FrameworkDescription);
        if (!OperatingSystem.IsWindows())
        {
            Console.WriteLine("Requires Windows built-in COM interop.");
            return 2;
        }

        int native = 4;
        SafeArray* array = SafeArrayCreateVector(VT_VARIANT, 0, 1);
        if (array is null)
        {
            Console.WriteLine("SafeArrayCreateVector failed.");
            return 2;
        }

        // Mark the array as static so the runtime copies back into this buffer instead of
        // allocating a replacement, and point its single VARIANT at the native integer.
        array->Features |= FADF_STATIC;
        Variant* elements = (Variant*)array->Data;
        elements[0].Vt = VT_BYREF | VT_I4;
        elements[0].Data1 = (nint)(&native);

        var target = new Target();
        nint dispatch = Marshal.GetIDispatchForObject(target);
        try
        {
            int dispId = GetDispId(dispatch, nameof(Target.Update));
            var argument = new Variant
            {
                Vt = VT_BYREF | VT_ARRAY | VT_VARIANT,
                Data1 = (nint)(&array)
            };

            Invoke(dispatch, dispId, &argument);

            Console.WriteLine($"Managed method saw: {target.Observed}");
            Console.WriteLine($"Referenced native integer: {native} (expected 7)");
            Console.WriteLine($"Element VARIANT type: 0x{elements[0].Vt:X4} (expected 0x{VT_BYREF | VT_I4:X4} VT_BYREF|VT_I4)");

            bool passed = native == 7 && elements[0].Vt == (VT_BYREF | VT_I4);
            Console.WriteLine(passed
                ? "PASS"
                : "BUG: the existing VT_BYREF element was zeroed, so the write-through was lost.");
            return passed ? 0 : 1;
        }
        finally
        {
            Marshal.Release(dispatch);
            array->Features &= unchecked((ushort)~FADF_STATIC);
            SafeArrayDestroy(array);
        }
    }

    private static int GetDispId(nint dispatch, string name)
    {
        Guid riid = Guid.Empty;
        fixed (char* namePtr = name)
        {
            char* names = namePtr;
            int dispId;
            var getIdsOfNames = (delegate* unmanaged[Stdcall]<nint, Guid*, char**, uint, uint, int*, int>)
                (*(*(void***)dispatch + 5));
            Marshal.ThrowExceptionForHR(getIdsOfNames(dispatch, &riid, &names, 1, 0, &dispId));
            return dispId;
        }
    }

    private static void Invoke(nint dispatch, int dispId, Variant* argument)
    {
        var parameters = new DispParams
        {
            Arguments = argument,
            NamedArguments = null,
            ArgumentCount = 1,
            NamedArgumentCount = 0
        };

        byte* exceptionInfo = stackalloc byte[128];
        uint argumentError;
        Guid riid = Guid.Empty;
        var invoke = (delegate* unmanaged[Stdcall]<nint, int, Guid*, uint, ushort, DispParams*, Variant*, void*, uint*, int>)
            (*(*(void***)dispatch + 6));
        Marshal.ThrowExceptionForHR(
            invoke(dispatch, dispId, &riid, 0, DispatchMethod, &parameters, null, exceptionInfo, &argumentError));
    }

    [DllImport("oleaut32.dll")]
    private static extern SafeArray* SafeArrayCreateVector(ushort vt, int lowerBound, uint count);

    [DllImport("oleaut32.dll")]
    private static extern int SafeArrayDestroy(SafeArray* array);

    [StructLayout(LayoutKind.Sequential)]
    private struct Variant
    {
        public ushort Vt;
        public ushort Reserved1;
        public ushort Reserved2;
        public ushort Reserved3;
        public nint Data1;
        public nint Data2;
    }

    [StructLayout(LayoutKind.Sequential)]
    private struct SafeArray
    {
        public ushort Dimensions;
        public ushort Features;
        public uint ElementSize;
        public uint Locks;
        public nint Data;
        public uint Elements;
        public int LowerBound;
    }

    [StructLayout(LayoutKind.Sequential)]
    private struct DispParams
    {
        public Variant* Arguments;
        public int* NamedArguments;
        public uint ArgumentCount;
        public uint NamedArgumentCount;
    }
}

[ComVisible(true)]
public class Target
{
    public string? Observed { get; private set; }

    public void Update(ref object[] values)
    {
        Observed = $"[{string.Join(", ", values)}]";
        values[0] = 7;
    }
}

Build once with dotnet build Repro.csproj. Run the generated apphost against the two runtimes:

$env:DOTNET_ROLL_FORWARD_TO_PRERELEASE = '1'
$env:DOTNET_ROLL_FORWARD = 'LatestPatch'
& .\bin\Debug\net10.0\Repro.exe

$env:DOTNET_ROLL_FORWARD = 'LatestMajor'
& .\bin\Debug\net10.0\Repro.exe

These commands assume the apphost resolves to a .NET installation containing the tested .NET 10 and .NET 11 runtimes, with no newer major version. If needed, set DOTNET_ROOT to that installation. Check the framework version printed by the program rather than assuming which runtime was selected. Remove the temporary roll-forward environment variables after the comparison.

Expected behavior

Copy back into the existing array contents while preserving a compatible byref element: the referenced integer becomes 7 and the slot remains VT_BYREF|VT_I4.

.NET 10 output:

.NET 10.0.5
Managed method saw: [4]
Referenced native integer: 7 (expected 7)
Element VARIANT type: 0x4003 (expected 0x4003 VT_BYREF|VT_I4)
PASS

Actual behavior

.NET 11 output from the same compiled program:

.NET 11.0.0-rc.1.26420.103
Managed method saw: [4]
Referenced native integer: 4 (expected 7)
Element VARIANT type: 0x0003 (expected 0x4003 VT_BYREF|VT_I4)
BUG: the existing VT_BYREF element was zeroed, so the write-through was lost.

Regression

Confirmed by running the same net10.0-targeted program on .NET 10.0.5 and .NET 11.0.0-rc.1.26420.103.

The exact introducing commit has not been established by a runtime build-and-bisect. The implementation observations below are based on source inspection.

Configuration

• Windows x64, CoreCLR, built-in runtime marshalling.
• Passing runtime: .NET 10.0.5.
• Failing runtime: .NET 11.0.0-rc.1.26420.103.
• Compiled with SDK 11.0.100-rc.1.26420.103, targeting net10.0.
• The repository runtime was not rebuilt for this comparison. Linux, ARM64, x86, Mono, NativeAOT, and source-generated marshalling were not tested.

Implementation observations

In DispatchInfo::MarshalParamManagedToNativeRef, the call to GetInstantiatedSafeArrayMethod supplies TRUE as the fourth argument. That argument is bHeterogeneous, while the separate bNativeDataValid argument defaults to FALSE.

Consequently, VariantArrayElementMarshaler clears the existing VARIANT before converting it, discarding the caller's byref pointer. The issue is treating initialized copy-back storage as uninitialized.

The repro marks the allocated SAFEARRAY as FADF_STATIC for the call to exercise this caller-owned-storage path, then removes the flag before destroying the allocation. Its referenced integer remains alive for the entire call. The base array type is VT_VARIANT; VT_BYREF belongs to an individual element.

This is distinct from the different-type VARIANT replacement/coercion regression: this example replaces integer 4 with integer 7 and should simply preserve the existing reference.

Note

This report was prepared with GitHub Copilot assistance; the included runtime outputs were reproduced locally.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/interop-contrib
See info in area-owners.md if you want to be subscribed.