[mono] Runtime tests failing with 'Attempting to JIT compile method' for Mono FullAOT mode

[ivanpovazan]

Summary

Runtime tests:

• /JIT/Directed/callconv/PlatformDefaultMemberFunction/PlatformDefaultMemberFunctionTest/*
• /JIT/Directed/callconv/CdeclMemberFunction/CdeclMemberFunctionTest/*
• /JIT/Directed/callconv/StdCallMemberFunction/StdCallMemberFunctionTest/*
• /JIT/Directed/callconv/ThisCall/ThisCallTest/*

are failing in FullAOT mode on Mono with nearly the same Attempting to JIT compile method xxx while running in aot-only mode.

StdCallMemberFunctionTest

Output

Full output for /JIT/Directed/callconv/StdCallMemberFunction/StdCallMemberFunctionTest/*:

System.ExecutionEngineException: Attempting to JIT compile method '(wrapper managed-to-native) StdCallMemberFunctionNative/SizeF modopt(System.Runtime.CompilerServices.CallConvMemberFunction) modopt(System.Runtime.CompilerServices.CallConvStdcall) StdCallMemberFunctionTest:wrapper_native_indirect_0x600003ce7140 (intptr&,StdCallMemberFunctionNative/C*,int)' while running in aot-only mode. See https://docs.microsoft.com/xamarin/ios/internals/limitations for more information.

   at StdCallMemberFunctionTest.Test8ByteHFA(C* instance) in /Users/ivan/repos/runtime-mono-AOT/src/tests/JIT/Directed/callconv/StdCallMemberFunction/StdCallMemberFunctionTest.cs:line 96
   at StdCallMemberFunctionTest.Main(String[] args)
Expected: 100
Actual: 101
END EXECUTION - FAILED

Repro

1. Enable building the test with Mono AOT

diff --git a/src/tests/JIT/Directed/callconv/Directory.Build.props b/src/tests/JIT/Directed/callconv/Directory.Build.props
index 49b99b7a101..9ed3161df40 100644
--- a/src/tests/JIT/Directed/callconv/Directory.Build.props
+++ b/src/tests/JIT/Directed/callconv/Directory.Build.props
@@ -2,6 +2,6 @@
   <Import Project="$([MSBuild]::GetPathOfFileAbove(Directory.Build.props, $(MSBuildThisFileDirectory)..))" />
 
   <PropertyGroup>
-    <MonoAotIncompatible>true</MonoAotIncompatible>
+    <MonoAotIncompatible>false</MonoAotIncompatible>
   </PropertyGroup>
 </Project>

2. Build Mono

./build.sh mono+libs+clr.hosts -c debug /p:MonoEnableLlvm=true /p:MonoLLVMUseCxx11Abi=true

3. Build the test

./src/tests/build.sh mono debug /p:LibrariesConfiguration=Debug -test:JIT/Directed/callconv/StdCallMemberFunction/StdCallMemberFunctionTest.csproj

4. AOT compile the test

./src/tests/build.sh mono_fullaot debug /p:LibrariesConfiguration=Debug -test:JIT/Directed/callconv/StdCallMemberFunction/StdCallMemberFunctionTest.csproj

5. Run the test (note: adjust the paths)

MONO_ENV_OPTIONS=--full-aot bash /Users/ivan/repos/runtime-mono-AOT/artifacts/tests/coreclr/OSX.arm64.Debug/JIT/Directed/callconv/StdCallMemberFunction/StdCallMemberFunctionTest/StdCallMemberFunctionTest.sh  -coreroot=/Users/ivan/repos/runtime-mono-AOT/artifacts/tests/coreclr/OSX.arm64.Debug/Tests/Core_Root

Additional notes

Tests exclusion details should be updated to reference this tracking issue: Done.

Possibly related to: #50440

cc: @lambdageek @lateralusX

[ivanpovazan]

@lambdageek should we try to fix this for .NET7?

[SamMonoRT]

Assigning to @LeVladIonescu related to current work