SIGSEGV crash in mono_interp_exec_method on .NET 9 iOS

[kotlarmilos]

After upgrading from .NET 8 to .NET 9, we’re experiencing a non-deterministic SIGSEGV crash on iOS release builds. The crash appears to originate from mono_interp_exec_method.

Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Subtype: KERN_INVALID_ADDRESS at 0x0000000100000019
Exception Codes: 0x0000000000000001, 0x0000000100000019
VM Region Info: 0x100000019 is not in any region. Bytes before following region: 15433703
REGION TYPE START - END [ VSIZE] PRT/MAX SHRMOD REGION DETAIL
UNUSED SPACE AT START
--->
__TEXT 100eb8000-1010dc000 [ 2192K] r-x/r-x SM=COW ...edClient.Maui
Triggered by Thread: 16

Thread 16 Crashed:
0 Librestream.UnifiedClient.Maui 0x00000001010a81e8 mono_interp_exec_method (in Librestream.UnifiedClient.Maui) (interp.c:6221) + https://github.com/dotnet/runtime/commit/2032104fe419bf9a5637a7efd50e85be8c1eed44
1 Librestream.UnifiedClient.Maui 0x00000001010a0084 interp_runtime_invoke (in Librestream.UnifiedClient.Maui) (interp.c:2220) + 1998980
2 Librestream.UnifiedClient.Maui 0x000000010106f038 mono_jit_runtime_invoke (in Librestream.UnifiedClient.Maui) (mini-runtime.c:3686) + 1798200
3 Librestream.UnifiedClient.Maui 0x000000010101a990 mono_runtime_invoke_checked (in Librestream.UnifiedClient.Maui) (object.c:2788) + 1452432
4 Librestream.UnifiedClient.Maui 0x000000010102ca90 start_wrapper (in Librestream.UnifiedClient.Maui) (threads.c:1276) + 1526416
5 libsystem_pthread.dylib 0x000000021d0900ec _pthread_start + 116 (pthread.c:893)
6 libsystem_pthread.dylib 0x000000021d08e72c thread_start + 8 (:-1)

#114262 (comment)
#114262 (comment)

[dotnet-policy-service]

Tagging subscribers to 'os-ios': @vitek-karas, @kotlarmilos, @ivanpovazan, @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.

[dotnet-policy-service]

Tagging subscribers to this area: @BrzVlad, @kotlarmilos
See info in area-owners.md if you want to be subscribed.

[kotlarmilos]

@linnkrb, @ConwayLS Could you help us create a minimal repro? If you can share the threading patterns you're using, that would really help us trigger the issue on our end.

[linnkrb]

We´ll try and create a repo, but a bit hard, but will give it a try. We are getting some data with http, using a Task.whenAll , saving the data if any changes in the local sqllite database and this is run every 15 min with a timer. There are even more stuff going on, but we think this is the cause of it all...

Can confirm thought that we have the same exception as the one in the start of this thread before the unreleased June fix

`Date/Time: 2025-05-26 13:08:17.7878 +0200
Launch Time: 2025-05-26 11:08:06.8614 +0200
OS Version: iPhone OS 18.5 (22F76)
Release Type: User
Report Version: 104

Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Subtype: KERN_INVALID_ADDRESS at 0x0000000100000019
Exception Codes: 0x0000000000000001, 0x0000000100000019
VM Region Info: 0x100000019 is not in any region. Bytes before following region: 36814823
REGION TYPE START - END [ VSIZE] PRT/MAX SHRMOD REGION DETAIL
UNUSED SPACE AT START
--->
__TEXT 10231c000-102aa4000 [ 7712K] r-x/r-x SM=COW /var/containers/Bundle/Application/79680FF5-2127-4119-8961-BCC57E69C6AC/FrontSystems.app/FrontSystems
Triggered by Thread: 18

Thread 18 name:
Thread 18 Crashed:
0 FrontSystems 0x00000001028dea68 mono_interp_exec_method (in FrontSystems) (interp.c:6221) + 6040168
1 FrontSystems 0x00000001028d6904 interp_runtime_invoke (in FrontSystems) (interp.c:2220) + 6007044
2 FrontSystems 0x000000010289a670 mono_jit_runtime_invoke (in FrontSystems) (mini-runtime.c:3686) + 5760624
3 FrontSystems 0x0000000102836990 mono_runtime_invoke_checked (in FrontSystems) (object.c:2788) + 5351824
4 FrontSystems 0x0000000102850814 start_wrapper (in FrontSystems) (threads.c:1276) + 5457940
5 libsystem_pthread.dylib 0x000000022abc3344 0x22abc0000 + 13124
6 libsystem_pthread.dylib 0x000000022abc0ab8 0x22abc0000 + 2744`

and with #115573 (comment)
the crash has changed a bit

`Date/Time: 2025-05-26 01:51:01.7524 +0200
Launch Time: 2025-05-25 22:15:40.3426 +0200
OS Version: iPhone OS 18.5 (22F76)
Release Type: User
Report Version: 104

Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Subtype: KERN_INVALID_ADDRESS at 0x0000000100000019
Exception Codes: 0x0000000000000001, 0x0000000100000019
VM Region Info: 0x100000019 is not in any region. Bytes before following region: 67862503
REGION TYPE START - END [ VSIZE] PRT/MAX SHRMOD REGION DETAIL
UNUSED SPACE AT START
--->
__TEXT 1040b8000-104ce0000 [ 12.2M] r-x/r-x SM=COW /var/containers/Bundle/Application/D3ED8009-61D7-434C-8FD5-E902FA864C31/FrontSystems.app/FrontSystems
Triggered by Thread: 14

Thread 14 Crashed:
0 FrontSystems 0x0000000104b20324 mono_interp_exec_method + 4040 (interp.c:6221)
1 FrontSystems 0x0000000104b1db80 interp_runtime_invoke + 236 (interp.c:2220)
2 FrontSystems 0x0000000104ae54b4 mono_jit_runtime_invoke + 1256 (mini-runtime.c:0)
3 FrontSystems 0x0000000104a88738 mono_runtime_invoke_checked + 148 (object.c:2788)
4 FrontSystems 0x0000000104a9fa8c start_wrapper + 572 (threads.c:1276)
5 libsystem_pthread.dylib 0x000000022645b344 0x226458000 + 13124
6 libsystem_pthread.dylib 0x0000000226458ab8 0x226458000 + 2744
`

[lateralusX]

@linnkrb, did you verify that you actually build against the local nuget including fix + doing a full clean obj/bin to make sure no old runtime bits are ending up being used? Possible to run the code in this test https://github.com/lateralusX/runtime/blob/97c1c3c6218c439de0ab9712c685360ec6f92f49/src/tests/FunctionalTests/iOS/Device/ParallelForEachAsync/Program.cs targeting the runtime pack with the fix? Before the fix, this triggered the exception at least a couple of times every 10:th run on iOS device.

[lateralusX]

@linnkrb, did you have this code running on .net8 or is this new code on .net9? I assume it only repro on Release build and not Debug builds as well? Does it repro when interpreter is disabled? Do you get any managed callstacks dumped to the log when hitting this issue?

[ConwayLS]

@kotlarmilos When the crash happens for us, we are selecting a chat in a chat list to view messages for that chat. This action makes async https calls out to the chat server (ACS) and our own companion API. When those async calls complete and if there are changes, property changed notifications alert the UI so that it can update. We keep selecting chats until the crash happens; it can take anywhere from 1 to 10 chats. I'm struggling to come up with an approach to create a simple app to reproduce this.

@lateralusX I know you were asking @linnkrb, but I'll give answers from our perspective. This code runs fine in .Net 8 and the issue only started happening after updating to .Net 9. We backed out the .Net 9 update. It only happens in Release builds. I've tried disabling the interpreter, but our app crashes early on during startup so I gave up. There are no managed callstacks in the log; just the mono runtime callstack. As noted in the other github issue, we get the mono_interp_exec_method callstack identified in this issue, but we also get a second crash callstack. I've copied it here for convenience:

Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Subtype: KERN_INVALID_ADDRESS at 0x0000000000000008
Exception Codes: 0x0000000000000001, 0x0000000000000008
VM Region Info: 0x8 is not in any region. Bytes before following region: 4333764600
REGION TYPE START - END [ VSIZE] PRT/MAX SHRMOD REGION DETAIL
UNUSED SPACE AT START
--->
__TEXT 102500000-102724000 [ 2192K] r-x/r-x SM=COW /var/containers/Bundle/Application/C30BE77A-E3A7-4BB6-B379-04375B978BF2/Librestream.UnifiedClient.Maui.app/Librestream.UnifiedClient.Maui
Triggered by Thread: 0

Thread 0 Crashed:
0 Librestream.UnifiedClient.Maui 0x000000010271e490 interp_optimize_code (in Librestream.UnifiedClient.Maui) (transform-opt.c:3981) + 2221200
1 Librestream.UnifiedClient.Maui 0x0000000102704b74 generate (in Librestream.UnifiedClient.Maui) (transform.c:9532) + 2116468
2 Librestream.UnifiedClient.Maui 0x0000000102704550 mono_interp_transform_method (in Librestream.UnifiedClient.Maui) (transform.c:9844) + 2114896
3 Librestream.UnifiedClient.Maui 0x0000000102722270 tier_up_method (in Librestream.UnifiedClient.Maui) (tiering.c:86) + 2237040
4 Librestream.UnifiedClient.Maui 0x0000000102721fe4 mono_interp_tier_up_frame_enter (in Librestream.UnifiedClient.Maui) (tiering.c:174) + 2236388
5 Librestream.UnifiedClient.Maui 0x00000001026f4da0 mono_interp_exec_method (in Librestream.UnifiedClient.Maui) (interp.c:7672) + 2051488
6 Librestream.UnifiedClient.Maui 0x00000001026e7880 interp_entry_from_trampoline (in Librestream.UnifiedClient.Maui) (interp.c:3229) + 1996928
7 Librestream.UnifiedClient.Maui 0x0000000102514d00 native_to_interp_trampoline (in Librestream.UnifiedClient.Maui) + 112 + 85248
8 libdispatch.dylib 0x0000000190a37584 _dispatch_client_callout + 16 (client_callout.mm:85)
9 libdispatch.dylib 0x0000000190a54574 _dispatch_main_queue_drain.cold.5 + 812 (queue.c:8104)
10 libdispatch.dylib 0x0000000190a2cd30 _dispatch_main_queue_drain + 180 (queue.c:8085)
11 libdispatch.dylib 0x0000000190a2cc6c _dispatch_main_queue_callback_4CF + 44 (queue.c:8264)
12 CoreFoundation 0x0000000188b122b4 CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE + 16 (CFRunLoop.c:1793)
13 CoreFoundation 0x0000000188b100b0 __CFRunLoopRun + 1980 (CFRunLoop.c:3163)
14 CoreFoundation 0x0000000188b34700 CFRunLoopRunSpecific + 572 (CFRunLoop.c:3434)
15 GraphicsServices 0x00000001d5675190 GSEventRunModal + 168 (GSEvent.c:2196)
16 UIKitCore 0x000000018b752240 -[UIApplication _run] + 816 (:-1)
17 UIKitCore 0x000000018b750470 UIApplicationMain + 336 (:-1)
18 Librestream.UnifiedClient.Maui 0x000000010252cf68 xamarin_UIApplicationMain (in Librestream.UnifiedClient.Maui) (bindings.m:131) + 184168
19 Librestream.UnifiedClient.Maui 0x00000001026f897c do_icall (in Librestream.UnifiedClient.Maui) (interp.c:2426) + 2066812
20 Librestream.UnifiedClient.Maui 0x00000001026f6f50 do_icall_wrapper (in Librestream.UnifiedClient.Maui) (interp.c:2467) + 2060112
21 Librestream.UnifiedClient.Maui 0x00000001026ea4c8 mono_interp_exec_method (in Librestream.UnifiedClient.Maui) (interp.c:0) + 2008264
22 Librestream.UnifiedClient.Maui 0x00000001026e8084 interp_runtime_invoke (in Librestream.UnifiedClient.Maui) (interp.c:2220) + 1998980
23 Librestream.UnifiedClient.Maui 0x00000001026b7038 mono_jit_runtime_invoke (in Librestream.UnifiedClient.Maui) (mini-runtime.c:3686) + 1798200
24 Librestream.UnifiedClient.Maui 0x0000000102662990 mono_runtime_invoke_checked (in Librestream.UnifiedClient.Maui) (object.c:2788) + 1452432
25 Librestream.UnifiedClient.Maui 0x00000001026686c4 mono_runtime_exec_main_checked (in Librestream.UnifiedClient.Maui) (object.c:4808) + 1476292
26 Librestream.UnifiedClient.Maui 0x00000001026bd8e4 mono_jit_exec (in Librestream.UnifiedClient.Maui) (driver.c:1311) + 1824996
27 Librestream.UnifiedClient.Maui 0x0000000102537b38 xamarin_main (in Librestream.UnifiedClient.Maui) (monotouch-main.m:0) + 228152
28 Librestream.UnifiedClient.Maui 0x0000000102727540 main (in Librestream.UnifiedClient.Maui) (main.arm64.mm:79) + 2258240
29 dyld 0x00000001af537ad8 start + 5964 (dyldMain.cpp:1443)