Skip to content

browser.test_audioworklet_worker stack overflow causes atexit() handlers to be run twice #26823

Closed

Description

@juj

A timing-related issue that occurs on my CI currently only on latest Firefox on Apple Silicon macOS.

test/runner browser64.test_audio_worklet_worker

hangs the browser page on a futex wait.

The slow script dialog abort callstack looks like this:

Image
  1. The application starts, and creates a Wasm Worker with emscripten_create_wasm_worker().
  2. The Worker calls emscripten_force_exit(), which postMessages the exit call back to the main thread. Additionally, emscripten_force_exit() calls to exitJS, which leads to calling global atexit() functions in libc ___funcs_on_exit in the Worker.
  3. The message is received and processed via _wasmWorkerRunPostMessage on main thread.
  4. $_wasmWorkerRunPostMessage calls to callUserCallback().
  5. callUserCallback() has a finally block to check for maybeExit()
  6. maybeExit() thinks that (as response to having received a postMessage from Wasm Worker), it is time to quit the application, and calls exitJS()
  7. exitJS() calls _funcs_on_exit on the main thread.
  8. which locks up in the __funcs_on_exit() function, since that function has already been called.

If I add any additional EM_ASM(console.log())s, or printf()s into the program, the hang goes away, since timing is disrupted.

Here is a callstack dump showing how both the Worker and the main thread are calling libc ___funcs_on_exit, the second one hanging, because the current semantics are to keep the ___funcs_on_exit lock indefinitely.

Image

Activity

  1. juj commented on Apr 30, 2026

    @juj
    CollaboratorAuthor

    A naive attempt to fix might be to release the lock at

    diff --git a/system/lib/libc/musl/src/exit/atexit.c b/system/lib/libc/musl/src/exit/atexit.c
    index f2401780b3..1006906c4c 100644
    --- a/system/lib/libc/musl/src/exit/atexit.c
    +++ b/system/lib/libc/musl/src/exit/atexit.c
    @@ -34,6 +34,7 @@ void __funcs_on_exit()
                    func(arg);
                    LOCK(lock);
            }
    +       UNLOCK(lock);
     }
    

    but thinking about it, that would be racy, and violate the ordering guarantee.

    Image

    https://man7.org/linux/man-pages/man3/atexit.3.html

    Worse, we currently run the atexit handlers while other threads may still be running.

  2. sbc100 commented on Apr 30, 2026

    @sbc100
    Collaborator

    I think that analysis in (2) is slightly off.

    Wasm Workers cannot themselves ever call emscripten_force_exit because of emscripten_force_exit__proxy: 'sync'.
    So what happens here is the worker sends the do_exit function back to the main thread using emscripten_wasm_worker_post_function_v(EMSCRIPTEN_WASM_WORKER_ID_PARENT, &do_exit);.

    I guess the result in similar but maybe worth mentioning.

  3. sbc100 commented on Apr 30, 2026

    @sbc100
    Collaborator

    Also note that we already have an open bug for not running atexit handlers on the correct thread.

    Its not necessarily directly related here, but atexit handler are always supposed to run on the thread that actually calls exit. Instead, because of our proxying system we always run the atexit handlers on the main thread even when its one of the pthreads that calls exit: #26040. I have yet to find a reasonable solution for this.

  4. juj commented on Apr 30, 2026

    @juj
    CollaboratorAuthor

    Yeah, you're right. I see that runtime exit is being called recursively on the main thread. It re-enters, but I can't quite yet fathom why it re-enters.

    When debugging this in Firefox profiler, I am met with bugs in the debugger UI.. Firefox is unable to correctly show wasm frames at times, and it jumps to wrong lines in the JS view, making this kinda hairy to follow. (plus I am doing this over a remote desktop connection, making it extra tricky to navigate). I'll keep prodding.

  5. added a commit that references this issue on Apr 30, 2026
  6. juj commented on May 1, 2026

    @juj
    CollaboratorAuthor

    Ok, found the root cause.

    The test browser64.test_audio_worklet_worker is running with a tiny 1KB stack for Wasm Workers, but that is not enough for the Wasm Worker to hold stack space to call emscripten_outf(). the vsnprintf code overruns its stack space over to the libc atexit() internal data structures, writing garbage to the linked list data structure at

    static struct fl
    {
    struct fl *next;
    void (*f[COUNT])(void *);
    void *a[COUNT];
    } builtin, *head;

    This causes a Wasm RuntimeError: index out of bounds from within executing __funcs_on_exit(), which would reach finally block in

    maybeExit();
    and re-enter calling back to the atexit() handlers in __funcs_on_exit().

    __funcs_on_exit() has a LOCK() that is not recursive, so the test would then hang on attempting to acquire the taken lock.

    Posted PR #26835 to fix.

  7. added 2 commits that reference this issue on May 1, 2026
    29f35f4
    0d03fd8
  8. changed the title [-]Global atexit dtors are called in multiple threads in MT builds[/-] [+]browser.test_audioworklet_worker stack overflow causes atexit() handlers to be run twice[/+] on May 2, 2026
  9. juj commented on May 2, 2026

    @juj
    CollaboratorAuthor

    Closing as fixed.

  10. sbc100 commented on May 3, 2026

    @sbc100
    Collaborator

    So IIUC the "Exception during __funcs_on_exit causes infinite loop" issue is still present? But we are not triggering it anymore in this test?

  11. juj commented on May 4, 2026

    @juj
    CollaboratorAuthor

    The root cause that caused the re-entrancy in browser.test_audio_worklet_worker was a too small stack, so that vsnprintf in emscripten_outf would overwrite the internal atexit linked list, which would then cause the exception.

    So that exception source is gone.

    atexit handlers should not throw exceptions per spec. The new re-entrancy guard will ensure that debug builds will catch such a scenario.

    If we wanted to detect this in release builds, that would mean to e.g. create a test that throw std::exception()s or __builtin_trap()s in an atexit() handler (or call out to JS to throw a JS exception), to see what happens.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions