Skip to content

Windows/MSYS2-gcc15: self-built main hangs at scale; self-built v0.11.0 crashes (mimalloc corrupted free list) — released binary is fine #2606

Open

Description

@vinay-veerappa

Summary

Two build-related reliability problems observed on this Windows box (32 GB RAM, MSYS2 UCRT gcc 15.2.0, Windows 11 x64), neither reproducible with the released 0.11.0 binary on the same machine:

  1. The current main (4d9592b, after the v0.11.0 tag) hangs during indexing at scale. On a 1,820-file mixed workspace, v0.11.0 completes a fast index in 11 seconds; main (self-built, same flags below) was abandoned after 30 minutes with the last worker log ending mid-run. On a 370-file Rust-only tree the hang reproduces quickly:

    level=info msg=pass.start pass=structure files=307
    level=info msg=pass.done pass=structure nodes=369 edges=367
    <no further progress; worker eventually marked outcome=hang>
    

    and the supervisor quarantines files with phase=hang_reason=quarantined on unrelated file types (logs/kb_test/narrative_with_kb_close_ES1.md, web/openalgo-chart/src/components/Topbar/Topbar.module.css, a .py inside crates/), restarting repeatedly (118 worker_failed records over ~90 minutes).

  2. v0.11.0 self-built with MSYS2 UCRT gcc 15 crashes at large scale. The released 0.11.0 binary (CI toolchain) indexes the same 1,820-file workspace in 11 s; a self-built v0.11.0 (tag 8972ea6) crashes deterministically (~5 min in, during parallel.resolve, right after cross_lsp.rust_registry scale=types=6559_funcs=20787):

    mimalloc: error: thread 0x71B0AE5000: corrupted free list entry of size 16b at 0x01DD0E220010
    exit code -1073740791 (0xC0000409 STATUS_STACK_BUFFER_OVERRUN)
    

    Reproduced on 3 projects (1,440–33k nodes); small projects (12–71 s) index fine. The corruption is heap-level (a 16-byte free-list entry), so this smells like a latent UB/gcclib difference — quite possibly in the vendored allocator override interplay (-Wl,--wrap=malloc + mimalloc) rather than in the graph code, and quite possibly also the cause of (1) if HEAD grows similar pressure.

Build command used (only deviation from release: -Wno-error instead of -Werror, MSYS2 UCRT gcc 15.2):

make -j8 -f Makefile.cbm cbm BUILD_DIR=build/c-mingw \
  CFLAGS_COMMON="-std=c11 -D_DEFAULT_SOURCE -D_GNU_SOURCE -Wall -Wno-error -Wdate-time \
  -Wno-unused-parameter -Wno-sign-compare -Wno-format-truncation -Wno-unused-result"

Also: internal/cbm/doc_pdf.c:1337 uses %zu in cbm_arena_sprintf, which fails -Werror=format under MSYS2 (unknown conversion type character 'z') — either the format attribute disagrees with the actual printf backend on MinGW, or the arena printf needs %zu support. A -Wno-format build passes.

Happy to test candidate patches (I can build both debug and release variants here).

Activity

  1. DeusData commented on Oct 19, 2026

    @DeusData
    Owner

    Thank you for comparing released and locally built binaries on the same machine and for documenting the toolchain and flags. The hang, allocator-corruption report, and format warning need separate verification; the shared root cause is not established yet. There is no verified patch to ask you to test at this point.

  2. DeusData commented on Oct 25, 2026

    @DeusData
    Owner

    Thank you again for the released-versus-self-built comparison. The hang, allocator report and format warning still need separate attribution rather than a shared-root-cause assumption. Sorry for the delay; I’ll follow up soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingstability/performanceServer crashes, OOM, hangs, high CPU/memorywindowsWindows-specific issues

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions