Skip to content

Qualify Intel Mac AMD Metal output correctness and fail-safe selection #116

Open

Description

@leehack

Triage

P2 owner investigation of an upstream-reported correctness defect on a shipped-source platform. This is not yet a reproduced llamadart-native defect and no PR-caused regression is alleged. If the published owner path reproduces silent corruption, reassess as P1 correctness; lack of local hardware must not downgrade a confirmed owner P1.

Evidence / exact layers

  • Upstream llama.cpp #30702 reports deterministic garbled Qwen2.5-7B Q4_K_M output on Intel i9 / AMD Radeon Pro 5500M at 620d2225cc436ab0b001e3200e590e3b286ef9ab (v0.7.0). Same binary/model with GPU layers 0 produces correct text. Reporter links older #19563; no introducing commit is established.
  • Latest native v0.7.0 manifest bundles that exact upstream plus three recorded patches, none documented as an AMD Metal repair. Native commit e0454f824cfd9d883ed536e92a08c78c2bd604ee; workflow head 5899378c304fcb497021b576da07557dce07f778.
  • Owner macos-x86_64-full preset enables Metal. This establishes an exposure needing qualification, not reproduction against our bundle.
  • Core 0.14.0 adoption consumes v0.7.0, but its local runtime evidence is Apple Silicon, not Intel/AMD hardware. Web assets use WASM and are not automatically implicated.

Acceptance / bounded next action

  • Audit native/consumer device selection and existing guards; reproduce with the exact published macOS x86_64 archive on Intel/AMD hardware, exact model hash/prompt/seed, comparing Metal and explicit CPU.
  • Record actual backend/device, nonempty output and semantic/reference correctness; successful load/export/build is insufficient.
  • Determine whether older published owner versions also fail before claiming a regression or first-bad PR.
  • If affected, implement owner-level fail-closed unsupported-GPU detection or a narrow audited repair; do not silently report successful GPU work or silently alter requested placement.
  • Test supported Apple Silicon Metal and CPU controls, unsupported/version-skew diagnostics, device enumeration/selection and consumer capability truth. Link consumer work only if its routing/docs need change.
  • Document safe CPU selection and unsupported hardware precisely. Preserve native Investigate Galaxy S24 Vulkan crash during first GGUF generation #79 and SD-native Disable MSVC IPO for llama-common #19 as separate defects.

No hardware run, upstream issue mutation, source change, publication or downstream pin change is authorized by this ticket.

Activity

  1. added
    bugSomething isn't working
    priority:P2Important improvement or regression; schedule next
    on Oct 17, 2026
  2. leehack commented on Oct 29, 2026

    @leehack
    OwnerAuthor

    Upstream status (2026-10-29)

    Not fixed upstream, and not in v0.8.0.

    • llama.cpp #30702 is open (bug-unconfirmed), with no comments or linked PR.
    • Its predecessor #19563 and its fix PR #19567 (disable concurrent dispatch on non-UMA devices) were closed as stale; the fix never merged. #25866 was also closed as stale.
    • The only change in flight is #28568 (disable concurrent dispatch on non-Apple GPU families). It is open and unreviewed.
    • Between v0.7.0 620d2225 and v0.8.0 8642541b, 9 commits touch ggml/src/ggml-metal/. None of them gates concurrency, GPU family, or the non-unified-memory buffer path.
    • Upstream's own macOS x64 release builds use -DGGML_METAL=OFF.
    • Reports disagree on GGML_METAL_CONCURRENCY_DISABLE=1: it fixed a 5300M, but a 5500M still failed with it set.

    Our exposure

    macos-x86_64-full has shipped with GGML_METAL=ON since v0.3.0, so this is long-standing, not a v0.7.0 regression. No carried patch touches Metal. llamadart's defaults offload every layer to Metal on an Intel Mac with an AMD GPU.

    Plan

    This stays open for hardware qualification, blocked on Intel/AMD hardware. The fail-safe without hardware is to route GpuBackend.auto to the CPU on macOS x86_64, while still honouring an explicit Metal request with a warning. That is proposed in leehack/llamadart#1597. A later native probe (MTLGPUFamilyApple7 / hasUnifiedMemory) could make the rule precise. Revisit the default if upstream #28568 or a family gate lands and is qualified.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingpriority:P2Important improvement or regression; schedule next

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions