Repository navigation
fix(llama-cpp): keep decision heads on CPU for CPU placement - #1176
Merged
Merged
Conversation
…x/015-o-decision-head-cpu # Conflicts: # CHANGELOG.md # website/docs/changelog/recent-releases.md
Contributor
|
Chat app preview removed for |
12 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
splitMode: ModelSplitMode.noneand a negativemainGpuselect no GPU; reportCPUthroughDecisionModelInfo.deviceName.Production-readiness scope
DecisionEngine; deterministic fake GPU registry and synthetic native ModernBERT/head checks on macOS. Existing native experimental support levels remain unchanged.LlamaUnsupportedException; explicitComputeDevice.gpuplus CPU-only placement keeps its existing validation error. Web placement is unchanged. Physical iOS/simulator GPU execution was not qualified here.Completeness checklist
loaded.info.deviceNameand needs no code change.PR type guidance
expected null; got 12before the fix. Both head CPU detection and GPU-device selection now consultselectsNoGpuDevice. Native-only affected path; Web and LiteRT-LM unchanged.Test Plan
dart run tool/prepare_workspace.dart(clean checkout; no tracked generated changes).dart format --output=none --set-exit-if-changed .(1,140 files; 0 changed).dart analyze(no issues, including zero infos inlib/).dart test -p vm -j 1 --exclude-tags local-only --reporter expanded— 08:10 +6987 ~124: All tests passed!; full retry outside sandbox after two unchanged process-liveness tests were deniedps.dart test -p chrome --exclude-tags local-only --reporter expanded— 01:38 +3332: All tests passed!; sandboxed Chrome aborted on launch, and the unchanged full command passed outside the sandbox../tool/docs/build_site.sh --port 43839passed with 2,292 pages and no broken internal links.Matrix Evidence
static-format-analyzeroot-vmGGML_METAL_DEVICES=0; existing CI model directoryroot-chromehigh-risk-exact-head-independent-qa3b1a5129546cfb93d72165d6ed0b33a43d8a9c4cagainst591dfd844: no P1/P2. Real Metal probe: none/-1 and none/-2 heads now run on CPU with confidence identical to pure CPU; explicit mainGpu and layer split stay on MTL0. Full VM +6990 ~124 (GGML_METAL_DEVICES=0); decision suites with Metal visible +928; decision-model-smoke E2E passes on Metal and CPU; 8/10 mutations killed (2 survivors are redundant guards, P3).docs-sitevalidate_links.shomitted per task.decision-model-smokeReview Notes
3b1a5129546cfb93d72165d6ed0b33a43d8a9c4cagainst591dfd844: no P1/P2. Real Metal probe: none/-1 and none/-2 heads now run on CPU with confidence identical to pure CPU; explicit mainGpu and layer split stay on MTL0. Full VM +6990 ~124 (GGML_METAL_DEVICES=0); decision suites with Metal visible +928; decision-model-smoke E2E passes on Metal and CPU; 8/10 mutations killed (2 survivors are redundant guards, P3).3b1a5129546cfb93d72165d6ed0b33a43d8a9c4c (d515a65 + merge of main); current localorigin/mainis591dfd844005b7582ad25e4018d9db88ed68f6ba.High-risk regression review
backendRuntime.codex/015-o-decision-head-cpu.3b1a5129546cfb93d72165d6ed0b33a43d8a9c4c(fresh auditor).3b1a5129546cfb93d72165d6ed0b33a43d8a9c4c (d515a65 + merge of main)/591dfd844005b7582ad25e4018d9db88ed68f6ba. Tested branch base isc5dc87136a73965c427599d1438548478d4bf15b; current main advanced during validation. Integrate with main and rerun checks before independent audit; refresh the remote-tracking base then.decisionHeadRunsOnCpuplacement condition -> 2 failures; remove thedecisionHeadDeviceIndexguard -> fake-registry failure; omit forwardingsplitMode-> fake-registry failure. All run with GPU devices disabled, all restored, and 22 regression cases then passed. Independent assessment PENDING.CPU; fake registry contains Vulkan and two Metal devices. These are deterministic routing artifacts, not real Laya weights or device numerical qualification.