Repository navigation
Conversation
Connect CUDA and Metal pockets, native causal pullbacks and shared persistence with bounded admission and completion-safe ownership. Add recording-device, lifecycle, save and precision contracts, with explicit raw directional diagnostics and pending device qualification.
Keep the public alias, record order and existing resource-refusal test unchanged.
3 of 10 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wire the native Qwen4 dense pocket and learner on CUDA and Metal through the shared Living Weights family,
fitting policy, shard writer and save guard. Both native adapters feed a bounded native trainer from the serving
forward's recorded HC/GDN/QSA nodes and selected IDs. Empty and untaught pockets encode no additional GPU work;
appended checkpoint neurons are loaded separately while original packed rows and columns remain exact.
Experimental draft for #1804. Public
--slideremains refused by name onqwen4andqwen4_texton CUDA and Metal until all six raw BF16 gradient cells pass and device qualification succeeds. Depends on the host selected-attention backward in #2021. The internal experimental writable layout is
MLX affine4/group64 MLPs, BF16 or affine4/group64 other training projections, 2 through 64 layers, and no PLE.
PLE backward and unsupported quantization/layouts refuse explicitly. Other family adapters retain their refusals.
Host validation includes actual native launch wrappers with recording devices, nonzero native forwards,
untaught-stream isolation, failed-completion retention, atomic pocket replacement, shared learner refusal,
shared save failure/retry/reopen, independent GDN/QSA and full-graph frozen-store finite differences, accumulator
witnesses and seven real source mutations. The final focused host/contract graph passes 293 tests; admission
checks pass 16, and the separate host/shared/golden/fuzz graph passes 166 with zero unexpected golden differences.
The plain Linux cross-build passes. Lean adds zero problems over the pinned 424 inherited findings.
The raw BF16 whole-graph rule-v2 panel failed five of six tiny synthetic cells, with unchanged selected IDs.
Those failures are retained and reproducible with
zig build diagnose-qwen4-raw-gradient-host -j1. The passing frozen-store derivative checks do not resolve or replace them.Successful real-model keep/recall/removal, raw gradient qualification, cross-backend saves, off-path tokens/speed,
CUDA compiler/device limits and complete Metal numeric/resource qualification are still owed. The full
device-bearing standard test graph is also owed on this tree. No support or performance qualification
is inferred from compilation, recording devices, or the ideal host attention formula.
The native four-kernel selected-attention chain now agrees with the independent host backward from #2021
at 17 causal rows, 24 query heads, four KV heads and width 256, including mixed sparse/dense selections
and early key/value gradients from answer-only loss. This comparison uses continuous FP32 softmax probabilities;
it does not qualify CUDA probability rounding. The off-path dense input store is gated on an active learner,
and the primitive contract callback table has static storage. The follow-up host graph passes 175 tests in 47 steps; three planted source controls fail and restore,
and lean adds zero findings. No device-bearing full-graph rerun applies to this follow-up tree.