Repository navigation
Conversation
Admission reads EXL3 mcg checkpoints at 1 to 8 bits: "bits" may be absent (a mixed checkpoint gives "avg_bits"), and a given one must lie in range. The loader takes each trellis's width from its last dimension, slices TP shares by that width's tile bytes, places each expert's trellis at its own offset and uploads a width table a projection (one table, as before, when the three agree). Layer gets uniform_k2_gu and uniform_k2_d; uniform_k2 stays nonzero only when every projection agrees, so a 4-bit layer uploads upstream's bytes and keeps the prompt kernels. Scratch is sized by the path each layer takes: the prompt kernels' when they take it, else the grouped path's, the buffers taking the largest over every layer and the MTP layer (a mixed layer's prompt chunks run grouped). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015bV57ikYD5vn1va1KmaPjw
glm-exl3 runs Mia-AiLab's 4-bit experts, 3-bit gate and up over 4-bit down, and 2/3/4-bit gate/up/down, each projection on its own width table, decode and prompt chunks (1030 rows, two calls) against float64, rows as alone, NaN kept to its row. glm-exl3-layer <ckpt> <layer> <rank> <world> loads one checkpoint layer through cuda_exl3.load at a rank's TP share and runs it (decode windows and a prompt chunk) against float64 matrices decoded from the whole trellises (core's unpack, cut, rotated, scaled), with a planted control (gate and up swapped) that must fail the same tolerance. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015bV57ikYD5vn1va1KmaPjw
… by projection group tf_exl3_prompt_gu_k6 and tf_exl3_prompt_down_k6 run the transposed prompt kernels on 96-byte (3-bit) mcg tiles, with tf_exl3_prompt_gu/_down's arguments. The kernels are templated on the half-bit width; the 4-bit entry points keep their SASS byte for byte on sm_80/86/89/120/121 (nvcc 13.3). The down kernel reads its item in the kernel itself so that its launch bound still limits the thread arithmetic. Layer gains uniform_k2_gu and uniform_k2_d (the loader's per-group verified widths, uniform_k2 when zero). The prompt path takes a layer whose gate/up width and down width are each 6 or 8 and launches each group's own kernel; widths that disagree with uniform_k2 stay unverified and keep the grouped path. Scratch is width-independent. Contract: both launches registered pending nvcc 13.0's receipts, with nvcc 13.3's five-SM usage measured by kernel-contract-limits. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015bV57ikYD5vn1va1KmaPjw
tf-cuda-test exl3-prompt-k6 [DIR|none] [BASE]: one synthetic GLM layer (4096 x 1024 per rank, 288 experts, top-8) at every gate/up and down width pair through the prompt kernels against runSlots and against float64 (glm_exl3_tests' tolerance) at 1, 64, 1000 and 4096 rows; a row's bytes in one call, in 1024-row calls and with a rotation a pick; BASE, an earlier exl3_experts fatbin, must give the 4-bit kernels' bytes; then one layer's time. DIR adds layer 4 of a GLMQ checkpoint (rank 0 of 2) the same way. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015bV57ikYD5vn1va1KmaPjw
|
The #2031 native Linux sweep on main Since this PR updates the same fixture, could you account for that field when bringing it onto current main? Draft CI-coverage PR #2095 keeps this check included and records the compile failure as a prerequisite. Its paired explicit-step/aggregate runs reproduce the same failure. I've kept the EXL3 fixture repair with this work rather than adding a competing patch. |
|
Thanks @MiaAI-Lab, the host side reads well: widths come from the trellis shape, admission still refuses by name, and the real-layer float64 check has its own planted control. To land it on the terms we set on #2093, a few things are still owed:
On our side, we'll check on our GB10s that the pinned 4-bit checkpoint's replies equal main's, with decode level. |
Refs #2093.
GLM's CUDA family admits only 4-bit EXL3
mcgrouted experts; anything else iserror.NotExl3Mcg4. This takes each tensor's half-bit width from its trellis shape (whole bits 1..8, mixed per projection inside one layer), templates the decode and prompt expert kernels on it, and keeps the 4-bit path exactly today's: the upstream 4-bit loader runs verbatim in a test and must upload equal bytes; the 4-bit entry points keep their SASS at nvcc 13.3.Four commits, one concern each:
glm cuda: EXL3 mcg experts at any whole-bit width, mixed by projection- admission keeps the codebook check and verifies each width's range; the loader reads widths from the trellises and holds them per projection; scratch stays width-independent.tests: GLM EXL3 at mixed widths, synthetic and a checkpoint's own layers- synthetic mixed-width rigs andglm-exl3-layer, one real checkpoint layer's experts against float64 unpacks.exl3 cuda: the common prompt kernels take 3-bit mcg gate/up and down, by projection group-tf_exl3_prompt_gu_k6/tf_exl3_prompt_down_k6; contract records with 13.0 receipts pending and 13.3's measured usage.tests: exl3-prompt-k6, the 3- and 4-bit prompt kernels at GLM's shapes- the k6 pair against the 4-bit pair's references, edges and shapes.Gates so far (host, no GPU claims):
zig build installgreen with-Dnvcc -Dsm=80,86,89,120,121(nvcc 13.3.1, Zig 0.17.0 per.zig-version);test-cuda-host4705/4708 with 2 fails indrama_voice_tool.zig, untouched by this branch (pre-exist onmain's tree here);tools/zig/lean_check.pyzero findings overmain's baseline count;zig fmtclean. Kernel-contract and per-SM launch receipts on sm_120 hardware post below when the run completes; the 3-bit launches' 13.0 resources stay pending records per the registry's convention, 13.3 measured.Metal, XPU, other codebooks and other families: untouched.