Repository navigation
perf(gdn): optimize post-conv and causal-conv kernels - #155
Merged
localai-bot merged 6 commits intoAug 11, 2026
Merged
Conversation
mudler
added a commit
that referenced
this pull request
Aug 8, 2026
…re-push sandbox (#159) Two guards on main were RED and between them blocked every open external contributor PR (#127, #154, #155) and every push. Both premises were verified in the tree before changing anything. 1) check-device-leakage: src/vllm/v1/worker/gpu/runner.cpp named vt::DeviceType::kCUDA in the device-agnostic shared layer (DSR bucket 'kcuda' 1 > baseline 0). It came in with the QueueSupportsAsyncInputCombine rescope during the PR #140 fix round - ours, not the contributors'; richiejp reported it in #127's honest gaps. Fixed the way the guard's own message prescribes, mirroring the SupportsAuxStream precedent: ask the backend, not the device. New vt::Backend::SupportsAsyncSampledTokenReadback() (base false) answers whether the host may validly read the sampled token id back between steps; CPU overrides true (host and device memory are one allocation) and CUDA overrides true (the id is device-mirrored). The runner asks vt::TryGetBackend(queue.device.type), whose nullptr for a device absent from the build also subsumes the old #ifdef VLLM_CPP_CUDA guard. SEMANTICS UNCHANGED: CPU async-ON, CUDA async-ON, discrete non-CUDA (ROCm gfx1201) async-OFF - the "!"-token hazard stays closed. 2) .githooks/pre-push ran check-policy.py inside a PARTIAL export (README.md docs scripts .agents), but policy_contract.py:428 asserts AGENTS.md is a non-symlink regular file and resolves its Markdown links against that sandbox. AGENTS.md and its .env.example link were both missing, so the hook failed closed on content that is fine in the real tree - every push refused. EXPORT_PATHS is now a superset of what the CHECKERS read. Gates: check-device-leakage RED->GREEN (kcuda=0, DSR 32 == baseline 32); all four hook checkers OK in the reproduced sandbox; test_async_llm 8/8-347, test_engine_core 6/6-44, test_llm_engine 11/11-204 (CPU still resolves async-ON); clean -Werror CPU build; full 11-gate record battery green. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude (Opus 5) via Claude Code
richiejp
force-pushed
the
row/KERNEL-SSM-MAMBA-POSTCONV-TILE
branch
3 times, most recently
from
August 10, 2026 13:56
b3f019a to
a651838
Compare
Select the measured sm_120 post-conv residual, reject the existing split grid, and bind the upstream 16-token CUDA experiment and gates. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Codex:gpt-5.6-sol [Codex]
Port the upstream 16-token per-head schedule behind an opt-in flag, preserve the existing reduction order byte-exactly, and record the sm_120 A/B evidence. Row: row/KERNEL-SSM-MAMBA-POSTCONV-TILE FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Codex:gpt-5.6-sol [Codex]
FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Codex:gpt-5.6-sol [Codex]
FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Codex:gpt-5.6-sol [Codex]
Route runtime-width and both K=4 CUDA launch arms through the same portable, mutation-tested callback dispatcher. Pin strict arm-2 parsing and resolved non-K4 fallback semantics. row/KERNEL-SSM-MAMBA-POSTCONV-TILE FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Codex:gpt-5.6-sol [Codex]
FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Codex:gpt-5.6-sol [Codex]
richiejp
force-pushed
the
row/KERNEL-SSM-MAMBA-POSTCONV-TILE
branch
from
August 10, 2026 14:12
a651838 to
7c8ee10
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Row
KERNEL-SSM-MAMBA— Qwen GDN post-conv and causal-conv kernel sublevers; one row per PR.Before starting
scripts/ready-for-helper.pyresult when applicable:.agents/kernel-matrix.md:157(KERNEL-SSM-MAMBA,INVENTORIED), feedingROAD-V1-C2-LOCAL-BF16. Spikes:.agents/specs/sm120-qwen35-postconv-token-tile-2026-08-08.mdand.agents/specs/sm120-qwen35-conv-channel-tile-2026-08-08.md.src/vt/cuda/cuda_gdn.cu,src/vt/cuda/gdn_prefill_conv.h,tests/vt/test_gdn_prefill_conv.cpp,tests/vt/test_ops_gdn.cpp, and pinned vLLM/FLA causal-conv plus fused post-conv kernels recorded withfile:lineanchors in the two spikes.What changed
Add two independently selectable, default-off CUDA specializations.
VT_GDN_POSTCONV_TOKEN_TILE=1ports the upstream-shaped 16-token/four-warp post-conv schedule while preserving the existing 128-lane reduction tree byte-for-byte.VT_CONV_CHANNEL_TILE=1specializes the causal-conv path for width four at the existing one-channel-per-thread grid; arm=2is retained as the measured 256-channel/two-channels-per-thread falsification arm. Production dispatch and the portable mutation gate share one callback dispatcher, so the tests prove that each selected arm is actually invoked.Evidence
scripts/agent-preflight.shpasses after rebasing onto canonicalupstream/mainc05cee1d2; all record gates, mutation suites, live-row audit, and committed-range document checks are green.vllm-benchrelink; portable dispatch 9/9 cases, 88/88 assertions; CUDA GDN 67/67 cases, 4631/4631 assertions; Qwen3.5 paged-forward 4/4 cases, 8/8 assertions. The accepted production arms produce identical token hashes.docs/STATUS.mdanddocs/BENCHMARKS.mdupdated; the model/feature surface did not expand. Full profiler evidence and the void stale-link series are retained in the two specs and.agents/benchmark-record.md.Speed claims
${GPU_LOCK}and they are recorded indocs/BENCHMARKS.mdwith the repro recipe.On the local RTX 5070 Ti / Qwen3.5-4B c32 workload, the post-conv tile reduces graph-node time 227.887 to 122.587 ms (1.859x) and improves every observed enclosing axis, including total/output throughput by 0.532%. The causal-conv K4 arm reduces graph-node time 234.605 to 219.506 ms (6.44%) and improves whole-run total/output throughput by about 0.12%. The 256-channel arm is 4.05% slower than the K4 arm and is recorded as falsified.
Honest gaps
vllm-benchhad not relinked the rebuilt CUDA library. The accepted series explicitly relinked the benchmark and trace-proved the specialized kernels.mainc05cee1d2independently failscheck-device-leakage.py:src/vllm/v1/worker/gpu/runner.cppcontains one new hard-codedDeviceType::kCUDA(DSRkcuda1 > baseline 0). perf(gdn): dispatch exact causal-conv chunks #127 and this PR inherit that merged-tree failure; neither kernel diff introduced it.