Repository navigation
cuda : fold cuBLAS conversion into tensor affine activation - #136
Draft
GenerelSchwerz wants to merge 10 commits into
Draft
GenerelSchwerz wants to merge 10 commits into
GenerelSchwerz wants to merge 10 commits into
Conversation
Carry the reviewed kernel sources from PR125, PR129 and PR132 as the matching control for the separate tensor-affine HC follow-up. Assisted-by: Codex
Preserve gate intermediates, original rounding and live HC outputs. Retain strict input alias checks and standard allocation dependencies. Assisted-by: Codex
Use the exact tested control sources from published PR130, PR129 and PR132. Assisted-by: Codex
Reuse the allocated affine match to find the planned HC_POST and retain prepared emission priority. Reject expanded normalization cuts whose terminal is not written by this kernel. Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fold tensor MUL -> ADD -> SIGMOID/SiLU -> optional SCALE into the existing typed cuBLAS output conversion. This covers a complete F32 result or a canonical column window repeated across every row and batch, retaining the full raw matrix and all live stages.
The original vendor/precision selection, strict input ownership checks, allocation dependencies and earlier HC/GLU/topk priority stay in place. Other windows remain available through the raw result. Unsupported layouts, intervening compute, direct-F32 output and native/MMQ/MMVQ paths keep their existing execution.
Component measurements
Six alternating pairs of 2000 graph replays on RTX 5070 Ti. These are separate comparisons; percentages are not additive.
6c02602660e93df94da2df30BF16/intervening/whole/native control paths are flat in the window comparison. Whole/native measure 0.019% / 0.127% more time, each faster in 3/6 pairs. No model-serving, MTP acceptance, private-executor or older-GPU runtime speedup is claimed.
Validation
test-backend-ops -b CUDA0 -o DSV4_HC_PRE,MUL_MAT_UNARYwith each qualified library directory.Earlier whole-result qualification remains recorded separately: 21 matched cases, 51/51 stock tests per arm, eight sanitizer jobs and older compile checks. The extension preserves that converter and validates its unchanged execution again. Raw scripts, patches, hashes and records are under
/home/gencoolpc/moe-cache-tests/results/upstream-kernel-series-20261001/HYPERCONNECTION/PACKED-CUBLAS-AFFINE-ASSESSMENT; earlier whole-result records are in siblingCUBLAS-AFFINE-CONVERSION.This PR depends on PR135 and its listed HC/scalar prerequisites, with official
reference/upstream-kernels-20261001as review base. It folds one eligible window at conversion; it does not fold arbitrary later branches or change model-specific upstream operator limits.AI assistance: Codex implemented and tested the changes; bounded independent GPT-6.1 Sol High review checked source and evidence. The owner handles upstream submission.